2024-04-24T08:57:02.092 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-24T08:57:02.092 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671225 --verbose --name rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/suites/fsx}} --owner scheduled_rishabh@teuthology -- /home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671225/orig.config.yaml 2024-04-24T08:57:02.145 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-24T08:57:02.237 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671225 branch: rishabh-main-apr17-a654945 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/suites/fsx}} email: null first_in_suite: false job_id: '7671225' kernel: client: branch: testing flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: rishabh-main-apr17-a654945 ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: 1f81fb397ae98da7563d451a78c61574c8f4e6e0 subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.dir_split install: ceph: flavor: default sha1: 1f81fb397ae98da7563d451a78c61574c8f4e6e0 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: rishabh-main-apr17-a654945 sha1: 1f81fb397ae98da7563d451a78c61574c8f4e6e0 owner: scheduled_rishabh@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 7954 sha1: 1f81fb397ae98da7563d451a78c61574c8f4e6e0 sleep_before_teardown: 0 subset: 364/512 suite: fs suite_branch: rishabh-main-apr17-a654945 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1f81fb397ae98da7563d451a78c61574c8f4e6e0/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1f81fb397ae98da7563d451a78c61574c8f4e6e0 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi090.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK1UBMLD+nRVl0oMMWDQf2IwkCKv5zuhPfJbc1j1JbUDzNNDUJjbducopq88qDIxujCiscRPaSnx8IN4vtqh3/M= smithi114.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH98UXW7tWWhJR9vA8aAMTU0MvISAGp74/oo/ZsBqHkECDNHSaUieKk9zatMsSgsDznu90xl8L2kToTnbF6DWIM= smithi149.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB3WggCYnS3qkyggvta3gVJxUZVLDSdh1fIUvPO8pAGA+AMRKJwQy6xS1kxpVX6ZvddQQJxCECFayXG71dns0+M= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - workunit: clients: all: - suites/fsx.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-24_07:32:23 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-24T08:57:02.237 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1f81fb397ae98da7563d451a78c61574c8f4e6e0/qa; will attempt to use it 2024-04-24T08:57:02.238 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1f81fb397ae98da7563d451a78c61574c8f4e6e0/qa/tasks 2024-04-24T08:57:02.238 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-24T08:57:02.241 INFO:teuthology.task.internal:Checking packages... 2024-04-24T08:57:02.264 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1f81fb397ae98da7563d451a78c61574c8f4e6e0' 2024-04-24T08:57:02.265 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-24T08:57:02.265 INFO:teuthology.packaging:ref: None 2024-04-24T08:57:02.265 INFO:teuthology.packaging:tag: None 2024-04-24T08:57:02.265 INFO:teuthology.packaging:branch: rishabh-main-apr17-a654945 2024-04-24T08:57:02.265 INFO:teuthology.packaging:sha1: 1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T08:57:02.265 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=rishabh-main-apr17-a654945 2024-04-24T08:57:02.492 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3116.g1f81fb39 2024-04-24T08:57:02.493 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-24T08:57:02.500 INFO:teuthology.task.internal:no buildpackages task found 2024-04-24T08:57:02.500 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-24T08:57:02.523 INFO:teuthology.task.internal:Saving configuration 2024-04-24T08:57:02.565 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-24T08:57:02.572 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-24T08:57:02.596 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi090.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671225', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 08:50:44.519274', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK1UBMLD+nRVl0oMMWDQf2IwkCKv5zuhPfJbc1j1JbUDzNNDUJjbducopq88qDIxujCiscRPaSnx8IN4vtqh3/M='} 2024-04-24T08:57:02.621 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi114.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671225', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 08:50:44.515765', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH98UXW7tWWhJR9vA8aAMTU0MvISAGp74/oo/ZsBqHkECDNHSaUieKk9zatMsSgsDznu90xl8L2kToTnbF6DWIM='} 2024-04-24T08:57:02.645 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi149.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671225', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 08:50:44.517670', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB3WggCYnS3qkyggvta3gVJxUZVLDSdh1fIUvPO8pAGA+AMRKJwQy6xS1kxpVX6ZvddQQJxCECFayXG71dns0+M='} 2024-04-24T08:57:02.645 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-24T08:57:02.653 INFO:teuthology.task.internal:roles: ubuntu@smithi090.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-24T08:57:02.654 INFO:teuthology.task.internal:roles: ubuntu@smithi114.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-24T08:57:02.654 INFO:teuthology.task.internal:roles: ubuntu@smithi149.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-24T08:57:02.654 INFO:teuthology.run_tasks:Running task console_log... 2024-04-24T08:57:02.760 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc6a339a160>, signals=[15]) 2024-04-24T08:57:02.760 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-24T08:57:02.766 INFO:teuthology.task.internal:Opening connections... 2024-04-24T08:57:02.766 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi090.front.sepia.ceph.com 2024-04-24T08:57:02.769 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T08:57:02.845 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi114.front.sepia.ceph.com 2024-04-24T08:57:02.846 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T08:57:02.922 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi149.front.sepia.ceph.com 2024-04-24T08:57:02.923 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T08:57:02.998 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-24T08:57:03.005 DEBUG:teuthology.orchestra.run.smithi090:> uname -m 2024-04-24T08:57:03.023 INFO:teuthology.orchestra.run.smithi090.stdout:x86_64 2024-04-24T08:57:03.024 DEBUG:teuthology.orchestra.run.smithi090:> cat /etc/os-release 2024-04-24T08:57:03.081 INFO:teuthology.orchestra.run.smithi090.stdout:NAME="CentOS Stream" 2024-04-24T08:57:03.081 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION="9" 2024-04-24T08:57:03.081 INFO:teuthology.orchestra.run.smithi090.stdout:ID="centos" 2024-04-24T08:57:03.081 INFO:teuthology.orchestra.run.smithi090.stdout:ID_LIKE="rhel fedora" 2024-04-24T08:57:03.081 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION_ID="9" 2024-04-24T08:57:03.081 INFO:teuthology.orchestra.run.smithi090.stdout:PLATFORM_ID="platform:el9" 2024-04-24T08:57:03.081 INFO:teuthology.orchestra.run.smithi090.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T08:57:03.081 INFO:teuthology.orchestra.run.smithi090.stdout:ANSI_COLOR="0;31" 2024-04-24T08:57:03.081 INFO:teuthology.orchestra.run.smithi090.stdout:LOGO="fedora-logo-icon" 2024-04-24T08:57:03.081 INFO:teuthology.orchestra.run.smithi090.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T08:57:03.081 INFO:teuthology.orchestra.run.smithi090.stdout:HOME_URL="https://centos.org/" 2024-04-24T08:57:03.082 INFO:teuthology.orchestra.run.smithi090.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T08:57:03.082 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T08:57:03.082 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T08:57:03.082 INFO:teuthology.lock.ops:Updating smithi090.front.sepia.ceph.com on lock server 2024-04-24T08:57:03.104 DEBUG:teuthology.orchestra.run.smithi114:> uname -m 2024-04-24T08:57:03.122 INFO:teuthology.orchestra.run.smithi114.stdout:x86_64 2024-04-24T08:57:03.122 DEBUG:teuthology.orchestra.run.smithi114:> cat /etc/os-release 2024-04-24T08:57:03.178 INFO:teuthology.orchestra.run.smithi114.stdout:NAME="CentOS Stream" 2024-04-24T08:57:03.178 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION="9" 2024-04-24T08:57:03.179 INFO:teuthology.orchestra.run.smithi114.stdout:ID="centos" 2024-04-24T08:57:03.179 INFO:teuthology.orchestra.run.smithi114.stdout:ID_LIKE="rhel fedora" 2024-04-24T08:57:03.179 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION_ID="9" 2024-04-24T08:57:03.179 INFO:teuthology.orchestra.run.smithi114.stdout:PLATFORM_ID="platform:el9" 2024-04-24T08:57:03.179 INFO:teuthology.orchestra.run.smithi114.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T08:57:03.179 INFO:teuthology.orchestra.run.smithi114.stdout:ANSI_COLOR="0;31" 2024-04-24T08:57:03.179 INFO:teuthology.orchestra.run.smithi114.stdout:LOGO="fedora-logo-icon" 2024-04-24T08:57:03.179 INFO:teuthology.orchestra.run.smithi114.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T08:57:03.179 INFO:teuthology.orchestra.run.smithi114.stdout:HOME_URL="https://centos.org/" 2024-04-24T08:57:03.179 INFO:teuthology.orchestra.run.smithi114.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T08:57:03.179 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T08:57:03.179 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T08:57:03.180 INFO:teuthology.lock.ops:Updating smithi114.front.sepia.ceph.com on lock server 2024-04-24T08:57:03.195 DEBUG:teuthology.orchestra.run.smithi149:> uname -m 2024-04-24T08:57:03.213 INFO:teuthology.orchestra.run.smithi149.stdout:x86_64 2024-04-24T08:57:03.214 DEBUG:teuthology.orchestra.run.smithi149:> cat /etc/os-release 2024-04-24T08:57:03.271 INFO:teuthology.orchestra.run.smithi149.stdout:NAME="CentOS Stream" 2024-04-24T08:57:03.271 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION="9" 2024-04-24T08:57:03.271 INFO:teuthology.orchestra.run.smithi149.stdout:ID="centos" 2024-04-24T08:57:03.271 INFO:teuthology.orchestra.run.smithi149.stdout:ID_LIKE="rhel fedora" 2024-04-24T08:57:03.271 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION_ID="9" 2024-04-24T08:57:03.271 INFO:teuthology.orchestra.run.smithi149.stdout:PLATFORM_ID="platform:el9" 2024-04-24T08:57:03.271 INFO:teuthology.orchestra.run.smithi149.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T08:57:03.272 INFO:teuthology.orchestra.run.smithi149.stdout:ANSI_COLOR="0;31" 2024-04-24T08:57:03.272 INFO:teuthology.orchestra.run.smithi149.stdout:LOGO="fedora-logo-icon" 2024-04-24T08:57:03.272 INFO:teuthology.orchestra.run.smithi149.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T08:57:03.272 INFO:teuthology.orchestra.run.smithi149.stdout:HOME_URL="https://centos.org/" 2024-04-24T08:57:03.272 INFO:teuthology.orchestra.run.smithi149.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T08:57:03.272 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T08:57:03.272 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T08:57:03.272 INFO:teuthology.lock.ops:Updating smithi149.front.sepia.ceph.com on lock server 2024-04-24T08:57:03.296 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-24T08:57:03.310 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-24T08:57:03.315 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-24T08:57:03.315 DEBUG:teuthology.orchestra.run.smithi090:> test '!' -e /home/ubuntu/cephtest 2024-04-24T08:57:03.318 DEBUG:teuthology.orchestra.run.smithi114:> test '!' -e /home/ubuntu/cephtest 2024-04-24T08:57:03.320 DEBUG:teuthology.orchestra.run.smithi149:> test '!' -e /home/ubuntu/cephtest 2024-04-24T08:57:03.337 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-24T08:57:03.342 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-24T08:57:03.342 DEBUG:teuthology.orchestra.run.smithi090:> test -z $(ls -A /var/lib/ceph) 2024-04-24T08:57:03.374 DEBUG:teuthology.orchestra.run.smithi114:> test -z $(ls -A /var/lib/ceph) 2024-04-24T08:57:03.377 DEBUG:teuthology.orchestra.run.smithi149:> test -z $(ls -A /var/lib/ceph) 2024-04-24T08:57:03.465 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-24T08:57:03.544 INFO:teuthology.run_tasks:Running task kernel... 2024-04-24T08:57:03.560 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-04-24T08:57:03.561 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-04-24T08:57:03.561 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-04-24T08:57:03.561 DEBUG:teuthology.orchestra.run.smithi090:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T08:57:03.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T08:57:03.578 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-24T08:57:03.755 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T08:57:03.756 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-24T08:57:03.756 DEBUG:teuthology.orchestra.run.smithi090:> uname -r 2024-04-24T08:57:03.773 INFO:teuthology.orchestra.run.smithi090.stdout:5.14.0-437.el9.x86_64 2024-04-24T08:57:03.773 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T08:57:03.773 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-24T08:57:03.774 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum upgrade -y linux-firmware 2024-04-24T08:57:07.725 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - BaseOS 4.4 MB/s | 8.0 MB 00:01 2024-04-24T08:57:11.618 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - AppStream 6.8 MB/s | 19 MB 00:02 2024-04-24T08:57:15.604 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.2 MB 00:00 2024-04-24T08:57:17.104 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - Extras packages 54 kB/s | 16 kB 00:00 2024-04-24T08:57:18.814 INFO:teuthology.orchestra.run.smithi090.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-24T08:57:23.798 INFO:teuthology.orchestra.run.smithi090.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-04-24T08:57:25.770 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-24T08:57:25.772 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2024-04-24T08:57:25.772 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-24T08:57:25.882 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-24T08:57:25.882 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T08:57:26.027 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-24T08:57:26.028 DEBUG:teuthology.orchestra.run.smithi090:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-24T08:57:26.030 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi090.front.sepia.ceph.com to complete... 2024-04-24T08:57:38.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24 08:57:38 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-24T08:57:38.149 INFO:teuthology.orchestra.run.smithi090.stderr:FINISHED --2024-04-24 08:57:38-- 2024-04-24T08:57:38.149 INFO:teuthology.orchestra.run.smithi090.stderr:Total wall clock time: 12s 2024-04-24T08:57:38.149 INFO:teuthology.orchestra.run.smithi090.stderr:Downloaded: 1 files, 341M in 12s (28.8 MB/s) 2024-04-24T08:57:38.153 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-24T08:57:38.153 DEBUG:teuthology.orchestra.run.smithi090:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-24T08:57:40.296 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying... ######################################## 2024-04-24T08:57:40.481 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing... ######################################## 2024-04-24T08:57:40.597 INFO:teuthology.orchestra.run.smithi090.stdout:Updating / installing... 2024-04-24T08:57:48.588 INFO:teuthology.orchestra.run.smithi090.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-24T08:58:33.977 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi090.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-24T08:58:33.977 DEBUG:teuthology.orchestra.run.smithi090:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-24T08:58:34.068 INFO:teuthology.orchestra.run.smithi090.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-24T08:58:34.068 INFO:teuthology.orchestra.run.smithi090.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-24T08:58:34.068 INFO:teuthology.orchestra.run.smithi090.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-24T08:58:34.068 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-24T08:58:34.068 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-24T08:58:34.068 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-24T08:58:34.068 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-24T08:58:34.068 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-24T08:58:34.068 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-24T08:58:34.068 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-24T08:58:34.068 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-24T08:58:34.069 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-24T08:58:34.069 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-24T08:58:34.069 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-24T08:58:34.069 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-24T08:58:34.069 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-24T08:58:34.069 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-24T08:58:34.069 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-24T08:58:34.069 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-24T08:58:34.069 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-24T08:58:34.069 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-24T08:58:34.069 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-24T08:58:34.069 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-24T08:58:34.069 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-24T08:58:34.069 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-24T08:58:34.070 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-24T08:58:34.070 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-24T08:58:34.070 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-24T08:58:34.070 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-24T08:58:34.070 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-24T08:58:34.070 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-24T08:58:34.070 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-24T08:58:34.070 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-24T08:58:34.070 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-24T08:58:34.070 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-24T08:58:34.070 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-24T08:58:34.070 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-24T08:58:34.071 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-24T08:58:34.071 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-24T08:58:34.071 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-24T08:58:34.071 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-24T08:58:34.071 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-24T08:58:34.071 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-24T08:58:34.071 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-24T08:58:34.071 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-24T08:58:34.071 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-24T08:58:34.071 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-24T08:58:34.071 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-24T08:58:34.071 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-24T08:58:34.071 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-24T08:58:34.072 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-24T08:58:34.072 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-24T08:58:34.072 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-24T08:58:34.072 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-24T08:58:34.072 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-24T08:58:34.072 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-24T08:58:34.072 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-24T08:58:34.072 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-24T08:58:34.072 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-24T08:58:34.072 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-24T08:58:34.072 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-24T08:58:34.072 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-24T08:58:34.072 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-24T08:58:34.072 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-24T08:58:34.073 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-24T08:58:34.073 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-24T08:58:34.073 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-24T08:58:34.073 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-24T08:58:34.073 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-24T08:58:34.073 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-24T08:58:34.073 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-24T08:58:34.073 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-24T08:58:34.073 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-24T08:58:34.073 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-24T08:58:34.073 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-24T08:58:34.073 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-24T08:58:34.073 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-24T08:58:34.074 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-24T08:58:34.074 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-24T08:58:34.074 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-24T08:58:34.074 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-24T08:58:34.074 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-24T08:58:34.074 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-24T08:58:34.074 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-24T08:58:34.074 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-24T08:58:34.074 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-24T08:58:34.074 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-24T08:58:34.074 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-24T08:58:34.074 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-24T08:58:34.074 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-24T08:58:34.074 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-24T08:58:34.075 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-24T08:58:34.075 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-24T08:58:34.075 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-24T08:58:34.075 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-24T08:58:34.075 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-24T08:58:34.075 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-24T08:58:34.075 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-24T08:58:34.075 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-24T08:58:34.075 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-24T08:58:34.075 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-24T08:58:34.075 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-24T08:58:34.075 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-24T08:58:34.075 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-24T08:58:34.076 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-24T08:58:34.076 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-24T08:58:34.076 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-24T08:58:34.076 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-24T08:58:34.076 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-24T08:58:34.076 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-24T08:58:34.076 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-24T08:58:34.076 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-24T08:58:34.076 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-24T08:58:34.076 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-24T08:58:34.076 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-24T08:58:34.076 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-24T08:58:34.076 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-24T08:58:34.076 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-24T08:58:34.076 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-24T08:58:34.077 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-24T08:58:34.077 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-24T08:58:34.077 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-24T08:58:34.077 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-24T08:58:34.077 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-24T08:58:34.077 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-24T08:58:34.077 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-24T08:58:34.077 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-24T08:58:34.077 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-24T08:58:34.077 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-24T08:58:34.077 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-24T08:58:34.077 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-24T08:58:34.077 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-24T08:58:34.077 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-24T08:58:34.078 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-24T08:58:34.078 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-24T08:58:34.078 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-24T08:58:34.078 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-24T08:58:34.078 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-24T08:58:34.078 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-24T08:58:34.078 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-24T08:58:34.078 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-24T08:58:34.078 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-24T08:58:34.078 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-24T08:58:34.078 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-24T08:58:34.078 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-24T08:58:34.079 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-24T08:58:34.079 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-24T08:58:34.079 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-24T08:58:34.079 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-24T08:58:34.079 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-24T08:58:34.079 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-24T08:58:34.079 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-24T08:58:34.079 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-24T08:58:34.079 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-24T08:58:34.079 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-24T08:58:34.079 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-24T08:58:34.079 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-24T08:58:34.079 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-24T08:58:34.079 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-24T08:58:34.080 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-24T08:58:34.080 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-24T08:58:34.080 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-24T08:58:34.080 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-24T08:58:34.080 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-24T08:58:34.080 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-24T08:58:34.080 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-24T08:58:34.080 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-24T08:58:34.080 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-24T08:58:34.080 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-24T08:58:34.080 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-24T08:58:34.080 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-24T08:58:34.080 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-24T08:58:34.080 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-24T08:58:34.080 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-24T08:58:34.081 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-24T08:58:34.081 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-24T08:58:34.081 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-24T08:58:34.081 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-24T08:58:34.081 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-24T08:58:34.081 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-24T08:58:34.081 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-24T08:58:34.081 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-24T08:58:34.081 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-24T08:58:34.081 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-24T08:58:34.081 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-24T08:58:34.081 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-24T08:58:34.081 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-24T08:58:34.081 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-24T08:58:34.082 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-24T08:58:34.082 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-24T08:58:34.082 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-24T08:58:34.082 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-24T08:58:34.082 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-24T08:58:34.082 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-24T08:58:34.082 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-24T08:58:34.082 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-24T08:58:34.082 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-24T08:58:34.082 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-24T08:58:34.082 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-24T08:58:34.082 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-24T08:58:34.082 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-24T08:58:34.082 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-24T08:58:34.083 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-24T08:58:34.083 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-24T08:58:34.083 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-24T08:58:34.083 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-24T08:58:34.083 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-24T08:58:34.083 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-24T08:58:34.083 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-24T08:58:34.083 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-24T08:58:34.083 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-24T08:58:34.083 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-24T08:58:34.083 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-24T08:58:34.083 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-24T08:58:34.083 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-24T08:58:34.083 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-24T08:58:34.083 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-24T08:58:34.084 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-24T08:58:34.084 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-24T08:58:34.084 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-24T08:58:34.084 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-24T08:58:34.084 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-24T08:58:34.084 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-24T08:58:34.084 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-24T08:58:34.084 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-24T08:58:34.084 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-24T08:58:34.084 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-24T08:58:34.084 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-24T08:58:34.084 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-24T08:58:34.084 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-24T08:58:34.084 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-24T08:58:34.084 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-24T08:58:34.084 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-24T08:58:34.085 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-24T08:58:34.085 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-24T08:58:34.085 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-24T08:58:34.085 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-24T08:58:34.085 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-24T08:58:34.085 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-24T08:58:34.085 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-24T08:58:34.085 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-24T08:58:34.085 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-24T08:58:34.085 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-24T08:58:34.085 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-24T08:58:34.085 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-24T08:58:34.086 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-24T08:58:34.086 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-24T08:58:34.086 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-24T08:58:34.086 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-24T08:58:34.086 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-24T08:58:34.086 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-24T08:58:34.086 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-24T08:58:34.086 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-24T08:58:34.086 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-24T08:58:34.086 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-24T08:58:34.086 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-24T08:58:34.086 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-24T08:58:34.086 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-24T08:58:34.087 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-24T08:58:34.087 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-24T08:58:34.087 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-24T08:58:34.087 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-24T08:58:34.087 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-24T08:58:34.087 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-24T08:58:34.087 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-24T08:58:34.087 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-24T08:58:34.087 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-24T08:58:34.087 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-24T08:58:34.087 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-24T08:58:34.087 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-24T08:58:34.087 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-24T08:58:34.087 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-24T08:58:34.088 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-24T08:58:34.088 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-24T08:58:34.088 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-24T08:58:34.088 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-24T08:58:34.088 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-24T08:58:34.088 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-24T08:58:34.088 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-24T08:58:34.088 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-24T08:58:34.088 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-24T08:58:34.088 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-24T08:58:34.088 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-24T08:58:34.088 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-24T08:58:34.088 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-24T08:58:34.088 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-24T08:58:34.088 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-24T08:58:34.089 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-24T08:58:34.089 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-24T08:58:34.089 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-24T08:58:34.089 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-24T08:58:34.089 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-24T08:58:34.089 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-24T08:58:34.089 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-24T08:58:34.089 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-24T08:58:34.089 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-24T08:58:34.089 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-24T08:58:34.089 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-24T08:58:34.089 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-24T08:58:34.089 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-24T08:58:34.089 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-24T08:58:34.089 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-24T08:58:34.090 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-24T08:58:34.090 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-24T08:58:34.090 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-24T08:58:34.090 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-24T08:58:34.090 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-24T08:58:34.090 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-24T08:58:34.090 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-24T08:58:34.090 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-24T08:58:34.090 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-24T08:58:34.090 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-24T08:58:34.090 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-24T08:58:34.090 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-24T08:58:34.090 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-24T08:58:34.090 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-24T08:58:34.091 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-24T08:58:34.091 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-24T08:58:34.091 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-24T08:58:34.091 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-24T08:58:34.091 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-24T08:58:34.091 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-24T08:58:34.091 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-24T08:58:34.091 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-24T08:58:34.091 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-24T08:58:34.091 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-24T08:58:34.091 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-24T08:58:34.091 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-24T08:58:34.091 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-24T08:58:34.091 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-24T08:58:34.092 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-24T08:58:34.092 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-24T08:58:34.092 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-24T08:58:34.092 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-24T08:58:34.092 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-24T08:58:34.092 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-24T08:58:34.092 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-24T08:58:34.092 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-24T08:58:34.092 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-24T08:58:34.092 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-24T08:58:34.092 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-24T08:58:34.092 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-24T08:58:34.092 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-24T08:58:34.092 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-24T08:58:34.092 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-24T08:58:34.093 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-24T08:58:34.093 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-24T08:58:34.093 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-24T08:58:34.093 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-24T08:58:34.093 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-24T08:58:34.093 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-24T08:58:34.093 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-24T08:58:34.093 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-24T08:58:34.093 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-24T08:58:34.093 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-24T08:58:34.093 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-24T08:58:34.093 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-24T08:58:34.093 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-24T08:58:34.094 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-24T08:58:34.094 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-24T08:58:34.094 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-24T08:58:34.094 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-24T08:58:34.094 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-24T08:58:34.094 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-24T08:58:34.094 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-24T08:58:34.094 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-24T08:58:34.094 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-24T08:58:34.094 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-24T08:58:34.094 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-24T08:58:34.094 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-24T08:58:34.094 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-24T08:58:34.095 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-24T08:58:34.095 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-24T08:58:34.095 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-24T08:58:34.095 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-24T08:58:34.095 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-24T08:58:34.095 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-24T08:58:34.095 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-24T08:58:34.095 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-24T08:58:34.095 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-24T08:58:34.095 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-24T08:58:34.095 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-24T08:58:34.095 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-24T08:58:34.095 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-24T08:58:34.095 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-24T08:58:34.096 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-24T08:58:34.096 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-24T08:58:34.096 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-24T08:58:34.096 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-24T08:58:34.096 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-24T08:58:34.096 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-24T08:58:34.096 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-24T08:58:34.096 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-24T08:58:34.096 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-24T08:58:34.096 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-24T08:58:34.096 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-24T08:58:34.096 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-24T08:58:34.096 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-24T08:58:34.096 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-24T08:58:34.097 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-24T08:58:34.097 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-24T08:58:34.097 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-24T08:58:34.097 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-24T08:58:34.097 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-24T08:58:34.097 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-24T08:58:34.097 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-24T08:58:34.097 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-24T08:58:34.097 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-24T08:58:34.097 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-24T08:58:34.097 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-24T08:58:34.097 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-24T08:58:34.097 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-24T08:58:34.097 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-24T08:58:34.097 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-24T08:58:34.098 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-24T08:58:34.098 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-24T08:58:34.098 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-24T08:58:34.098 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-24T08:58:34.098 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-24T08:58:34.098 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-24T08:58:34.098 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-24T08:58:34.099 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-24T08:58:34.099 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-24T08:58:34.099 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-24T08:58:34.099 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-24T08:58:34.099 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-24T08:58:34.099 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-24T08:58:34.099 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-24T08:58:34.099 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-24T08:58:34.099 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-24T08:58:34.099 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-24T08:58:34.099 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-24T08:58:34.099 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-24T08:58:34.100 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-24T08:58:34.100 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-24T08:58:34.100 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-24T08:58:34.100 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-24T08:58:34.100 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-24T08:58:34.100 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-24T08:58:34.100 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-24T08:58:34.100 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-24T08:58:34.100 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-24T08:58:34.100 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-24T08:58:34.100 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-24T08:58:34.100 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-24T08:58:34.100 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-24T08:58:34.100 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-24T08:58:34.101 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-24T08:58:34.101 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-24T08:58:34.101 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-24T08:58:34.101 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-24T08:58:34.101 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-24T08:58:34.101 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-24T08:58:34.101 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-24T08:58:34.101 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-24T08:58:34.101 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-24T08:58:34.101 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-24T08:58:34.101 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-24T08:58:34.101 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-24T08:58:34.101 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-24T08:58:34.102 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-24T08:58:34.102 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-24T08:58:34.102 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-24T08:58:34.102 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-24T08:58:34.102 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-24T08:58:34.102 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-24T08:58:34.102 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-24T08:58:34.102 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-24T08:58:34.102 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-24T08:58:34.102 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-24T08:58:34.102 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-24T08:58:34.102 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-24T08:58:34.103 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-24T08:58:34.103 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-24T08:58:34.103 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-24T08:58:34.103 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-24T08:58:34.103 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-24T08:58:34.103 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-24T08:58:34.103 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-24T08:58:34.103 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-24T08:58:34.103 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-24T08:58:34.103 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-24T08:58:34.103 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-24T08:58:34.103 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-24T08:58:34.103 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-24T08:58:34.103 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-24T08:58:34.104 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-24T08:58:34.104 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-24T08:58:34.104 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-24T08:58:34.104 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-24T08:58:34.104 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-24T08:58:34.104 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-24T08:58:34.104 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-24T08:58:34.104 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-24T08:58:34.104 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-24T08:58:34.104 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-24T08:58:34.104 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-24T08:58:34.104 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-24T08:58:34.104 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-24T08:58:34.104 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-24T08:58:34.105 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-24T08:58:34.105 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-24T08:58:34.105 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-24T08:58:34.105 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-24T08:58:34.105 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-24T08:58:34.105 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-24T08:58:34.105 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-24T08:58:34.105 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-24T08:58:34.105 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-24T08:58:34.105 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-24T08:58:34.105 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-24T08:58:34.105 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-24T08:58:34.105 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-24T08:58:34.105 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-24T08:58:34.106 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-24T08:58:34.106 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-24T08:58:34.106 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-24T08:58:34.106 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-24T08:58:34.106 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-24T08:58:34.106 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-24T08:58:34.106 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-24T08:58:34.106 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-24T08:58:34.106 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-24T08:58:34.106 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-24T08:58:34.106 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-24T08:58:34.106 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-24T08:58:34.106 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-24T08:58:34.106 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-24T08:58:34.107 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-24T08:58:34.107 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-24T08:58:34.107 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-24T08:58:34.107 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-24T08:58:34.107 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-24T08:58:34.107 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-24T08:58:34.107 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-24T08:58:34.107 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-24T08:58:34.107 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-24T08:58:34.107 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-24T08:58:34.107 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-24T08:58:34.107 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-24T08:58:34.107 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-24T08:58:34.107 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-24T08:58:34.107 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-24T08:58:34.107 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-24T08:58:34.108 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-24T08:58:34.108 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-24T08:58:34.108 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-24T08:58:34.108 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-24T08:58:34.108 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-24T08:58:34.108 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-24T08:58:34.108 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-24T08:58:34.108 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-24T08:58:34.108 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-24T08:58:34.108 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-24T08:58:34.108 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-24T08:58:34.108 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-24T08:58:34.108 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-24T08:58:34.108 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-24T08:58:34.108 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-24T08:58:34.109 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-24T08:58:34.109 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-24T08:58:34.109 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-24T08:58:34.109 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-24T08:58:34.109 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-24T08:58:34.109 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-24T08:58:34.109 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-24T08:58:34.109 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-24T08:58:34.109 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-24T08:58:34.109 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-24T08:58:34.109 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-24T08:58:34.109 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-24T08:58:34.109 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-24T08:58:34.109 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-24T08:58:34.109 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-24T08:58:34.110 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-24T08:58:34.110 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-24T08:58:34.110 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-24T08:58:34.110 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-24T08:58:34.110 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-24T08:58:34.110 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-24T08:58:34.110 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-24T08:58:34.110 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-24T08:58:34.110 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-24T08:58:34.110 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-24T08:58:34.110 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-24T08:58:34.110 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-24T08:58:34.110 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-24T08:58:34.110 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-24T08:58:34.110 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-24T08:58:34.111 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-24T08:58:34.111 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-24T08:58:34.111 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-24T08:58:34.111 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-24T08:58:34.111 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-24T08:58:34.111 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-24T08:58:34.111 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-24T08:58:34.111 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-24T08:58:34.111 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-24T08:58:34.111 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-24T08:58:34.111 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-24T08:58:34.111 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-24T08:58:34.111 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-24T08:58:34.111 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-24T08:58:34.111 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-24T08:58:34.111 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-24T08:58:34.112 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-24T08:58:34.112 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-24T08:58:34.112 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-24T08:58:34.112 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-24T08:58:34.112 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-24T08:58:34.112 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-24T08:58:34.112 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-24T08:58:34.112 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-24T08:58:34.112 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-24T08:58:34.112 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-24T08:58:34.112 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-24T08:58:34.112 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-24T08:58:34.112 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-24T08:58:34.112 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-24T08:58:34.112 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-24T08:58:34.112 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-24T08:58:34.113 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-24T08:58:34.113 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-24T08:58:34.113 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-24T08:58:34.113 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-24T08:58:34.113 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-24T08:58:34.113 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-24T08:58:34.113 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-24T08:58:34.113 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-24T08:58:34.113 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-24T08:58:34.113 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-24T08:58:34.113 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-24T08:58:34.113 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-24T08:58:34.113 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-24T08:58:34.113 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-24T08:58:34.113 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-24T08:58:34.113 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-24T08:58:34.114 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-24T08:58:34.114 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-24T08:58:34.114 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-24T08:58:34.114 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-24T08:58:34.114 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-24T08:58:34.114 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-24T08:58:34.114 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-24T08:58:34.114 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-24T08:58:34.114 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-24T08:58:34.114 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-24T08:58:34.114 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-24T08:58:34.114 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-24T08:58:34.114 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-24T08:58:34.115 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-24T08:58:34.115 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-24T08:58:34.115 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-24T08:58:34.115 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-24T08:58:34.115 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-24T08:58:34.115 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-24T08:58:34.115 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-24T08:58:34.115 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-24T08:58:34.115 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-24T08:58:34.115 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-24T08:58:34.115 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-24T08:58:34.115 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-24T08:58:34.115 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-24T08:58:34.115 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-24T08:58:34.115 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-24T08:58:34.116 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-24T08:58:34.116 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-24T08:58:34.116 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-24T08:58:34.116 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-24T08:58:34.116 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-24T08:58:34.116 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-24T08:58:34.116 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-24T08:58:34.116 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-24T08:58:34.116 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-24T08:58:34.116 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-24T08:58:34.116 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-24T08:58:34.116 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-24T08:58:34.116 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-24T08:58:34.117 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-24T08:58:34.117 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-24T08:58:34.117 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-24T08:58:34.117 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-24T08:58:34.117 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-24T08:58:34.117 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-24T08:58:34.117 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-24T08:58:34.117 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-24T08:58:34.117 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-24T08:58:34.117 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-24T08:58:34.117 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-24T08:58:34.117 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-24T08:58:34.117 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-24T08:58:34.117 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-24T08:58:34.117 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-24T08:58:34.118 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-24T08:58:34.118 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-24T08:58:34.118 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-24T08:58:34.118 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-24T08:58:34.118 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-24T08:58:34.118 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-24T08:58:34.118 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-24T08:58:34.118 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-24T08:58:34.118 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-24T08:58:34.118 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-24T08:58:34.118 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-24T08:58:34.118 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-24T08:58:34.118 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-24T08:58:34.118 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-24T08:58:34.119 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-24T08:58:34.119 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-24T08:58:34.119 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-24T08:58:34.119 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-24T08:58:34.119 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-24T08:58:34.119 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-24T08:58:34.119 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-24T08:58:34.119 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-24T08:58:34.119 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-24T08:58:34.119 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-24T08:58:34.119 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-24T08:58:34.119 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-24T08:58:34.119 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-24T08:58:34.119 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-24T08:58:34.120 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-24T08:58:34.120 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-24T08:58:34.120 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-24T08:58:34.120 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-24T08:58:34.120 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-24T08:58:34.120 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-24T08:58:34.120 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-24T08:58:34.120 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-24T08:58:34.120 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-24T08:58:34.120 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-24T08:58:34.120 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-24T08:58:34.120 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-24T08:58:34.120 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-24T08:58:34.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-24T08:58:34.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-24T08:58:34.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-24T08:58:34.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-24T08:58:34.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-24T08:58:34.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-24T08:58:34.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-24T08:58:34.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-24T08:58:34.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-24T08:58:34.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-24T08:58:34.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-24T08:58:34.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-24T08:58:34.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-24T08:58:34.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-24T08:58:34.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-24T08:58:34.122 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-24T08:58:34.122 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-24T08:58:34.122 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-24T08:58:34.122 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-24T08:58:34.122 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-24T08:58:34.122 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-24T08:58:34.122 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-24T08:58:34.122 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-24T08:58:34.122 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-24T08:58:34.122 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-24T08:58:34.122 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-24T08:58:34.122 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-24T08:58:34.122 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-24T08:58:34.122 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-24T08:58:34.123 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-24T08:58:34.123 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-24T08:58:34.123 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-24T08:58:34.123 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-24T08:58:34.123 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-24T08:58:34.123 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-24T08:58:34.123 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-24T08:58:34.123 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-24T08:58:34.123 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-24T08:58:34.123 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-24T08:58:34.123 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-24T08:58:34.123 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-24T08:58:34.123 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-24T08:58:34.123 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-24T08:58:34.124 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-24T08:58:34.124 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-24T08:58:34.124 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-24T08:58:34.124 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-24T08:58:34.124 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-24T08:58:34.124 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-24T08:58:34.124 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-24T08:58:34.124 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-24T08:58:34.124 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-24T08:58:34.124 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-24T08:58:34.124 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-24T08:58:34.124 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-24T08:58:34.124 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-24T08:58:34.124 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-24T08:58:34.125 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-24T08:58:34.125 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-24T08:58:34.125 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-24T08:58:34.125 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-24T08:58:34.125 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-24T08:58:34.125 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-24T08:58:34.125 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-24T08:58:34.125 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-24T08:58:34.125 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-24T08:58:34.125 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-24T08:58:34.125 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-24T08:58:34.125 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-24T08:58:34.125 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-24T08:58:34.125 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-24T08:58:34.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-24T08:58:34.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-24T08:58:34.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-24T08:58:34.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-24T08:58:34.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-24T08:58:34.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-24T08:58:34.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-24T08:58:34.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-24T08:58:34.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-24T08:58:34.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-24T08:58:34.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-24T08:58:34.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-24T08:58:34.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-24T08:58:34.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-24T08:58:34.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-24T08:58:34.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-24T08:58:34.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-24T08:58:34.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-24T08:58:34.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-24T08:58:34.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-24T08:58:34.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-24T08:58:34.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-24T08:58:34.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-24T08:58:34.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-24T08:58:34.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-24T08:58:34.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-24T08:58:34.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-24T08:58:34.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-24T08:58:34.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-24T08:58:34.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-24T08:58:34.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-24T08:58:34.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-24T08:58:34.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-24T08:58:34.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-24T08:58:34.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-24T08:58:34.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-24T08:58:34.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-24T08:58:34.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-24T08:58:34.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-24T08:58:34.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-24T08:58:34.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-24T08:58:34.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-24T08:58:34.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-24T08:58:34.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-24T08:58:34.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-24T08:58:34.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-24T08:58:34.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-24T08:58:34.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-24T08:58:34.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-24T08:58:34.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-24T08:58:34.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-24T08:58:34.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-24T08:58:34.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-24T08:58:34.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-24T08:58:34.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-24T08:58:34.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-24T08:58:34.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-24T08:58:34.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-24T08:58:34.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-24T08:58:34.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-24T08:58:34.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-24T08:58:34.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-24T08:58:34.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-24T08:58:34.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-24T08:58:34.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-24T08:58:34.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-24T08:58:34.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-24T08:58:34.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-24T08:58:34.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-24T08:58:34.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-24T08:58:34.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-24T08:58:34.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-24T08:58:34.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-24T08:58:34.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-24T08:58:34.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-24T08:58:34.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-24T08:58:34.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-24T08:58:34.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-24T08:58:34.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-24T08:58:34.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-24T08:58:34.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-24T08:58:34.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-24T08:58:34.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-24T08:58:34.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-24T08:58:34.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-24T08:58:34.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-24T08:58:34.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-24T08:58:34.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-24T08:58:34.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-24T08:58:34.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-24T08:58:34.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-24T08:58:34.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-24T08:58:34.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-24T08:58:34.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-24T08:58:34.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-24T08:58:34.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-24T08:58:34.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-24T08:58:34.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-24T08:58:34.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-24T08:58:34.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-24T08:58:34.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-24T08:58:34.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-24T08:58:34.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-24T08:58:34.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-24T08:58:34.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-24T08:58:34.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-24T08:58:34.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-24T08:58:34.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-24T08:58:34.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-24T08:58:34.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-24T08:58:34.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-24T08:58:34.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-24T08:58:34.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-24T08:58:34.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-24T08:58:34.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-24T08:58:34.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-24T08:58:34.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-24T08:58:34.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-24T08:58:34.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-24T08:58:34.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-24T08:58:34.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-24T08:58:34.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-24T08:58:34.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-24T08:58:34.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-24T08:58:34.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-24T08:58:34.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-24T08:58:34.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-24T08:58:34.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-24T08:58:34.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-24T08:58:34.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-24T08:58:34.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-24T08:58:34.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-24T08:58:34.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-24T08:58:34.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-24T08:58:34.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-24T08:58:34.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-24T08:58:34.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-24T08:58:34.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-24T08:58:34.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-24T08:58:34.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-24T08:58:34.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-24T08:58:34.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-24T08:58:34.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-24T08:58:34.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-24T08:58:34.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-24T08:58:34.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-24T08:58:34.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-24T08:58:34.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-24T08:58:34.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-24T08:58:34.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-24T08:58:34.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-24T08:58:34.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-24T08:58:34.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-24T08:58:34.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-24T08:58:34.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-24T08:58:34.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-24T08:58:34.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-24T08:58:34.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-24T08:58:34.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-24T08:58:34.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-24T08:58:34.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-24T08:58:34.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-24T08:58:34.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-24T08:58:34.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-24T08:58:34.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-24T08:58:34.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-24T08:58:34.138 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-24T08:58:34.138 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-24T08:58:34.138 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-24T08:58:34.138 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-24T08:58:34.138 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-24T08:58:34.138 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-24T08:58:34.138 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-24T08:58:34.138 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-24T08:58:34.138 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-24T08:58:34.138 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-24T08:58:34.138 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-24T08:58:34.138 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-24T08:58:34.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-24T08:58:34.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-24T08:58:34.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-24T08:58:34.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-24T08:58:34.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-24T08:58:34.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-24T08:58:34.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-24T08:58:34.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-24T08:58:34.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-24T08:58:34.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-24T08:58:34.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-24T08:58:34.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-24T08:58:34.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-24T08:58:34.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-24T08:58:34.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-24T08:58:34.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-24T08:58:34.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-24T08:58:34.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-24T08:58:34.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-24T08:58:34.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-24T08:58:34.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-24T08:58:34.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-24T08:58:34.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-24T08:58:34.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-24T08:58:34.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-24T08:58:34.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-24T08:58:34.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-24T08:58:34.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-24T08:58:34.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-24T08:58:34.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-24T08:58:34.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-24T08:58:34.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-24T08:58:34.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-24T08:58:34.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-24T08:58:34.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-24T08:58:34.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-24T08:58:34.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-24T08:58:34.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-24T08:58:34.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-24T08:58:34.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-24T08:58:34.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-24T08:58:34.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-24T08:58:34.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-24T08:58:34.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-24T08:58:34.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-24T08:58:34.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-24T08:58:34.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-24T08:58:34.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-24T08:58:34.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-24T08:58:34.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-24T08:58:34.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-24T08:58:34.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-24T08:58:34.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-24T08:58:34.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-24T08:58:34.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-24T08:58:34.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-24T08:58:34.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-24T08:58:34.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-24T08:58:34.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-24T08:58:34.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-24T08:58:34.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-24T08:58:34.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-24T08:58:34.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-24T08:58:34.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-24T08:58:34.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-24T08:58:34.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-24T08:58:34.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-24T08:58:34.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-24T08:58:34.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-24T08:58:34.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-24T08:58:34.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-24T08:58:34.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-24T08:58:34.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-24T08:58:34.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-24T08:58:34.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-24T08:58:34.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-24T08:58:34.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-24T08:58:34.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-24T08:58:34.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-24T08:58:34.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-24T08:58:34.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-24T08:58:34.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-24T08:58:34.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-24T08:58:34.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-24T08:58:34.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-24T08:58:34.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-24T08:58:34.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-24T08:58:34.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-24T08:58:34.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-24T08:58:34.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-24T08:58:34.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-24T08:58:34.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-24T08:58:34.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-24T08:58:34.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-24T08:58:34.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-24T08:58:34.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-24T08:58:34.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-24T08:58:34.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-24T08:58:34.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-24T08:58:34.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-24T08:58:34.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-24T08:58:34.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-24T08:58:34.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-24T08:58:34.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-24T08:58:34.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-24T08:58:34.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-24T08:58:34.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-24T08:58:34.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-24T08:58:34.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-24T08:58:34.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-24T08:58:34.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-24T08:58:34.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-24T08:58:34.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-24T08:58:34.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-24T08:58:34.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-24T08:58:34.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-24T08:58:34.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-24T08:58:34.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-24T08:58:34.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-24T08:58:34.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-24T08:58:34.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-24T08:58:34.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-24T08:58:34.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-24T08:58:34.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-24T08:58:34.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-24T08:58:34.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-24T08:58:34.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-24T08:58:34.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-24T08:58:34.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-24T08:58:34.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-24T08:58:34.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-24T08:58:34.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-24T08:58:34.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-24T08:58:34.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-24T08:58:34.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-24T08:58:34.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-24T08:58:34.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-24T08:58:34.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-24T08:58:34.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-24T08:58:34.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-24T08:58:34.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-24T08:58:34.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-24T08:58:34.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-24T08:58:34.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-24T08:58:34.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-24T08:58:34.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-24T08:58:34.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-24T08:58:34.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-24T08:58:34.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-24T08:58:34.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-24T08:58:34.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-24T08:58:34.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-24T08:58:34.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-24T08:58:34.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-24T08:58:34.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-24T08:58:34.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-24T08:58:34.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-24T08:58:34.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-24T08:58:34.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-24T08:58:34.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-24T08:58:34.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-24T08:58:34.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-24T08:58:34.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-24T08:58:34.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-24T08:58:34.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-24T08:58:34.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-24T08:58:34.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-24T08:58:34.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-24T08:58:34.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-24T08:58:34.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-24T08:58:34.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-24T08:58:34.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-24T08:58:34.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-24T08:58:34.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-24T08:58:34.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-24T08:58:34.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-24T08:58:34.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-24T08:58:34.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-24T08:58:34.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-24T08:58:34.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-24T08:58:34.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-24T08:58:34.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-24T08:58:34.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-24T08:58:34.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-24T08:58:34.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-24T08:58:34.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-24T08:58:34.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-24T08:58:34.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-24T08:58:34.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-24T08:58:34.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-24T08:58:34.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-24T08:58:34.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-24T08:58:34.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-24T08:58:34.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-24T08:58:34.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-24T08:58:34.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-24T08:58:34.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-24T08:58:34.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-24T08:58:34.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-24T08:58:34.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-24T08:58:34.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-24T08:58:34.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-24T08:58:34.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-24T08:58:34.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-24T08:58:34.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-24T08:58:34.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-24T08:58:34.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-24T08:58:34.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-24T08:58:34.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-24T08:58:34.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-24T08:58:34.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-24T08:58:34.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-24T08:58:34.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-24T08:58:34.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-24T08:58:34.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-24T08:58:34.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-24T08:58:34.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-24T08:58:34.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-24T08:58:34.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-24T08:58:34.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-24T08:58:34.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-24T08:58:34.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-24T08:58:34.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-24T08:58:34.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-24T08:58:34.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-24T08:58:34.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-24T08:58:34.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-24T08:58:34.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-24T08:58:34.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-24T08:58:34.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-24T08:58:34.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-24T08:58:34.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-24T08:58:34.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-24T08:58:34.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-24T08:58:34.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-24T08:58:34.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-24T08:58:34.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-24T08:58:34.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-24T08:58:34.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-24T08:58:34.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-24T08:58:34.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-24T08:58:34.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-24T08:58:34.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-24T08:58:34.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-24T08:58:34.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-24T08:58:34.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-24T08:58:34.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-24T08:58:34.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-24T08:58:34.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-24T08:58:34.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-24T08:58:34.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-24T08:58:34.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-24T08:58:34.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-24T08:58:34.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-24T08:58:34.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-24T08:58:34.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-24T08:58:34.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-24T08:58:34.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-24T08:58:34.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-24T08:58:34.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-24T08:58:34.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-24T08:58:34.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-24T08:58:34.157 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-24T08:58:34.157 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-24T08:58:34.157 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-24T08:58:34.157 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-24T08:58:34.157 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-24T08:58:34.157 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-24T08:58:34.157 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-24T08:58:34.157 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-24T08:58:34.157 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-24T08:58:34.157 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-24T08:58:34.157 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-24T08:58:34.157 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-24T08:58:34.157 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-24T08:58:34.157 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-24T08:58:34.157 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-24T08:58:34.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-24T08:58:34.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-24T08:58:34.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-24T08:58:34.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-24T08:58:34.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-24T08:58:34.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-24T08:58:34.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-24T08:58:34.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-24T08:58:34.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-24T08:58:34.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-24T08:58:34.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-24T08:58:34.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-24T08:58:34.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-24T08:58:34.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-24T08:58:34.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-24T08:58:34.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-24T08:58:34.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-24T08:58:34.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-24T08:58:34.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-24T08:58:34.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-24T08:58:34.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-24T08:58:34.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-24T08:58:34.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-24T08:58:34.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-24T08:58:34.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-24T08:58:34.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-24T08:58:34.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-24T08:58:34.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-24T08:58:34.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-24T08:58:34.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-24T08:58:34.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-24T08:58:34.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-24T08:58:34.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-24T08:58:34.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-24T08:58:34.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-24T08:58:34.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-24T08:58:34.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-24T08:58:34.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-24T08:58:34.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-24T08:58:34.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-24T08:58:34.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-24T08:58:34.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-24T08:58:34.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-24T08:58:34.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-24T08:58:34.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-24T08:58:34.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-24T08:58:34.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-24T08:58:34.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-24T08:58:34.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-24T08:58:34.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-24T08:58:34.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-24T08:58:34.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-24T08:58:34.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-24T08:58:34.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-24T08:58:34.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-24T08:58:34.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-24T08:58:34.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-24T08:58:34.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-24T08:58:34.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-24T08:58:34.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-24T08:58:34.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-24T08:58:34.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-24T08:58:34.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-24T08:58:34.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-24T08:58:34.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-24T08:58:34.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-24T08:58:34.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-24T08:58:34.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-24T08:58:34.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-24T08:58:34.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-24T08:58:34.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-24T08:58:34.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-24T08:58:34.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-24T08:58:34.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-24T08:58:34.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-24T08:58:34.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-24T08:58:34.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-24T08:58:34.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-24T08:58:34.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-24T08:58:34.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-24T08:58:34.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-24T08:58:34.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-24T08:58:34.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-24T08:58:34.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-24T08:58:34.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-24T08:58:34.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-24T08:58:34.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-24T08:58:34.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-24T08:58:34.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-24T08:58:34.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-24T08:58:34.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-24T08:58:34.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-24T08:58:34.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-24T08:58:34.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-24T08:58:34.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-24T08:58:34.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-24T08:58:34.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-24T08:58:34.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-24T08:58:34.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-24T08:58:34.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-24T08:58:34.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-24T08:58:34.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-24T08:58:34.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-24T08:58:34.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-24T08:58:34.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-24T08:58:34.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-24T08:58:34.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-24T08:58:34.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-24T08:58:34.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-24T08:58:34.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-24T08:58:34.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-24T08:58:34.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-24T08:58:34.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-24T08:58:34.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-24T08:58:34.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-24T08:58:34.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-24T08:58:34.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-24T08:58:34.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-24T08:58:34.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-24T08:58:34.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-24T08:58:34.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-24T08:58:34.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-24T08:58:34.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-24T08:58:34.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-24T08:58:34.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-24T08:58:34.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-24T08:58:34.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-24T08:58:34.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-24T08:58:34.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-24T08:58:34.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-24T08:58:34.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-24T08:58:34.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-24T08:58:34.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-24T08:58:34.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-24T08:58:34.167 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-24T08:58:34.167 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-24T08:58:34.167 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-24T08:58:34.167 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-24T08:58:34.167 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-24T08:58:34.167 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-24T08:58:34.167 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-24T08:58:34.167 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-24T08:58:34.167 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-24T08:58:34.167 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-24T08:58:34.167 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-24T08:58:34.167 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-24T08:58:34.167 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-24T08:58:34.167 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-24T08:58:34.167 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-24T08:58:34.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-24T08:58:34.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-24T08:58:34.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-24T08:58:34.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-24T08:58:34.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-24T08:58:34.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-24T08:58:34.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-24T08:58:34.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-24T08:58:34.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-24T08:58:34.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-24T08:58:34.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-24T08:58:34.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-24T08:58:34.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-24T08:58:34.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-24T08:58:34.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-24T08:58:34.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-24T08:58:34.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-24T08:58:34.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-24T08:58:34.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-24T08:58:34.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-24T08:58:34.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-24T08:58:34.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-24T08:58:34.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-24T08:58:34.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-24T08:58:34.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-24T08:58:34.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-24T08:58:34.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-24T08:58:34.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-24T08:58:34.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-24T08:58:34.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-24T08:58:34.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-24T08:58:34.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-24T08:58:34.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-24T08:58:34.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-24T08:58:34.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-24T08:58:34.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-24T08:58:34.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-24T08:58:34.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-24T08:58:34.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-24T08:58:34.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-24T08:58:34.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-24T08:58:34.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-24T08:58:34.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-24T08:58:34.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-24T08:58:34.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-24T08:58:34.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-24T08:58:34.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-24T08:58:34.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-24T08:58:34.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-24T08:58:34.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-24T08:58:34.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-24T08:58:34.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-24T08:58:34.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-24T08:58:34.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-24T08:58:34.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-24T08:58:34.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-24T08:58:34.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-24T08:58:34.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-24T08:58:34.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-24T08:58:34.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-24T08:58:34.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-24T08:58:34.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-24T08:58:34.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-24T08:58:34.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-24T08:58:34.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-24T08:58:34.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-24T08:58:34.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-24T08:58:34.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-24T08:58:34.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-24T08:58:34.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-24T08:58:34.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-24T08:58:34.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-24T08:58:34.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-24T08:58:34.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-24T08:58:34.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-24T08:58:34.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-24T08:58:34.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-24T08:58:34.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-24T08:58:34.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-24T08:58:34.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-24T08:58:34.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-24T08:58:34.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-24T08:58:34.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-24T08:58:34.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-24T08:58:34.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-24T08:58:34.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-24T08:58:34.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-24T08:58:34.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-24T08:58:34.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-24T08:58:34.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-24T08:58:34.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-24T08:58:34.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-24T08:58:34.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-24T08:58:34.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-24T08:58:34.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-24T08:58:34.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-24T08:58:34.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-24T08:58:34.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-24T08:58:34.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-24T08:58:34.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-24T08:58:34.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-24T08:58:34.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-24T08:58:34.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-24T08:58:34.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-24T08:58:34.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-24T08:58:34.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-24T08:58:34.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-24T08:58:34.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-24T08:58:34.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-24T08:58:34.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-24T08:58:34.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-24T08:58:34.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-24T08:58:34.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-24T08:58:34.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-24T08:58:34.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-24T08:58:34.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-24T08:58:34.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-24T08:58:34.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-24T08:58:34.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-24T08:58:34.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-24T08:58:34.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-24T08:58:34.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-24T08:58:34.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-24T08:58:34.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-24T08:58:34.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-24T08:58:34.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-24T08:58:34.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-24T08:58:34.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-24T08:58:34.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-24T08:58:34.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-24T08:58:34.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-24T08:58:34.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-24T08:58:34.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-24T08:58:34.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-24T08:58:34.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-24T08:58:34.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-24T08:58:34.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-24T08:58:34.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-24T08:58:34.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-24T08:58:34.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-24T08:58:34.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-24T08:58:34.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-24T08:58:34.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-24T08:58:34.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-24T08:58:34.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-24T08:58:34.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-24T08:58:34.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-24T08:58:34.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-24T08:58:34.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-24T08:58:34.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-24T08:58:34.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-24T08:58:34.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-24T08:58:34.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-24T08:58:34.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-24T08:58:34.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-24T08:58:34.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-24T08:58:34.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-24T08:58:34.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-24T08:58:34.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-24T08:58:34.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-24T08:58:34.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-24T08:58:34.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-24T08:58:34.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-24T08:58:34.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-24T08:58:34.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-24T08:58:34.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-24T08:58:34.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-24T08:58:34.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-24T08:58:34.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-24T08:58:34.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-24T08:58:34.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-24T08:58:34.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-24T08:58:34.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-24T08:58:34.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-24T08:58:34.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-24T08:58:34.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-24T08:58:34.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-24T08:58:34.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-24T08:58:34.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-24T08:58:34.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-24T08:58:34.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-24T08:58:34.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-24T08:58:34.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-24T08:58:34.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-24T08:58:34.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-24T08:58:34.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-24T08:58:34.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-24T08:58:34.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-24T08:58:34.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-24T08:58:34.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-24T08:58:34.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-24T08:58:34.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-24T08:58:34.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-24T08:58:34.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-24T08:58:34.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-24T08:58:34.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-24T08:58:34.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-24T08:58:34.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-24T08:58:34.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-24T08:58:34.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-24T08:58:34.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-24T08:58:34.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-24T08:58:34.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-24T08:58:34.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-24T08:58:34.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-24T08:58:34.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-24T08:58:34.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-24T08:58:34.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-24T08:58:34.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-24T08:58:34.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-24T08:58:34.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-24T08:58:34.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-24T08:58:34.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-24T08:58:34.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-24T08:58:34.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-24T08:58:34.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-24T08:58:34.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-24T08:58:34.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-24T08:58:34.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-24T08:58:34.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-24T08:58:34.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-24T08:58:34.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-24T08:58:34.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-24T08:58:34.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-24T08:58:34.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-24T08:58:34.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-24T08:58:34.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-24T08:58:34.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-24T08:58:34.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-24T08:58:34.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-24T08:58:34.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-24T08:58:34.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-24T08:58:34.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-24T08:58:34.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-24T08:58:34.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-24T08:58:34.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-24T08:58:34.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-24T08:58:34.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-24T08:58:34.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-24T08:58:34.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-24T08:58:34.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-24T08:58:34.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-24T08:58:34.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-24T08:58:34.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-24T08:58:34.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-24T08:58:34.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-24T08:58:34.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-24T08:58:34.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-24T08:58:34.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-24T08:58:34.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-24T08:58:34.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-24T08:58:34.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-24T08:58:34.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-24T08:58:34.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-24T08:58:34.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-24T08:58:34.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-24T08:58:34.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-24T08:58:34.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-24T08:58:34.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-24T08:58:34.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-24T08:58:34.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-24T08:58:34.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-24T08:58:34.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-24T08:58:34.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-24T08:58:34.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-24T08:58:34.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-24T08:58:34.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-24T08:58:34.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-24T08:58:34.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-24T08:58:34.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-24T08:58:34.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-24T08:58:34.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-24T08:58:34.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-24T08:58:34.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-24T08:58:34.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-24T08:58:34.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-24T08:58:34.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-24T08:58:34.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-24T08:58:34.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-24T08:58:34.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-24T08:58:34.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-24T08:58:34.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-24T08:58:34.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-24T08:58:34.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-24T08:58:34.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-24T08:58:34.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-24T08:58:34.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-24T08:58:34.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-24T08:58:34.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-24T08:58:34.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-24T08:58:34.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-24T08:58:34.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-24T08:58:34.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-24T08:58:34.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-24T08:58:34.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-24T08:58:34.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-24T08:58:34.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-24T08:58:34.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-24T08:58:34.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-24T08:58:34.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-24T08:58:34.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-24T08:58:34.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-24T08:58:34.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-24T08:58:34.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-24T08:58:34.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-24T08:58:34.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-24T08:58:34.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-24T08:58:34.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-24T08:58:34.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-24T08:58:34.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-24T08:58:34.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-24T08:58:34.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-24T08:58:34.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-24T08:58:34.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-24T08:58:34.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-24T08:58:34.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-24T08:58:34.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-24T08:58:34.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-24T08:58:34.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-24T08:58:34.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-24T08:58:34.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-24T08:58:34.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-24T08:58:34.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-24T08:58:34.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-24T08:58:34.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-24T08:58:34.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-24T08:58:34.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-24T08:58:34.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-24T08:58:34.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-24T08:58:34.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-24T08:58:34.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-24T08:58:34.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-24T08:58:34.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-24T08:58:34.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-24T08:58:34.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-24T08:58:34.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-24T08:58:34.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-24T08:58:34.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-24T08:58:34.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-24T08:58:34.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-24T08:58:34.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-24T08:58:34.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-24T08:58:34.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-24T08:58:34.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-24T08:58:34.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-24T08:58:34.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-24T08:58:34.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-24T08:58:34.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-24T08:58:34.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-24T08:58:34.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-24T08:58:34.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-24T08:58:34.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-24T08:58:34.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-24T08:58:34.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-24T08:58:34.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-24T08:58:34.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-24T08:58:34.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-24T08:58:34.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-24T08:58:34.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-24T08:58:34.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-24T08:58:34.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-24T08:58:34.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-24T08:58:34.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-24T08:58:34.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-24T08:58:34.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-24T08:58:34.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-24T08:58:34.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-24T08:58:34.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-24T08:58:34.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-24T08:58:34.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-24T08:58:34.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-24T08:58:34.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-24T08:58:34.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-24T08:58:34.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-24T08:58:34.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-24T08:58:34.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-24T08:58:34.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-24T08:58:34.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-24T08:58:34.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-24T08:58:34.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-24T08:58:34.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-24T08:58:34.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-24T08:58:34.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-24T08:58:34.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-24T08:58:34.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-24T08:58:34.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-24T08:58:34.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-24T08:58:34.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-24T08:58:34.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-24T08:58:34.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-24T08:58:34.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-24T08:58:34.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-24T08:58:34.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-24T08:58:34.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-24T08:58:34.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-24T08:58:34.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-24T08:58:34.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-24T08:58:34.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-24T08:58:34.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-24T08:58:34.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-24T08:58:34.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-24T08:58:34.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-24T08:58:34.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-24T08:58:34.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-24T08:58:34.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-24T08:58:34.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-24T08:58:34.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-24T08:58:34.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-24T08:58:34.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-24T08:58:34.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-24T08:58:34.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-24T08:58:34.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-24T08:58:34.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-24T08:58:34.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-24T08:58:34.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-24T08:58:34.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-24T08:58:34.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-24T08:58:34.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-24T08:58:34.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-24T08:58:34.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-24T08:58:34.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-24T08:58:34.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-24T08:58:34.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-24T08:58:34.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-24T08:58:34.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-24T08:58:34.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-24T08:58:34.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-24T08:58:34.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-24T08:58:34.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-24T08:58:34.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-24T08:58:34.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-24T08:58:34.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-24T08:58:34.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-24T08:58:34.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-24T08:58:34.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-24T08:58:34.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-24T08:58:34.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-24T08:58:34.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-24T08:58:34.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-24T08:58:34.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-24T08:58:34.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-24T08:58:34.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-24T08:58:34.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-24T08:58:34.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-24T08:58:34.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-24T08:58:34.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-24T08:58:34.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-24T08:58:34.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-24T08:58:34.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-24T08:58:34.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-24T08:58:34.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-24T08:58:34.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-24T08:58:34.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-24T08:58:34.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-24T08:58:34.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-24T08:58:34.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-24T08:58:34.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-24T08:58:34.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-24T08:58:34.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-24T08:58:34.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-24T08:58:34.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-24T08:58:34.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-24T08:58:34.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-24T08:58:34.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-24T08:58:34.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-24T08:58:34.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-24T08:58:34.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-24T08:58:34.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-24T08:58:34.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-24T08:58:34.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-24T08:58:34.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-24T08:58:34.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-24T08:58:34.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-24T08:58:34.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-24T08:58:34.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-24T08:58:34.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-24T08:58:34.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-24T08:58:34.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-24T08:58:34.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-24T08:58:34.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-24T08:58:34.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-24T08:58:34.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-24T08:58:34.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-24T08:58:34.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-24T08:58:34.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-24T08:58:34.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-24T08:58:34.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-24T08:58:34.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-24T08:58:34.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-24T08:58:34.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-24T08:58:34.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-24T08:58:34.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-24T08:58:34.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-24T08:58:34.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-24T08:58:34.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-24T08:58:34.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-24T08:58:34.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-24T08:58:34.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-24T08:58:34.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-24T08:58:34.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-24T08:58:34.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-24T08:58:34.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-24T08:58:34.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-24T08:58:34.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-24T08:58:34.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-24T08:58:34.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-24T08:58:34.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-24T08:58:34.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-24T08:58:34.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-24T08:58:34.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-24T08:58:34.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-24T08:58:34.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-24T08:58:34.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-24T08:58:34.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-24T08:58:34.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-24T08:58:34.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-24T08:58:34.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-24T08:58:34.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-24T08:58:34.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-24T08:58:34.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-24T08:58:34.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-24T08:58:34.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-24T08:58:34.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-24T08:58:34.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-24T08:58:34.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-24T08:58:34.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-24T08:58:34.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-24T08:58:34.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-24T08:58:34.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-24T08:58:34.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-24T08:58:34.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-24T08:58:34.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-24T08:58:34.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-24T08:58:34.204 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-24T08:58:34.204 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-24T08:58:34.204 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-24T08:58:34.204 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-24T08:58:34.204 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-24T08:58:34.208 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-24T08:58:34.208 DEBUG:teuthology.orchestra.run.smithi090:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-24T08:58:34.233 INFO:teuthology.orchestra.run.smithi090.stdout:postinstall scriptlet (using /bin/sh): 2024-04-24T08:58:34.233 INFO:teuthology.orchestra.run.smithi090.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-24T08:58:34.233 INFO:teuthology.orchestra.run.smithi090.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-24T08:58:34.233 INFO:teuthology.orchestra.run.smithi090.stdout:fi 2024-04-24T08:58:34.233 INFO:teuthology.orchestra.run.smithi090.stdout:for file in vmlinuz System.map config; do 2024-04-24T08:58:34.233 INFO:teuthology.orchestra.run.smithi090.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-24T08:58:34.233 INFO:teuthology.orchestra.run.smithi090.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-24T08:58:34.233 INFO:teuthology.orchestra.run.smithi090.stdout: fi 2024-04-24T08:58:34.233 INFO:teuthology.orchestra.run.smithi090.stdout:done 2024-04-24T08:58:34.234 INFO:teuthology.orchestra.run.smithi090.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-24T08:58:34.234 INFO:teuthology.orchestra.run.smithi090.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-24T08:58:34.234 INFO:teuthology.orchestra.run.smithi090.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-24T08:58:34.234 INFO:teuthology.orchestra.run.smithi090.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-24T08:58:34.234 INFO:teuthology.orchestra.run.smithi090.stdout:fi 2024-04-24T08:58:34.234 INFO:teuthology.orchestra.run.smithi090.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-24T08:58:34.234 INFO:teuthology.orchestra.run.smithi090.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-24T08:58:34.234 INFO:teuthology.orchestra.run.smithi090.stdout:fi 2024-04-24T08:58:34.234 INFO:teuthology.orchestra.run.smithi090.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-24T08:58:34.234 INFO:teuthology.orchestra.run.smithi090.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-24T08:58:34.234 INFO:teuthology.orchestra.run.smithi090.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-24T08:58:34.234 INFO:teuthology.orchestra.run.smithi090.stdout:fi 2024-04-24T08:58:34.235 DEBUG:teuthology.orchestra.run.smithi090:> sudo rpm -qi grub2-tools 2024-04-24T08:58:34.310 INFO:teuthology.orchestra.run.smithi090.stdout:Name : grub2-tools 2024-04-24T08:58:34.310 INFO:teuthology.orchestra.run.smithi090.stdout:Epoch : 1 2024-04-24T08:58:34.310 INFO:teuthology.orchestra.run.smithi090.stdout:Version : 2.06 2024-04-24T08:58:34.310 INFO:teuthology.orchestra.run.smithi090.stdout:Release : 68.el9 2024-04-24T08:58:34.310 INFO:teuthology.orchestra.run.smithi090.stdout:Architecture: x86_64 2024-04-24T08:58:34.310 INFO:teuthology.orchestra.run.smithi090.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-24T08:58:34.310 INFO:teuthology.orchestra.run.smithi090.stdout:Group : Unspecified 2024-04-24T08:58:34.310 INFO:teuthology.orchestra.run.smithi090.stdout:Size : 8264254 2024-04-24T08:58:34.310 INFO:teuthology.orchestra.run.smithi090.stdout:License : GPLv3+ 2024-04-24T08:58:34.311 INFO:teuthology.orchestra.run.smithi090.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-24T08:58:34.311 INFO:teuthology.orchestra.run.smithi090.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-24T08:58:34.311 INFO:teuthology.orchestra.run.smithi090.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-24T08:58:34.311 INFO:teuthology.orchestra.run.smithi090.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-24T08:58:34.311 INFO:teuthology.orchestra.run.smithi090.stdout:Packager : builder@centos.org 2024-04-24T08:58:34.311 INFO:teuthology.orchestra.run.smithi090.stdout:Vendor : CentOS 2024-04-24T08:58:34.311 INFO:teuthology.orchestra.run.smithi090.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-24T08:58:34.311 INFO:teuthology.orchestra.run.smithi090.stdout:Summary : Support tools for GRUB. 2024-04-24T08:58:34.311 INFO:teuthology.orchestra.run.smithi090.stdout:Description : 2024-04-24T08:58:34.311 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:58:34.311 INFO:teuthology.orchestra.run.smithi090.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-24T08:58:34.311 INFO:teuthology.orchestra.run.smithi090.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-24T08:58:34.311 INFO:teuthology.orchestra.run.smithi090.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-24T08:58:34.311 INFO:teuthology.orchestra.run.smithi090.stdout:hardware devices. 2024-04-24T08:58:34.311 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T08:58:34.311 INFO:teuthology.orchestra.run.smithi090.stdout:This subpackage provides tools for support of all platforms. 2024-04-24T08:58:34.312 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-24T08:58:34.313 INFO:teuthology.task.kernel:Updating grub on smithi090 to boot 6.9.0-rc3-g806ec36b2112 2024-04-24T08:58:34.313 DEBUG:teuthology.orchestra.run.smithi090:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-24T08:58:35.011 INFO:teuthology.orchestra.run.smithi090.stderr:Generating grub configuration file ... 2024-04-24T08:58:36.743 INFO:teuthology.orchestra.run.smithi090.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-24T08:58:36.777 INFO:teuthology.orchestra.run.smithi090.stderr:done 2024-04-24T08:58:36.780 DEBUG:teuthology.orchestra.run.smithi090:> mktemp 2024-04-24T08:58:36.805 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/tmp.H4r3MLdtfF 2024-04-24T08:58:36.805 DEBUG:teuthology.orchestra.run.smithi090:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.H4r3MLdtfF 2024-04-24T08:58:36.880 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 0666 /tmp/tmp.H4r3MLdtfF 2024-04-24T08:58:37.029 DEBUG:teuthology.orchestra.remote:smithi090:/tmp/tmp.H4r3MLdtfF is 6KB 2024-04-24T08:58:37.077 DEBUG:teuthology.orchestra.run.smithi090:> rm -fr /tmp/tmp.H4r3MLdtfF 2024-04-24T08:58:37.091 DEBUG:teuthology.orchestra.run.smithi090:> sudo /bin/ls /boot/loader/entries || true 2024-04-24T08:58:37.160 INFO:teuthology.orchestra.run.smithi090.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-24T08:58:37.160 INFO:teuthology.orchestra.run.smithi090.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-24T08:58:37.160 INFO:teuthology.orchestra.run.smithi090.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-24T08:58:37.160 INFO:teuthology.orchestra.run.smithi090.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-24T08:58:37.162 DEBUG:teuthology.orchestra.run.smithi090:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-24T08:58:37.304 DEBUG:teuthology.orchestra.run.smithi090:> sudo shutdown -r now 2024-04-24T08:59:07.307 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-24T08:59:07.308 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-04-24T08:59:07.309 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T08:59:25.854 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.90 2024-04-24T08:59:34.856 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-04-24T08:59:34.856 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T08:59:37.918 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.90 2024-04-24T08:59:49.919 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-04-24T08:59:49.920 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T08:59:52.990 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.90 2024-04-24T09:00:07.991 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-04-24T09:00:07.992 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T09:00:40.509 DEBUG:teuthology.orchestra.run.smithi090:> true 2024-04-24T09:00:41.265 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-04-24T09:00:41.266 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-04-24T09:00:41.266 DEBUG:teuthology.orchestra.run.smithi090:> uname -r 2024-04-24T09:00:41.312 INFO:teuthology.orchestra.run.smithi090.stdout:6.9.0-rc3-g806ec36b2112 2024-04-24T09:00:41.312 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-24T09:00:41.312 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-24T09:00:41.312 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-24T09:00:42.313 DEBUG:teuthology.parallel:result is None 2024-04-24T09:00:42.313 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-24T09:00:42.320 INFO:teuthology.task.internal:Creating test directory... 2024-04-24T09:00:42.320 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T09:00:42.328 DEBUG:teuthology.orchestra.run.smithi114:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T09:00:42.330 DEBUG:teuthology.orchestra.run.smithi149:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T09:00:42.368 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-24T09:00:42.378 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-24T09:00:42.387 INFO:teuthology.task.internal:Creating archive directory... 2024-04-24T09:00:42.387 DEBUG:teuthology.orchestra.run.smithi090:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T09:00:42.416 DEBUG:teuthology.orchestra.run.smithi114:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T09:00:42.418 DEBUG:teuthology.orchestra.run.smithi149:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T09:00:42.548 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-24T09:00:42.555 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-24T09:00:42.555 DEBUG:teuthology.orchestra.run.smithi114:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T09:00:42.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T09:00:42.571 DEBUG:teuthology.orchestra.run.smithi149:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T09:00:42.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T09:00:42.588 DEBUG:teuthology.orchestra.run.smithi090:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T09:00:42.593 DEBUG:teuthology.orchestra.run.smithi114:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T09:00:42.614 DEBUG:teuthology.orchestra.run.smithi149:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T09:00:42.646 INFO:teuthology.orchestra.run.smithi114.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T09:00:42.660 INFO:teuthology.orchestra.run.smithi114.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T09:00:42.692 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T09:00:42.713 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T09:00:42.721 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T09:00:42.780 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T09:00:42.784 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-24T09:00:42.791 INFO:teuthology.task.internal:Configuring sudo... 2024-04-24T09:00:42.791 DEBUG:teuthology.orchestra.run.smithi090:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T09:00:42.796 DEBUG:teuthology.orchestra.run.smithi114:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T09:00:42.798 DEBUG:teuthology.orchestra.run.smithi149:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T09:00:42.859 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-24T09:00:42.869 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-24T09:00:42.869 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T09:00:42.873 DEBUG:teuthology.orchestra.run.smithi114:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T09:00:42.876 DEBUG:teuthology.orchestra.run.smithi149:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T09:00:42.914 DEBUG:teuthology.orchestra.run.smithi090:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T09:00:43.029 DEBUG:teuthology.orchestra.run.smithi090:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T09:00:43.111 DEBUG:teuthology.orchestra.run.smithi090:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T09:00:43.187 DEBUG:teuthology.orchestra.run.smithi090:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T09:00:43.258 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-24T09:00:43.258 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T09:00:43.388 DEBUG:teuthology.orchestra.run.smithi114:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T09:00:43.433 DEBUG:teuthology.orchestra.run.smithi114:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T09:00:43.487 DEBUG:teuthology.orchestra.run.smithi114:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T09:00:43.566 DEBUG:teuthology.orchestra.run.smithi114:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T09:00:43.612 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-24T09:00:43.612 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T09:00:43.677 DEBUG:teuthology.orchestra.run.smithi149:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T09:00:43.724 DEBUG:teuthology.orchestra.run.smithi149:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T09:00:43.782 DEBUG:teuthology.orchestra.run.smithi149:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T09:00:43.861 DEBUG:teuthology.orchestra.run.smithi149:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T09:00:43.906 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-04-24T09:00:43.906 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T09:00:43.971 DEBUG:teuthology.orchestra.run.smithi090:> sudo service rsyslog restart 2024-04-24T09:00:43.976 DEBUG:teuthology.orchestra.run.smithi114:> sudo service rsyslog restart 2024-04-24T09:00:43.978 DEBUG:teuthology.orchestra.run.smithi149:> sudo service rsyslog restart 2024-04-24T09:00:44.012 INFO:teuthology.orchestra.run.smithi114.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T09:00:44.044 INFO:teuthology.orchestra.run.smithi149.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T09:00:44.084 INFO:teuthology.orchestra.run.smithi090.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T09:00:44.462 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-24T09:00:44.470 INFO:teuthology.task.internal:Starting timer... 2024-04-24T09:00:44.470 INFO:teuthology.run_tasks:Running task pcp... 2024-04-24T09:00:44.483 INFO:teuthology.run_tasks:Running task selinux... 2024-04-24T09:00:44.491 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-24T09:00:44.492 DEBUG:teuthology.orchestra.run.smithi090:> sudo service auditd rotate 2024-04-24T09:00:44.638 INFO:teuthology.orchestra.run.smithi090.stdout:Rotating logs: 2024-04-24T09:00:44.643 DEBUG:teuthology.orchestra.run.smithi114:> sudo service auditd rotate 2024-04-24T09:00:44.712 INFO:teuthology.orchestra.run.smithi114.stdout:Rotating logs: 2024-04-24T09:00:44.714 DEBUG:teuthology.orchestra.run.smithi149:> sudo service auditd rotate 2024-04-24T09:00:44.777 INFO:teuthology.orchestra.run.smithi149.stdout:Rotating logs: 2024-04-24T09:00:44.779 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-24T09:00:44.779 DEBUG:teuthology.orchestra.run.smithi090:> /usr/sbin/getenforce 2024-04-24T09:00:44.858 INFO:teuthology.orchestra.run.smithi090.stdout:Permissive 2024-04-24T09:00:44.858 DEBUG:teuthology.orchestra.run.smithi114:> /usr/sbin/getenforce 2024-04-24T09:00:44.892 INFO:teuthology.orchestra.run.smithi114.stdout:Permissive 2024-04-24T09:00:44.892 DEBUG:teuthology.orchestra.run.smithi149:> /usr/sbin/getenforce 2024-04-24T09:00:44.922 INFO:teuthology.orchestra.run.smithi149.stdout:Permissive 2024-04-24T09:00:44.923 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi090.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi114.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi149.front.sepia.ceph.com': 'permissive'} 2024-04-24T09:00:44.923 DEBUG:teuthology.orchestra.run.smithi090:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T09:00:45.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T09:00:45.001 DEBUG:teuthology.orchestra.run.smithi114:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T09:00:45.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T09:00:45.030 DEBUG:teuthology.orchestra.run.smithi149:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T09:00:45.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T09:00:45.055 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-24T09:00:45.055 DEBUG:teuthology.orchestra.run.smithi090:> sudo /usr/sbin/setenforce permissive 2024-04-24T09:00:45.147 DEBUG:teuthology.orchestra.run.smithi114:> sudo /usr/sbin/setenforce permissive 2024-04-24T09:00:45.172 DEBUG:teuthology.orchestra.run.smithi149:> sudo /usr/sbin/setenforce permissive 2024-04-24T09:00:45.200 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-24T09:00:45.209 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-24T09:00:45.215 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-24T09:00:45.367 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-24T09:00:45.384 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-24T09:00:45.385 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi090.front.sepia.ceph.com,smithi114.front.sepia.ceph.com,smithi149.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-24T09:05:53.585 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi090.front.sepia.ceph.com'), Remote(name='ubuntu@smithi114.front.sepia.ceph.com'), Remote(name='ubuntu@smithi149.front.sepia.ceph.com')] 2024-04-24T09:05:53.586 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-04-24T09:05:53.588 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T09:05:53.699 DEBUG:teuthology.orchestra.run.smithi090:> true 2024-04-24T09:05:53.851 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-04-24T09:05:53.851 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-04-24T09:05:53.852 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T09:05:53.930 DEBUG:teuthology.orchestra.run.smithi114:> true 2024-04-24T09:05:54.011 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-04-24T09:05:54.011 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-04-24T09:05:54.011 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T09:05:54.084 DEBUG:teuthology.orchestra.run.smithi149:> true 2024-04-24T09:05:54.165 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-04-24T09:05:54.166 INFO:teuthology.run_tasks:Running task clock... 2024-04-24T09:05:54.178 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-24T09:05:54.178 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T09:05:54.179 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T09:05:54.186 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T09:05:54.187 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T09:05:54.189 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T09:05:54.189 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T09:05:54.218 INFO:teuthology.orchestra.run.smithi114.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T09:05:54.233 INFO:teuthology.orchestra.run.smithi114.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T09:05:54.235 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T09:05:54.249 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T09:05:54.258 INFO:teuthology.orchestra.run.smithi114.stderr:sudo: ntpd: command not found 2024-04-24T09:05:54.269 INFO:teuthology.orchestra.run.smithi114.stdout:506 Cannot talk to daemon 2024-04-24T09:05:54.272 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T09:05:54.273 INFO:teuthology.orchestra.run.smithi149.stderr:sudo: ntpd: command not found 2024-04-24T09:05:54.282 INFO:teuthology.orchestra.run.smithi114.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T09:05:54.283 INFO:teuthology.orchestra.run.smithi149.stdout:506 Cannot talk to daemon 2024-04-24T09:05:54.295 INFO:teuthology.orchestra.run.smithi114.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T09:05:54.297 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T09:05:54.310 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T09:05:54.311 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T09:05:54.361 INFO:teuthology.orchestra.run.smithi114.stderr:bash: line 1: ntpq: command not found 2024-04-24T09:05:54.364 INFO:teuthology.orchestra.run.smithi114.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T09:05:54.364 INFO:teuthology.orchestra.run.smithi114.stdout:=============================================================================== 2024-04-24T09:05:54.365 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T09:05:54.365 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T09:05:54.365 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T09:05:54.365 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T09:05:54.365 INFO:teuthology.orchestra.run.smithi149.stderr:bash: line 1: ntpq: command not found 2024-04-24T09:05:54.368 INFO:teuthology.orchestra.run.smithi149.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T09:05:54.368 INFO:teuthology.orchestra.run.smithi149.stdout:=============================================================================== 2024-04-24T09:05:54.368 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T09:05:54.368 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T09:05:54.368 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T09:05:54.368 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T09:05:54.384 INFO:teuthology.orchestra.run.smithi090.stderr:sudo: ntpd: command not found 2024-04-24T09:05:54.415 INFO:teuthology.orchestra.run.smithi090.stdout:506 Cannot talk to daemon 2024-04-24T09:05:54.458 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T09:05:54.500 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T09:05:54.638 INFO:teuthology.orchestra.run.smithi090.stderr:bash: line 1: ntpq: command not found 2024-04-24T09:05:54.648 INFO:teuthology.orchestra.run.smithi090.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T09:05:54.648 INFO:teuthology.orchestra.run.smithi090.stdout:=============================================================================== 2024-04-24T09:05:54.648 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T09:05:54.648 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T09:05:54.648 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T09:05:54.648 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T09:05:54.649 INFO:teuthology.run_tasks:Running task install... 2024-04-24T09:05:54.658 DEBUG:teuthology.task.install:project ceph 2024-04-24T09:05:54.658 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1f81fb397ae98da7563d451a78c61574c8f4e6e0'}} 2024-04-24T09:05:54.658 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1f81fb397ae98da7563d451a78c61574c8f4e6e0'} 2024-04-24T09:05:54.658 INFO:teuthology.task.install:Using flavor: default 2024-04-24T09:05:54.666 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-24T09:05:54.666 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-24T09:05:54.666 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1f81fb397ae98da7563d451a78c61574c8f4e6e0', 'tag': None, 'wait_for_package': False} 2024-04-24T09:05:54.667 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T09:05:54.668 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1f81fb397ae98da7563d451a78c61574c8f4e6e0', 'tag': None, 'wait_for_package': False} 2024-04-24T09:05:54.669 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T09:05:54.670 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1f81fb397ae98da7563d451a78c61574c8f4e6e0', 'tag': None, 'wait_for_package': False} 2024-04-24T09:05:54.670 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T09:05:54.826 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/flavors/default/ 2024-04-24T09:05:54.826 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3116.g1f81fb39 2024-04-24T09:05:54.939 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T09:05:54.940 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-04-24T09:05:54.940 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T09:05:54.941 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/flavors/default/ 2024-04-24T09:05:54.942 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3116.g1f81fb39 2024-04-24T09:05:54.965 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-24T09:05:54.966 DEBUG:teuthology.orchestra.run.smithi149:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1f81fb397ae98da7563d451a78c61574c8f4e6e0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T09:05:54.969 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/flavors/default/ 2024-04-24T09:05:54.969 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3116.g1f81fb39 2024-04-24T09:05:55.031 DEBUG:teuthology.orchestra.run.smithi149:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T09:05:55.104 DEBUG:teuthology.orchestra.run.smithi149:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T09:05:55.136 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T09:05:55.137 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-24T09:05:55.137 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T09:05:55.156 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T09:05:55.156 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-24T09:05:55.156 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T09:05:55.168 INFO:teuthology.orchestra.run.smithi149.stdout:check_obsoletes = 1 2024-04-24T09:05:55.169 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum clean all 2024-04-24T09:05:55.183 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-24T09:05:55.183 DEBUG:teuthology.orchestra.run.smithi114:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1f81fb397ae98da7563d451a78c61574c8f4e6e0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T09:05:55.226 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-24T09:05:55.227 DEBUG:teuthology.orchestra.run.smithi090:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1f81fb397ae98da7563d451a78c61574c8f4e6e0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T09:05:55.250 DEBUG:teuthology.orchestra.run.smithi114:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T09:05:55.325 DEBUG:teuthology.orchestra.run.smithi114:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T09:05:55.356 DEBUG:teuthology.orchestra.run.smithi090:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T09:05:55.388 INFO:teuthology.orchestra.run.smithi114.stdout:check_obsoletes = 1 2024-04-24T09:05:55.390 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum clean all 2024-04-24T09:05:55.395 INFO:teuthology.orchestra.run.smithi149.stdout:48 files removed 2024-04-24T09:05:55.423 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-24T09:05:55.464 DEBUG:teuthology.orchestra.run.smithi090:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T09:05:55.540 INFO:teuthology.orchestra.run.smithi090.stdout:check_obsoletes = 1 2024-04-24T09:05:55.545 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum clean all 2024-04-24T09:05:55.615 INFO:teuthology.orchestra.run.smithi114.stdout:48 files removed 2024-04-24T09:05:55.641 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-24T09:05:55.860 INFO:teuthology.orchestra.run.smithi090.stdout:48 files removed 2024-04-24T09:05:55.898 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-24T09:05:56.247 INFO:teuthology.orchestra.run.smithi149.stdout:ceph packages for x86_64 149 kB/s | 83 kB 00:00 2024-04-24T09:05:56.396 INFO:teuthology.orchestra.run.smithi114.stdout:ceph packages for x86_64 168 kB/s | 83 kB 00:00 2024-04-24T09:05:56.573 INFO:teuthology.orchestra.run.smithi149.stdout:ceph noarch packages 42 kB/s | 12 kB 00:00 2024-04-24T09:05:56.756 INFO:teuthology.orchestra.run.smithi090.stdout:ceph packages for x86_64 193 kB/s | 83 kB 00:00 2024-04-24T09:05:57.173 INFO:teuthology.orchestra.run.smithi149.stdout:ceph source packages 3.4 kB/s | 1.9 kB 00:00 2024-04-24T09:05:57.280 INFO:teuthology.orchestra.run.smithi114.stdout:ceph noarch packages 14 kB/s | 12 kB 00:00 2024-04-24T09:05:57.864 INFO:teuthology.orchestra.run.smithi114.stdout:ceph source packages 3.5 kB/s | 1.9 kB 00:00 2024-04-24T09:05:58.357 INFO:teuthology.orchestra.run.smithi090.stdout:ceph noarch packages 7.7 kB/s | 12 kB 00:01 2024-04-24T09:05:59.557 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - BaseOS 3.4 MB/s | 8.0 MB 00:02 2024-04-24T09:06:00.201 INFO:teuthology.orchestra.run.smithi090.stdout:ceph source packages 1.1 kB/s | 1.9 kB 00:01 2024-04-24T09:06:00.530 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - BaseOS 3.1 MB/s | 8.0 MB 00:02 2024-04-24T09:06:01.191 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - BaseOS 8.9 MB/s | 8.0 MB 00:00 2024-04-24T09:06:03.332 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - AppStream 7.0 MB/s | 19 MB 00:02 2024-04-24T09:06:08.728 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - AppStream 3.0 MB/s | 19 MB 00:06 2024-04-24T09:06:14.261 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - CRB 3.9 MB/s | 6.2 MB 00:01 2024-04-24T09:06:15.982 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - CRB 699 kB/s | 6.2 MB 00:09 2024-04-24T09:06:15.986 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - Extras packages 48 kB/s | 16 kB 00:00 2024-04-24T09:06:18.059 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - Extras packages 18 kB/s | 16 kB 00:00 2024-04-24T09:06:18.270 INFO:teuthology.orchestra.run.smithi090.stdout:Extra Packages for Enterprise Linux 10 MB/s | 21 MB 00:02 2024-04-24T09:06:18.855 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 19 MB 00:17 2024-04-24T09:06:19.493 INFO:teuthology.orchestra.run.smithi149.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-04-24T09:06:23.064 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - CRB 9.6 MB/s | 6.2 MB 00:00 2024-04-24T09:06:24.140 INFO:teuthology.orchestra.run.smithi090.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-04-24T09:06:24.698 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-04-24T09:06:24.859 INFO:teuthology.orchestra.run.smithi149.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-04-24T09:06:25.883 INFO:teuthology.orchestra.run.smithi114.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-04-24T09:06:26.508 INFO:teuthology.orchestra.run.smithi090.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T09:06:26.510 INFO:teuthology.orchestra.run.smithi090.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T09:06:26.530 INFO:teuthology.orchestra.run.smithi090.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-24T09:06:26.531 INFO:teuthology.orchestra.run.smithi090.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-24T09:06:26.532 INFO:teuthology.orchestra.run.smithi090.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-24T09:06:26.534 INFO:teuthology.orchestra.run.smithi090.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-24T09:06:26.536 INFO:teuthology.orchestra.run.smithi090.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-24T09:06:26.539 INFO:teuthology.orchestra.run.smithi090.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-24T09:06:26.541 INFO:teuthology.orchestra.run.smithi090.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-24T09:06:26.611 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-24T09:06:26.645 INFO:teuthology.orchestra.run.smithi090.stdout:======================================================================================= 2024-04-24T09:06:26.645 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2024-04-24T09:06:26.645 INFO:teuthology.orchestra.run.smithi090.stdout:======================================================================================= 2024-04-24T09:06:26.645 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-04-24T09:06:26.645 INFO:teuthology.orchestra.run.smithi090.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-24T09:06:26.645 INFO:teuthology.orchestra.run.smithi090.stdout: ceph x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 6.6 k 2024-04-24T09:06:26.646 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-base x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 5.4 M 2024-04-24T09:06:26.646 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-fuse x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 890 k 2024-04-24T09:06:26.646 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 145 k 2024-04-24T09:06:26.646 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 1.5 M 2024-04-24T09:06:26.646 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 142 k 2024-04-24T09:06:26.646 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 3.5 M 2024-04-24T09:06:26.646 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 7.4 M 2024-04-24T09:06:26.646 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-rook noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 49 k 2024-04-24T09:06:26.646 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-radosgw x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 11 M 2024-04-24T09:06:26.646 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-test x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 49 M 2024-04-24T09:06:26.646 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-volume noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 273 k 2024-04-24T09:06:26.646 INFO:teuthology.orchestra.run.smithi090.stdout: cephadm noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 762 k 2024-04-24T09:06:26.647 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-mirror x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 225 k 2024-04-24T09:06:26.647 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-shell x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 27 k 2024-04-24T09:06:26.647 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-top noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 18 k 2024-04-24T09:06:26.647 INFO:teuthology.orchestra.run.smithi090.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-24T09:06:26.647 INFO:teuthology.orchestra.run.smithi090.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-24T09:06:26.647 INFO:teuthology.orchestra.run.smithi090.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-24T09:06:26.647 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs-devel x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 32 k 2024-04-24T09:06:26.647 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs2 x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 746 k 2024-04-24T09:06:26.647 INFO:teuthology.orchestra.run.smithi090.stdout: librados-devel x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 127 k 2024-04-24T09:06:26.647 INFO:teuthology.orchestra.run.smithi090.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-24T09:06:26.647 INFO:teuthology.orchestra.run.smithi090.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-24T09:06:26.647 INFO:teuthology.orchestra.run.smithi090.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-24T09:06:26.647 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-24T09:06:26.647 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-24T09:06:26.648 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-24T09:06:26.648 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cephfs x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 161 k 2024-04-24T09:06:26.648 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rados x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 320 k 2024-04-24T09:06:26.648 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rbd x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 298 k 2024-04-24T09:06:26.648 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rgw x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 99 k 2024-04-24T09:06:26.648 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-fuse x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 85 k 2024-04-24T09:06:26.648 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-mirror x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 3.1 M 2024-04-24T09:06:26.648 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-nbd x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 170 k 2024-04-24T09:06:26.648 INFO:teuthology.orchestra.run.smithi090.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-24T09:06:26.648 INFO:teuthology.orchestra.run.smithi090.stdout:Upgrading: 2024-04-24T09:06:26.648 INFO:teuthology.orchestra.run.smithi090.stdout: librados2 x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 3.4 M 2024-04-24T09:06:26.648 INFO:teuthology.orchestra.run.smithi090.stdout: librbd1 x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 3.1 M 2024-04-24T09:06:26.648 INFO:teuthology.orchestra.run.smithi090.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-24T09:06:26.649 INFO:teuthology.orchestra.run.smithi090.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-24T09:06:26.649 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2024-04-24T09:06:26.649 INFO:teuthology.orchestra.run.smithi090.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-24T09:06:26.649 INFO:teuthology.orchestra.run.smithi090.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-24T09:06:26.649 INFO:teuthology.orchestra.run.smithi090.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-24T09:06:26.649 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-common x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 21 M 2024-04-24T09:06:26.649 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 34 k 2024-04-24T09:06:26.649 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mds x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 2.4 M 2024-04-24T09:06:26.649 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 249 k 2024-04-24T09:06:26.649 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mon x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 4.7 M 2024-04-24T09:06:26.649 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-osd x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 17 M 2024-04-24T09:06:26.649 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 16 k 2024-04-24T09:06:26.649 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-selinux x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 25 k 2024-04-24T09:06:26.650 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T09:06:26.650 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T09:06:26.650 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T09:06:26.650 INFO:teuthology.orchestra.run.smithi090.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T09:06:26.650 INFO:teuthology.orchestra.run.smithi090.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T09:06:26.650 INFO:teuthology.orchestra.run.smithi090.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-24T09:06:26.650 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-24T09:06:26.650 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-24T09:06:26.650 INFO:teuthology.orchestra.run.smithi090.stdout: libcephsqlite x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 163 k 2024-04-24T09:06:26.650 INFO:teuthology.orchestra.run.smithi090.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T09:06:26.650 INFO:teuthology.orchestra.run.smithi090.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T09:06:26.650 INFO:teuthology.orchestra.run.smithi090.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T09:06:26.650 INFO:teuthology.orchestra.run.smithi090.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T09:06:26.651 INFO:teuthology.orchestra.run.smithi090.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T09:06:26.651 INFO:teuthology.orchestra.run.smithi090.stdout: libradosstriper1 x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 501 k 2024-04-24T09:06:26.651 INFO:teuthology.orchestra.run.smithi090.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T09:06:26.651 INFO:teuthology.orchestra.run.smithi090.stdout: librgw2 x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 5.4 M 2024-04-24T09:06:26.651 INFO:teuthology.orchestra.run.smithi090.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T09:06:26.651 INFO:teuthology.orchestra.run.smithi090.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-24T09:06:26.651 INFO:teuthology.orchestra.run.smithi090.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T09:06:26.651 INFO:teuthology.orchestra.run.smithi090.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-24T09:06:26.651 INFO:teuthology.orchestra.run.smithi090.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-24T09:06:26.651 INFO:teuthology.orchestra.run.smithi090.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-24T09:06:26.651 INFO:teuthology.orchestra.run.smithi090.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T09:06:26.651 INFO:teuthology.orchestra.run.smithi090.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T09:06:26.651 INFO:teuthology.orchestra.run.smithi090.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-24T09:06:26.651 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-24T09:06:26.652 INFO:teuthology.orchestra.run.smithi090.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-24T09:06:26.652 INFO:teuthology.orchestra.run.smithi090.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-24T09:06:26.652 INFO:teuthology.orchestra.run.smithi090.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T09:06:26.652 INFO:teuthology.orchestra.run.smithi090.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-24T09:06:26.652 INFO:teuthology.orchestra.run.smithi090.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T09:06:26.652 INFO:teuthology.orchestra.run.smithi090.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T09:06:26.652 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T09:06:26.652 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-argparse x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 45 k 2024-04-24T09:06:26.652 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-common x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 135 k 2024-04-24T09:06:26.652 INFO:teuthology.orchestra.run.smithi090.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T09:06:26.652 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T09:06:26.652 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T09:06:26.652 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T09:06:26.652 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-24T09:06:26.653 INFO:teuthology.orchestra.run.smithi090.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-24T09:06:26.653 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T09:06:26.653 INFO:teuthology.orchestra.run.smithi090.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T09:06:26.653 INFO:teuthology.orchestra.run.smithi090.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T09:06:26.653 INFO:teuthology.orchestra.run.smithi090.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-24T09:06:26.653 INFO:teuthology.orchestra.run.smithi090.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-24T09:06:26.653 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T09:06:26.653 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T09:06:26.653 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T09:06:26.653 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T09:06:26.653 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T09:06:26.653 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T09:06:26.653 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T09:06:26.654 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T09:06:26.654 INFO:teuthology.orchestra.run.smithi090.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T09:06:26.654 INFO:teuthology.orchestra.run.smithi090.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T09:06:26.654 INFO:teuthology.orchestra.run.smithi090.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T09:06:26.654 INFO:teuthology.orchestra.run.smithi090.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T09:06:26.654 INFO:teuthology.orchestra.run.smithi090.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T09:06:26.654 INFO:teuthology.orchestra.run.smithi090.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T09:06:26.654 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T09:06:26.654 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T09:06:26.654 INFO:teuthology.orchestra.run.smithi090.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T09:06:26.654 INFO:teuthology.orchestra.run.smithi090.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-24T09:06:26.654 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T09:06:26.654 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T09:06:26.654 INFO:teuthology.orchestra.run.smithi090.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T09:06:26.655 INFO:teuthology.orchestra.run.smithi090.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T09:06:26.655 INFO:teuthology.orchestra.run.smithi090.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-24T09:06:26.655 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T09:06:26.655 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T09:06:26.655 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T09:06:26.655 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T09:06:26.655 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-24T09:06:26.655 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T09:06:26.658 INFO:teuthology.orchestra.run.smithi090.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T09:06:26.658 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T09:06:26.658 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T09:06:26.658 INFO:teuthology.orchestra.run.smithi090.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T09:06:26.658 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T09:06:26.658 INFO:teuthology.orchestra.run.smithi090.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T09:06:26.658 INFO:teuthology.orchestra.run.smithi090.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T09:06:26.659 INFO:teuthology.orchestra.run.smithi090.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T09:06:26.659 INFO:teuthology.orchestra.run.smithi090.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T09:06:26.659 INFO:teuthology.orchestra.run.smithi090.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T09:06:26.659 INFO:teuthology.orchestra.run.smithi090.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-24T09:06:26.659 INFO:teuthology.orchestra.run.smithi090.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T09:06:26.659 INFO:teuthology.orchestra.run.smithi090.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T09:06:26.659 INFO:teuthology.orchestra.run.smithi090.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T09:06:26.659 INFO:teuthology.orchestra.run.smithi090.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T09:06:26.659 INFO:teuthology.orchestra.run.smithi090.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-24T09:06:26.659 INFO:teuthology.orchestra.run.smithi090.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-24T09:06:26.659 INFO:teuthology.orchestra.run.smithi090.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-24T09:06:26.659 INFO:teuthology.orchestra.run.smithi090.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T09:06:26.659 INFO:teuthology.orchestra.run.smithi090.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T09:06:26.659 INFO:teuthology.orchestra.run.smithi090.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-24T09:06:26.659 INFO:teuthology.orchestra.run.smithi090.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T09:06:26.660 INFO:teuthology.orchestra.run.smithi090.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-24T09:06:26.660 INFO:teuthology.orchestra.run.smithi090.stdout:Installing weak dependencies: 2024-04-24T09:06:26.660 INFO:teuthology.orchestra.run.smithi090.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-24T09:06:26.660 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:06:26.660 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-24T09:06:26.660 INFO:teuthology.orchestra.run.smithi090.stdout:======================================================================================= 2024-04-24T09:06:26.660 INFO:teuthology.orchestra.run.smithi090.stdout:Install 148 Packages 2024-04-24T09:06:26.660 INFO:teuthology.orchestra.run.smithi090.stdout:Upgrade 4 Packages 2024-04-24T09:06:26.660 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:06:26.661 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 222 M 2024-04-24T09:06:26.661 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-24T09:06:26.977 INFO:teuthology.orchestra.run.smithi149.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T09:06:26.978 INFO:teuthology.orchestra.run.smithi149.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T09:06:26.988 INFO:teuthology.orchestra.run.smithi149.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-24T09:06:26.989 INFO:teuthology.orchestra.run.smithi149.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-24T09:06:26.990 INFO:teuthology.orchestra.run.smithi149.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-24T09:06:26.990 INFO:teuthology.orchestra.run.smithi149.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-24T09:06:26.992 INFO:teuthology.orchestra.run.smithi149.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-24T09:06:26.994 INFO:teuthology.orchestra.run.smithi149.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-24T09:06:26.995 INFO:teuthology.orchestra.run.smithi149.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-24T09:06:27.055 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-04-24T09:06:27.074 INFO:teuthology.orchestra.run.smithi149.stdout:======================================================================================= 2024-04-24T09:06:27.074 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2024-04-24T09:06:27.074 INFO:teuthology.orchestra.run.smithi149.stdout:======================================================================================= 2024-04-24T09:06:27.074 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-04-24T09:06:27.074 INFO:teuthology.orchestra.run.smithi149.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-24T09:06:27.074 INFO:teuthology.orchestra.run.smithi149.stdout: ceph x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 6.6 k 2024-04-24T09:06:27.075 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-base x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 5.4 M 2024-04-24T09:06:27.075 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-fuse x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 890 k 2024-04-24T09:06:27.075 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 145 k 2024-04-24T09:06:27.075 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 1.5 M 2024-04-24T09:06:27.075 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 142 k 2024-04-24T09:06:27.075 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 3.5 M 2024-04-24T09:06:27.075 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 7.4 M 2024-04-24T09:06:27.075 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-rook noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 49 k 2024-04-24T09:06:27.075 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-radosgw x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 11 M 2024-04-24T09:06:27.075 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-test x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 49 M 2024-04-24T09:06:27.075 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-volume noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 273 k 2024-04-24T09:06:27.075 INFO:teuthology.orchestra.run.smithi149.stdout: cephadm noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 762 k 2024-04-24T09:06:27.075 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-mirror x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 225 k 2024-04-24T09:06:27.075 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-shell x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 27 k 2024-04-24T09:06:27.076 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-top noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 18 k 2024-04-24T09:06:27.076 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-24T09:06:27.076 INFO:teuthology.orchestra.run.smithi149.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-24T09:06:27.076 INFO:teuthology.orchestra.run.smithi149.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-24T09:06:27.076 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs-devel x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 32 k 2024-04-24T09:06:27.076 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs2 x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 746 k 2024-04-24T09:06:27.076 INFO:teuthology.orchestra.run.smithi149.stdout: librados-devel x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 127 k 2024-04-24T09:06:27.076 INFO:teuthology.orchestra.run.smithi149.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-24T09:06:27.076 INFO:teuthology.orchestra.run.smithi149.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-24T09:06:27.076 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-24T09:06:27.076 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-24T09:06:27.076 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-24T09:06:27.076 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-24T09:06:27.076 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cephfs x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 161 k 2024-04-24T09:06:27.077 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rados x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 320 k 2024-04-24T09:06:27.077 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rbd x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 298 k 2024-04-24T09:06:27.077 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rgw x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 99 k 2024-04-24T09:06:27.077 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-fuse x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 85 k 2024-04-24T09:06:27.077 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-mirror x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 3.1 M 2024-04-24T09:06:27.077 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-nbd x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 170 k 2024-04-24T09:06:27.077 INFO:teuthology.orchestra.run.smithi149.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-24T09:06:27.077 INFO:teuthology.orchestra.run.smithi149.stdout:Upgrading: 2024-04-24T09:06:27.077 INFO:teuthology.orchestra.run.smithi149.stdout: librados2 x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 3.4 M 2024-04-24T09:06:27.077 INFO:teuthology.orchestra.run.smithi149.stdout: librbd1 x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 3.1 M 2024-04-24T09:06:27.077 INFO:teuthology.orchestra.run.smithi149.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-24T09:06:27.077 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-24T09:06:27.077 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-04-24T09:06:27.077 INFO:teuthology.orchestra.run.smithi149.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-24T09:06:27.078 INFO:teuthology.orchestra.run.smithi149.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-24T09:06:27.078 INFO:teuthology.orchestra.run.smithi149.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-24T09:06:27.078 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-common x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 21 M 2024-04-24T09:06:27.078 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 34 k 2024-04-24T09:06:27.078 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mds x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 2.4 M 2024-04-24T09:06:27.078 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 249 k 2024-04-24T09:06:27.078 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mon x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 4.7 M 2024-04-24T09:06:27.078 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-osd x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 17 M 2024-04-24T09:06:27.078 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 16 k 2024-04-24T09:06:27.078 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-selinux x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 25 k 2024-04-24T09:06:27.078 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T09:06:27.078 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T09:06:27.078 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T09:06:27.079 INFO:teuthology.orchestra.run.smithi149.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T09:06:27.079 INFO:teuthology.orchestra.run.smithi149.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T09:06:27.079 INFO:teuthology.orchestra.run.smithi149.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-24T09:06:27.079 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-24T09:06:27.079 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-24T09:06:27.079 INFO:teuthology.orchestra.run.smithi149.stdout: libcephsqlite x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 163 k 2024-04-24T09:06:27.079 INFO:teuthology.orchestra.run.smithi149.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T09:06:27.079 INFO:teuthology.orchestra.run.smithi149.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T09:06:27.079 INFO:teuthology.orchestra.run.smithi149.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T09:06:27.079 INFO:teuthology.orchestra.run.smithi149.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T09:06:27.079 INFO:teuthology.orchestra.run.smithi149.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T09:06:27.079 INFO:teuthology.orchestra.run.smithi149.stdout: libradosstriper1 x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 501 k 2024-04-24T09:06:27.079 INFO:teuthology.orchestra.run.smithi149.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T09:06:27.079 INFO:teuthology.orchestra.run.smithi149.stdout: librgw2 x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 5.4 M 2024-04-24T09:06:27.080 INFO:teuthology.orchestra.run.smithi149.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T09:06:27.080 INFO:teuthology.orchestra.run.smithi149.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-24T09:06:27.080 INFO:teuthology.orchestra.run.smithi149.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T09:06:27.080 INFO:teuthology.orchestra.run.smithi149.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-24T09:06:27.080 INFO:teuthology.orchestra.run.smithi149.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-24T09:06:27.080 INFO:teuthology.orchestra.run.smithi149.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-24T09:06:27.080 INFO:teuthology.orchestra.run.smithi149.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T09:06:27.080 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T09:06:27.080 INFO:teuthology.orchestra.run.smithi149.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-24T09:06:27.080 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-24T09:06:27.080 INFO:teuthology.orchestra.run.smithi149.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-24T09:06:27.080 INFO:teuthology.orchestra.run.smithi149.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-24T09:06:27.080 INFO:teuthology.orchestra.run.smithi149.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T09:06:27.080 INFO:teuthology.orchestra.run.smithi149.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-24T09:06:27.081 INFO:teuthology.orchestra.run.smithi149.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T09:06:27.081 INFO:teuthology.orchestra.run.smithi149.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T09:06:27.081 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T09:06:27.081 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-argparse x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 45 k 2024-04-24T09:06:27.081 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-common x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 135 k 2024-04-24T09:06:27.081 INFO:teuthology.orchestra.run.smithi149.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T09:06:27.081 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T09:06:27.081 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T09:06:27.081 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T09:06:27.081 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-24T09:06:27.081 INFO:teuthology.orchestra.run.smithi149.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-24T09:06:27.081 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T09:06:27.081 INFO:teuthology.orchestra.run.smithi149.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T09:06:27.081 INFO:teuthology.orchestra.run.smithi149.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T09:06:27.082 INFO:teuthology.orchestra.run.smithi149.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-24T09:06:27.082 INFO:teuthology.orchestra.run.smithi149.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-24T09:06:27.082 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T09:06:27.082 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T09:06:27.082 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T09:06:27.082 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T09:06:27.082 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T09:06:27.082 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T09:06:27.082 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T09:06:27.082 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T09:06:27.082 INFO:teuthology.orchestra.run.smithi149.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T09:06:27.082 INFO:teuthology.orchestra.run.smithi149.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T09:06:27.082 INFO:teuthology.orchestra.run.smithi149.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T09:06:27.083 INFO:teuthology.orchestra.run.smithi149.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T09:06:27.083 INFO:teuthology.orchestra.run.smithi149.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T09:06:27.083 INFO:teuthology.orchestra.run.smithi149.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T09:06:27.083 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T09:06:27.083 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T09:06:27.083 INFO:teuthology.orchestra.run.smithi149.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T09:06:27.083 INFO:teuthology.orchestra.run.smithi149.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-24T09:06:27.083 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T09:06:27.083 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T09:06:27.083 INFO:teuthology.orchestra.run.smithi149.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T09:06:27.083 INFO:teuthology.orchestra.run.smithi149.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T09:06:27.083 INFO:teuthology.orchestra.run.smithi149.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-24T09:06:27.083 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T09:06:27.083 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T09:06:27.084 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T09:06:27.084 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T09:06:27.084 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-24T09:06:27.084 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T09:06:27.084 INFO:teuthology.orchestra.run.smithi149.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T09:06:27.084 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T09:06:27.084 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T09:06:27.084 INFO:teuthology.orchestra.run.smithi149.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T09:06:27.084 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T09:06:27.084 INFO:teuthology.orchestra.run.smithi149.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T09:06:27.084 INFO:teuthology.orchestra.run.smithi149.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T09:06:27.084 INFO:teuthology.orchestra.run.smithi149.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T09:06:27.084 INFO:teuthology.orchestra.run.smithi149.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T09:06:27.084 INFO:teuthology.orchestra.run.smithi149.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T09:06:27.085 INFO:teuthology.orchestra.run.smithi149.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-24T09:06:27.085 INFO:teuthology.orchestra.run.smithi149.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T09:06:27.085 INFO:teuthology.orchestra.run.smithi149.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T09:06:27.085 INFO:teuthology.orchestra.run.smithi149.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T09:06:27.085 INFO:teuthology.orchestra.run.smithi149.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T09:06:27.085 INFO:teuthology.orchestra.run.smithi149.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-24T09:06:27.085 INFO:teuthology.orchestra.run.smithi149.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-24T09:06:27.085 INFO:teuthology.orchestra.run.smithi149.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-24T09:06:27.085 INFO:teuthology.orchestra.run.smithi149.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T09:06:27.085 INFO:teuthology.orchestra.run.smithi149.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T09:06:27.085 INFO:teuthology.orchestra.run.smithi149.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-24T09:06:27.085 INFO:teuthology.orchestra.run.smithi149.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T09:06:27.085 INFO:teuthology.orchestra.run.smithi149.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-24T09:06:27.085 INFO:teuthology.orchestra.run.smithi149.stdout:Installing weak dependencies: 2024-04-24T09:06:27.086 INFO:teuthology.orchestra.run.smithi149.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-24T09:06:27.086 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T09:06:27.086 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-04-24T09:06:27.086 INFO:teuthology.orchestra.run.smithi149.stdout:======================================================================================= 2024-04-24T09:06:27.086 INFO:teuthology.orchestra.run.smithi149.stdout:Install 148 Packages 2024-04-24T09:06:27.086 INFO:teuthology.orchestra.run.smithi149.stdout:Upgrade 4 Packages 2024-04-24T09:06:27.086 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T09:06:27.086 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 222 M 2024-04-24T09:06:27.087 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-04-24T09:06:27.660 INFO:teuthology.orchestra.run.smithi090.stdout:(1/152): ceph-19.0.0-3116.g1f81fb39.el9.x86_64. 21 kB/s | 6.6 kB 00:00 2024-04-24T09:06:28.041 INFO:teuthology.orchestra.run.smithi090.stdout:(2/152): ceph-base-19.0.0-3116.g1f81fb39.el9.x8 7.7 MB/s | 5.4 MB 00:00 2024-04-24T09:06:28.235 INFO:teuthology.orchestra.run.smithi090.stdout:(3/152): ceph-fuse-19.0.0-3116.g1f81fb39.el9.x8 1.5 MB/s | 890 kB 00:00 2024-04-24T09:06:28.327 INFO:teuthology.orchestra.run.smithi090.stdout:(4/152): ceph-immutable-object-cache-19.0.0-311 508 kB/s | 145 kB 00:00 2024-04-24T09:06:28.477 INFO:teuthology.orchestra.run.smithi149.stdout:(1/152): ceph-19.0.0-3116.g1f81fb39.el9.x86_64. 13 kB/s | 6.6 kB 00:00 2024-04-24T09:06:28.505 INFO:teuthology.orchestra.run.smithi090.stdout:(5/152): ceph-mgr-19.0.0-3116.g1f81fb39.el9.x86 8.4 MB/s | 1.5 MB 00:00 2024-04-24T09:06:28.786 INFO:teuthology.orchestra.run.smithi149.stdout:(2/152): ceph-base-19.0.0-3116.g1f81fb39.el9.x8 6.7 MB/s | 5.4 MB 00:00 2024-04-24T09:06:28.911 INFO:teuthology.orchestra.run.smithi149.stdout:(3/152): ceph-immutable-object-cache-19.0.0-311 1.1 MB/s | 145 kB 00:00 2024-04-24T09:06:29.178 INFO:teuthology.orchestra.run.smithi090.stdout:(6/152): ceph-mon-19.0.0-3116.g1f81fb39.el9.x86 7.0 MB/s | 4.7 MB 00:00 2024-04-24T09:06:29.253 INFO:teuthology.orchestra.run.smithi149.stdout:(4/152): ceph-mds-19.0.0-3116.g1f81fb39.el9.x86 6.9 MB/s | 2.4 MB 00:00 2024-04-24T09:06:29.261 INFO:teuthology.orchestra.run.smithi090.stdout:(7/152): ceph-mds-19.0.0-3116.g1f81fb39.el9.x86 2.3 MB/s | 2.4 MB 00:01 2024-04-24T09:06:29.528 INFO:teuthology.orchestra.run.smithi149.stdout:(5/152): ceph-mgr-19.0.0-3116.g1f81fb39.el9.x86 5.4 MB/s | 1.5 MB 00:00 2024-04-24T09:06:30.070 INFO:teuthology.orchestra.run.smithi149.stdout:(6/152): ceph-fuse-19.0.0-3116.g1f81fb39.el9.x8 559 kB/s | 890 kB 00:01 2024-04-24T09:06:30.237 INFO:teuthology.orchestra.run.smithi149.stdout:(7/152): ceph-mon-19.0.0-3116.g1f81fb39.el9.x86 6.6 MB/s | 4.7 MB 00:00 2024-04-24T09:06:30.737 INFO:teuthology.orchestra.run.smithi090.stdout:(8/152): ceph-osd-19.0.0-3116.g1f81fb39.el9.x86 11 MB/s | 17 MB 00:01 2024-04-24T09:06:30.812 INFO:teuthology.orchestra.run.smithi090.stdout:(9/152): ceph-selinux-19.0.0-3116.g1f81fb39.el9 340 kB/s | 25 kB 00:00 2024-04-24T09:06:31.017 INFO:teuthology.orchestra.run.smithi114.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-04-24T09:06:31.171 INFO:teuthology.orchestra.run.smithi149.stdout:(8/152): ceph-radosgw-19.0.0-3116.g1f81fb39.el9 11 MB/s | 11 MB 00:00 2024-04-24T09:06:31.329 INFO:teuthology.orchestra.run.smithi090.stdout:(10/152): ceph-common-19.0.0-3116.g1f81fb39.el9 5.3 MB/s | 21 MB 00:03 2024-04-24T09:06:31.462 INFO:teuthology.orchestra.run.smithi149.stdout:(9/152): ceph-selinux-19.0.0-3116.g1f81fb39.el9 86 kB/s | 25 kB 00:00 2024-04-24T09:06:31.871 INFO:teuthology.orchestra.run.smithi090.stdout:(11/152): cephfs-mirror-19.0.0-3116.g1f81fb39.e 416 kB/s | 225 kB 00:00 2024-04-24T09:06:31.988 INFO:teuthology.orchestra.run.smithi090.stdout:(12/152): cephfs-shell-19.0.0-3116.g1f81fb39.el 233 kB/s | 27 kB 00:00 2024-04-24T09:06:32.322 INFO:teuthology.orchestra.run.smithi090.stdout:(13/152): libcephfs-devel-19.0.0-3116.g1f81fb39 95 kB/s | 32 kB 00:00 2024-04-24T09:06:32.939 INFO:teuthology.orchestra.run.smithi090.stdout:(14/152): libcephfs2-19.0.0-3116.g1f81fb39.el9. 1.2 MB/s | 746 kB 00:00 2024-04-24T09:06:33.132 INFO:teuthology.orchestra.run.smithi090.stdout:(15/152): libcephsqlite-19.0.0-3116.g1f81fb39.e 852 kB/s | 163 kB 00:00 2024-04-24T09:06:33.150 INFO:teuthology.orchestra.run.smithi114.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T09:06:33.151 INFO:teuthology.orchestra.run.smithi114.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T09:06:33.162 INFO:teuthology.orchestra.run.smithi114.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-24T09:06:33.162 INFO:teuthology.orchestra.run.smithi114.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-24T09:06:33.163 INFO:teuthology.orchestra.run.smithi114.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-24T09:06:33.164 INFO:teuthology.orchestra.run.smithi114.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-24T09:06:33.166 INFO:teuthology.orchestra.run.smithi114.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-24T09:06:33.168 INFO:teuthology.orchestra.run.smithi114.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-24T09:06:33.169 INFO:teuthology.orchestra.run.smithi114.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-24T09:06:33.229 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-04-24T09:06:33.238 INFO:teuthology.orchestra.run.smithi114.stdout:======================================================================================= 2024-04-24T09:06:33.238 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repository Size 2024-04-24T09:06:33.238 INFO:teuthology.orchestra.run.smithi114.stdout:======================================================================================= 2024-04-24T09:06:33.239 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-04-24T09:06:33.239 INFO:teuthology.orchestra.run.smithi114.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-24T09:06:33.239 INFO:teuthology.orchestra.run.smithi114.stdout: ceph x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 6.6 k 2024-04-24T09:06:33.239 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-base x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 5.4 M 2024-04-24T09:06:33.239 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-fuse x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 890 k 2024-04-24T09:06:33.239 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 145 k 2024-04-24T09:06:33.239 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 1.5 M 2024-04-24T09:06:33.239 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 142 k 2024-04-24T09:06:33.239 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 3.5 M 2024-04-24T09:06:33.239 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 7.4 M 2024-04-24T09:06:33.239 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-rook noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 49 k 2024-04-24T09:06:33.239 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-radosgw x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 11 M 2024-04-24T09:06:33.239 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-test x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 49 M 2024-04-24T09:06:33.239 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-volume noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 273 k 2024-04-24T09:06:33.240 INFO:teuthology.orchestra.run.smithi114.stdout: cephadm noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 762 k 2024-04-24T09:06:33.240 INFO:teuthology.orchestra.run.smithi114.stdout: cephfs-mirror x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 225 k 2024-04-24T09:06:33.240 INFO:teuthology.orchestra.run.smithi114.stdout: cephfs-shell x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 27 k 2024-04-24T09:06:33.240 INFO:teuthology.orchestra.run.smithi114.stdout: cephfs-top noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 18 k 2024-04-24T09:06:33.240 INFO:teuthology.orchestra.run.smithi114.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-24T09:06:33.240 INFO:teuthology.orchestra.run.smithi114.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-24T09:06:33.240 INFO:teuthology.orchestra.run.smithi114.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-24T09:06:33.240 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs-devel x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 32 k 2024-04-24T09:06:33.240 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs2 x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 746 k 2024-04-24T09:06:33.240 INFO:teuthology.orchestra.run.smithi114.stdout: librados-devel x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 127 k 2024-04-24T09:06:33.240 INFO:teuthology.orchestra.run.smithi114.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-24T09:06:33.240 INFO:teuthology.orchestra.run.smithi114.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-24T09:06:33.240 INFO:teuthology.orchestra.run.smithi114.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-24T09:06:33.241 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-24T09:06:33.241 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-24T09:06:33.241 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-24T09:06:33.241 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cephfs x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 161 k 2024-04-24T09:06:33.241 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rados x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 320 k 2024-04-24T09:06:33.241 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rbd x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 298 k 2024-04-24T09:06:33.241 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rgw x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 99 k 2024-04-24T09:06:33.241 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-fuse x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 85 k 2024-04-24T09:06:33.241 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-mirror x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 3.1 M 2024-04-24T09:06:33.241 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-nbd x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 170 k 2024-04-24T09:06:33.241 INFO:teuthology.orchestra.run.smithi114.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-24T09:06:33.241 INFO:teuthology.orchestra.run.smithi114.stdout:Upgrading: 2024-04-24T09:06:33.241 INFO:teuthology.orchestra.run.smithi114.stdout: librados2 x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 3.4 M 2024-04-24T09:06:33.241 INFO:teuthology.orchestra.run.smithi114.stdout: librbd1 x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 3.1 M 2024-04-24T09:06:33.242 INFO:teuthology.orchestra.run.smithi114.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-24T09:06:33.242 INFO:teuthology.orchestra.run.smithi114.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-24T09:06:33.242 INFO:teuthology.orchestra.run.smithi114.stdout:Installing dependencies: 2024-04-24T09:06:33.242 INFO:teuthology.orchestra.run.smithi114.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-24T09:06:33.242 INFO:teuthology.orchestra.run.smithi114.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-24T09:06:33.242 INFO:teuthology.orchestra.run.smithi114.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-24T09:06:33.242 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-common x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 21 M 2024-04-24T09:06:33.242 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 34 k 2024-04-24T09:06:33.242 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mds x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 2.4 M 2024-04-24T09:06:33.242 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 249 k 2024-04-24T09:06:33.242 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mon x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 4.7 M 2024-04-24T09:06:33.242 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-osd x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 17 M 2024-04-24T09:06:33.242 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 16 k 2024-04-24T09:06:33.243 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-selinux x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 25 k 2024-04-24T09:06:33.243 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T09:06:33.243 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T09:06:33.243 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T09:06:33.243 INFO:teuthology.orchestra.run.smithi114.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T09:06:33.243 INFO:teuthology.orchestra.run.smithi114.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T09:06:33.243 INFO:teuthology.orchestra.run.smithi114.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-24T09:06:33.243 INFO:teuthology.orchestra.run.smithi114.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-24T09:06:33.243 INFO:teuthology.orchestra.run.smithi114.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-24T09:06:33.243 INFO:teuthology.orchestra.run.smithi114.stdout: libcephsqlite x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 163 k 2024-04-24T09:06:33.243 INFO:teuthology.orchestra.run.smithi114.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T09:06:33.243 INFO:teuthology.orchestra.run.smithi114.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T09:06:33.243 INFO:teuthology.orchestra.run.smithi114.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T09:06:33.243 INFO:teuthology.orchestra.run.smithi114.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T09:06:33.244 INFO:teuthology.orchestra.run.smithi114.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T09:06:33.244 INFO:teuthology.orchestra.run.smithi114.stdout: libradosstriper1 x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 501 k 2024-04-24T09:06:33.244 INFO:teuthology.orchestra.run.smithi114.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T09:06:33.244 INFO:teuthology.orchestra.run.smithi114.stdout: librgw2 x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 5.4 M 2024-04-24T09:06:33.244 INFO:teuthology.orchestra.run.smithi114.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T09:06:33.244 INFO:teuthology.orchestra.run.smithi114.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-24T09:06:33.244 INFO:teuthology.orchestra.run.smithi114.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T09:06:33.244 INFO:teuthology.orchestra.run.smithi114.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-24T09:06:33.244 INFO:teuthology.orchestra.run.smithi114.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-24T09:06:33.244 INFO:teuthology.orchestra.run.smithi114.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-24T09:06:33.244 INFO:teuthology.orchestra.run.smithi114.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T09:06:33.244 INFO:teuthology.orchestra.run.smithi114.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T09:06:33.244 INFO:teuthology.orchestra.run.smithi114.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-24T09:06:33.245 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-24T09:06:33.245 INFO:teuthology.orchestra.run.smithi114.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-24T09:06:33.245 INFO:teuthology.orchestra.run.smithi114.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-24T09:06:33.245 INFO:teuthology.orchestra.run.smithi114.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T09:06:33.245 INFO:teuthology.orchestra.run.smithi114.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-24T09:06:33.245 INFO:teuthology.orchestra.run.smithi114.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T09:06:33.245 INFO:teuthology.orchestra.run.smithi114.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T09:06:33.245 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T09:06:33.245 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-argparse x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 45 k 2024-04-24T09:06:33.245 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-common x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 135 k 2024-04-24T09:06:33.245 INFO:teuthology.orchestra.run.smithi114.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T09:06:33.245 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T09:06:33.245 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T09:06:33.245 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T09:06:33.246 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-24T09:06:33.246 INFO:teuthology.orchestra.run.smithi114.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-24T09:06:33.246 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T09:06:33.246 INFO:teuthology.orchestra.run.smithi114.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T09:06:33.246 INFO:teuthology.orchestra.run.smithi114.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T09:06:33.246 INFO:teuthology.orchestra.run.smithi114.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-24T09:06:33.246 INFO:teuthology.orchestra.run.smithi114.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-24T09:06:33.246 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T09:06:33.246 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T09:06:33.246 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T09:06:33.246 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T09:06:33.246 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T09:06:33.246 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T09:06:33.247 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T09:06:33.247 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T09:06:33.247 INFO:teuthology.orchestra.run.smithi114.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T09:06:33.247 INFO:teuthology.orchestra.run.smithi114.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T09:06:33.247 INFO:teuthology.orchestra.run.smithi114.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T09:06:33.247 INFO:teuthology.orchestra.run.smithi114.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T09:06:33.247 INFO:teuthology.orchestra.run.smithi114.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T09:06:33.247 INFO:teuthology.orchestra.run.smithi114.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T09:06:33.247 INFO:teuthology.orchestra.run.smithi114.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T09:06:33.247 INFO:teuthology.orchestra.run.smithi114.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T09:06:33.247 INFO:teuthology.orchestra.run.smithi114.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T09:06:33.247 INFO:teuthology.orchestra.run.smithi114.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-24T09:06:33.247 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T09:06:33.247 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T09:06:33.247 INFO:teuthology.orchestra.run.smithi114.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T09:06:33.248 INFO:teuthology.orchestra.run.smithi114.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T09:06:33.248 INFO:teuthology.orchestra.run.smithi114.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-24T09:06:33.248 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T09:06:33.248 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T09:06:33.248 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T09:06:33.248 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T09:06:33.248 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-24T09:06:33.248 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T09:06:33.248 INFO:teuthology.orchestra.run.smithi114.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T09:06:33.248 INFO:teuthology.orchestra.run.smithi114.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T09:06:33.248 INFO:teuthology.orchestra.run.smithi114.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T09:06:33.248 INFO:teuthology.orchestra.run.smithi114.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T09:06:33.248 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T09:06:33.248 INFO:teuthology.orchestra.run.smithi114.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T09:06:33.248 INFO:teuthology.orchestra.run.smithi114.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T09:06:33.249 INFO:teuthology.orchestra.run.smithi114.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T09:06:33.249 INFO:teuthology.orchestra.run.smithi114.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T09:06:33.249 INFO:teuthology.orchestra.run.smithi114.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T09:06:33.249 INFO:teuthology.orchestra.run.smithi114.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-24T09:06:33.249 INFO:teuthology.orchestra.run.smithi114.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T09:06:33.249 INFO:teuthology.orchestra.run.smithi114.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T09:06:33.249 INFO:teuthology.orchestra.run.smithi114.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T09:06:33.249 INFO:teuthology.orchestra.run.smithi114.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T09:06:33.249 INFO:teuthology.orchestra.run.smithi114.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-24T09:06:33.249 INFO:teuthology.orchestra.run.smithi114.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-24T09:06:33.249 INFO:teuthology.orchestra.run.smithi114.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-24T09:06:33.249 INFO:teuthology.orchestra.run.smithi114.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T09:06:33.249 INFO:teuthology.orchestra.run.smithi114.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T09:06:33.250 INFO:teuthology.orchestra.run.smithi114.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-24T09:06:33.250 INFO:teuthology.orchestra.run.smithi114.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T09:06:33.250 INFO:teuthology.orchestra.run.smithi114.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-24T09:06:33.250 INFO:teuthology.orchestra.run.smithi114.stdout:Installing weak dependencies: 2024-04-24T09:06:33.250 INFO:teuthology.orchestra.run.smithi114.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-24T09:06:33.250 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-24T09:06:33.250 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-04-24T09:06:33.250 INFO:teuthology.orchestra.run.smithi114.stdout:======================================================================================= 2024-04-24T09:06:33.250 INFO:teuthology.orchestra.run.smithi114.stdout:Install 148 Packages 2024-04-24T09:06:33.250 INFO:teuthology.orchestra.run.smithi114.stdout:Upgrade 4 Packages 2024-04-24T09:06:33.250 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-24T09:06:33.251 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 222 M 2024-04-24T09:06:33.251 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-04-24T09:06:33.290 INFO:teuthology.orchestra.run.smithi090.stdout:(16/152): librados-devel-19.0.0-3116.g1f81fb39. 804 kB/s | 127 kB 00:00 2024-04-24T09:06:33.691 INFO:teuthology.orchestra.run.smithi090.stdout:(17/152): libradosstriper1-19.0.0-3116.g1f81fb3 1.2 MB/s | 501 kB 00:00 2024-04-24T09:06:34.260 INFO:teuthology.orchestra.run.smithi114.stdout:(1/152): ceph-19.0.0-3116.g1f81fb39.el9.x86_64. 31 kB/s | 6.6 kB 00:00 2024-04-24T09:06:34.508 INFO:teuthology.orchestra.run.smithi090.stdout:(18/152): ceph-test-19.0.0-3116.g1f81fb39.el9.x 13 MB/s | 49 MB 00:03 2024-04-24T09:06:34.583 INFO:teuthology.orchestra.run.smithi090.stdout:(19/152): python3-ceph-argparse-19.0.0-3116.g1f 606 kB/s | 45 kB 00:00 2024-04-24T09:06:34.667 INFO:teuthology.orchestra.run.smithi090.stdout:(20/152): python3-ceph-common-19.0.0-3116.g1f81 1.6 MB/s | 135 kB 00:00 2024-04-24T09:06:34.750 INFO:teuthology.orchestra.run.smithi090.stdout:(21/152): python3-cephfs-19.0.0-3116.g1f81fb39. 1.9 MB/s | 161 kB 00:00 2024-04-24T09:06:34.842 INFO:teuthology.orchestra.run.smithi090.stdout:(22/152): python3-rados-19.0.0-3116.g1f81fb39.e 3.4 MB/s | 320 kB 00:00 2024-04-24T09:06:34.926 INFO:teuthology.orchestra.run.smithi090.stdout:(23/152): python3-rbd-19.0.0-3116.g1f81fb39.el9 3.5 MB/s | 298 kB 00:00 2024-04-24T09:06:34.976 INFO:teuthology.orchestra.run.smithi114.stdout:(2/152): ceph-fuse-19.0.0-3116.g1f81fb39.el9.x8 1.2 MB/s | 890 kB 00:00 2024-04-24T09:06:35.010 INFO:teuthology.orchestra.run.smithi090.stdout:(24/152): python3-rgw-19.0.0-3116.g1f81fb39.el9 1.2 MB/s | 99 kB 00:00 2024-04-24T09:06:35.093 INFO:teuthology.orchestra.run.smithi090.stdout:(25/152): rbd-fuse-19.0.0-3116.g1f81fb39.el9.x8 1.0 MB/s | 85 kB 00:00 2024-04-24T09:06:35.152 INFO:teuthology.orchestra.run.smithi114.stdout:(3/152): ceph-immutable-object-cache-19.0.0-311 827 kB/s | 145 kB 00:00 2024-04-24T09:06:35.560 INFO:teuthology.orchestra.run.smithi090.stdout:(26/152): rbd-mirror-19.0.0-3116.g1f81fb39.el9. 6.6 MB/s | 3.1 MB 00:00 2024-04-24T09:06:35.661 INFO:teuthology.orchestra.run.smithi090.stdout:(27/152): rbd-nbd-19.0.0-3116.g1f81fb39.el9.x86 1.7 MB/s | 170 kB 00:00 2024-04-24T09:06:35.744 INFO:teuthology.orchestra.run.smithi090.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3116.g 414 kB/s | 34 kB 00:00 2024-04-24T09:06:35.898 INFO:teuthology.orchestra.run.smithi090.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3116.g1f81fb3 930 kB/s | 142 kB 00:00 2024-04-24T09:06:36.210 INFO:teuthology.orchestra.run.smithi114.stdout:(4/152): ceph-common-19.0.0-3116.g1f81fb39.el9. 9.8 MB/s | 21 MB 00:02 2024-04-24T09:06:36.220 INFO:teuthology.orchestra.run.smithi149.stdout:(10/152): ceph-test-19.0.0-3116.g1f81fb39.el9.x 10 MB/s | 49 MB 00:04 2024-04-24T09:06:36.253 INFO:teuthology.orchestra.run.smithi090.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3116.g1f81f 10 MB/s | 3.5 MB 00:00 2024-04-24T09:06:36.294 INFO:teuthology.orchestra.run.smithi114.stdout:(5/152): ceph-mds-19.0.0-3116.g1f81fb39.el9.x86 2.1 MB/s | 2.4 MB 00:01 2024-04-24T09:06:36.321 INFO:teuthology.orchestra.run.smithi149.stdout:(11/152): cephfs-mirror-19.0.0-3116.g1f81fb39.e 2.2 MB/s | 225 kB 00:00 2024-04-24T09:06:36.336 INFO:teuthology.orchestra.run.smithi114.stdout:(6/152): ceph-mgr-19.0.0-3116.g1f81fb39.el9.x86 12 MB/s | 1.5 MB 00:00 2024-04-24T09:06:36.379 INFO:teuthology.orchestra.run.smithi149.stdout:(12/152): cephfs-shell-19.0.0-3116.g1f81fb39.el 463 kB/s | 27 kB 00:00 2024-04-24T09:06:36.479 INFO:teuthology.orchestra.run.smithi149.stdout:(13/152): libcephfs-devel-19.0.0-3116.g1f81fb39 318 kB/s | 32 kB 00:00 2024-04-24T09:06:36.964 INFO:teuthology.orchestra.run.smithi149.stdout:(14/152): libcephfs2-19.0.0-3116.g1f81fb39.el9. 1.5 MB/s | 746 kB 00:00 2024-04-24T09:06:37.055 INFO:teuthology.orchestra.run.smithi149.stdout:(15/152): libcephsqlite-19.0.0-3116.g1f81fb39.e 1.7 MB/s | 163 kB 00:00 2024-04-24T09:06:37.156 INFO:teuthology.orchestra.run.smithi149.stdout:(16/152): librados-devel-19.0.0-3116.g1f81fb39. 1.2 MB/s | 127 kB 00:00 2024-04-24T09:06:37.331 INFO:teuthology.orchestra.run.smithi149.stdout:(17/152): libradosstriper1-19.0.0-3116.g1f81fb3 2.8 MB/s | 501 kB 00:00 2024-04-24T09:06:37.462 INFO:teuthology.orchestra.run.smithi090.stdout:(31/152): ceph-mgr-diskprediction-local-19.0.0- 6.1 MB/s | 7.4 MB 00:01 2024-04-24T09:06:37.511 INFO:teuthology.orchestra.run.smithi114.stdout:(7/152): ceph-mon-19.0.0-3116.g1f81fb39.el9.x86 3.9 MB/s | 4.7 MB 00:01 2024-04-24T09:06:37.537 INFO:teuthology.orchestra.run.smithi090.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3116.g1f 3.3 MB/s | 249 kB 00:00 2024-04-24T09:06:37.629 INFO:teuthology.orchestra.run.smithi090.stdout:(33/152): ceph-mgr-rook-19.0.0-3116.g1f81fb39.e 543 kB/s | 49 kB 00:00 2024-04-24T09:06:37.713 INFO:teuthology.orchestra.run.smithi090.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3116.g1 200 kB/s | 16 kB 00:00 2024-04-24T09:06:37.778 INFO:teuthology.orchestra.run.smithi114.stdout:(8/152): ceph-osd-19.0.0-3116.g1f81fb39.el9.x86 12 MB/s | 17 MB 00:01 2024-04-24T09:06:37.780 INFO:teuthology.orchestra.run.smithi090.stdout:(35/152): ceph-volume-19.0.0-3116.g1f81fb39.el9 4.0 MB/s | 273 kB 00:00 2024-04-24T09:06:37.828 INFO:teuthology.orchestra.run.smithi114.stdout:(9/152): ceph-selinux-19.0.0-3116.g1f81fb39.el9 505 kB/s | 25 kB 00:00 2024-04-24T09:06:37.906 INFO:teuthology.orchestra.run.smithi090.stdout:(36/152): cephadm-19.0.0-3116.g1f81fb39.el9.noa 6.0 MB/s | 762 kB 00:00 2024-04-24T09:06:38.006 INFO:teuthology.orchestra.run.smithi090.stdout:(37/152): cephfs-top-19.0.0-3116.g1f81fb39.el9. 184 kB/s | 18 kB 00:00 2024-04-24T09:06:38.131 INFO:teuthology.orchestra.run.smithi090.stdout:(38/152): librgw2-19.0.0-3116.g1f81fb39.el9.x86 1.2 MB/s | 5.4 MB 00:04 2024-04-24T09:06:38.364 INFO:teuthology.orchestra.run.smithi149.stdout:(18/152): librgw2-19.0.0-3116.g1f81fb39.el9.x86 5.2 MB/s | 5.4 MB 00:01 2024-04-24T09:06:38.440 INFO:teuthology.orchestra.run.smithi149.stdout:(19/152): python3-ceph-argparse-19.0.0-3116.g1f 604 kB/s | 45 kB 00:00 2024-04-24T09:06:38.523 INFO:teuthology.orchestra.run.smithi149.stdout:(20/152): python3-ceph-common-19.0.0-3116.g1f81 1.6 MB/s | 135 kB 00:00 2024-04-24T09:06:38.607 INFO:teuthology.orchestra.run.smithi149.stdout:(21/152): python3-cephfs-19.0.0-3116.g1f81fb39. 1.9 MB/s | 161 kB 00:00 2024-04-24T09:06:38.724 INFO:teuthology.orchestra.run.smithi149.stdout:(22/152): python3-rados-19.0.0-3116.g1f81fb39.e 2.7 MB/s | 320 kB 00:00 2024-04-24T09:06:38.779 INFO:teuthology.orchestra.run.smithi114.stdout:(10/152): ceph-radosgw-19.0.0-3116.g1f81fb39.el 8.4 MB/s | 11 MB 00:01 2024-04-24T09:06:38.799 INFO:teuthology.orchestra.run.smithi149.stdout:(23/152): python3-rbd-19.0.0-3116.g1f81fb39.el9 3.9 MB/s | 298 kB 00:00 2024-04-24T09:06:38.808 INFO:teuthology.orchestra.run.smithi090.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 100 kB/s | 80 kB 00:00 2024-04-24T09:06:38.858 INFO:teuthology.orchestra.run.smithi149.stdout:(24/152): python3-rgw-19.0.0-3116.g1f81fb39.el9 1.7 MB/s | 99 kB 00:00 2024-04-24T09:06:38.870 INFO:teuthology.orchestra.run.smithi114.stdout:(11/152): ceph-base-19.0.0-3116.g1f81fb39.el9.x 1.1 MB/s | 5.4 MB 00:04 2024-04-24T09:06:38.896 INFO:teuthology.orchestra.run.smithi114.stdout:(12/152): cephfs-mirror-19.0.0-3116.g1f81fb39.e 1.9 MB/s | 225 kB 00:00 2024-04-24T09:06:38.938 INFO:teuthology.orchestra.run.smithi114.stdout:(13/152): cephfs-shell-19.0.0-3116.g1f81fb39.el 405 kB/s | 27 kB 00:00 2024-04-24T09:06:38.963 INFO:teuthology.orchestra.run.smithi114.stdout:(14/152): libcephfs-devel-19.0.0-3116.g1f81fb39 474 kB/s | 32 kB 00:00 2024-04-24T09:06:39.038 INFO:teuthology.orchestra.run.smithi114.stdout:(15/152): libcephsqlite-19.0.0-3116.g1f81fb39.e 2.1 MB/s | 163 kB 00:00 2024-04-24T09:06:39.113 INFO:teuthology.orchestra.run.smithi114.stdout:(16/152): librados-devel-19.0.0-3116.g1f81fb39. 1.7 MB/s | 127 kB 00:00 2024-04-24T09:06:39.150 INFO:teuthology.orchestra.run.smithi149.stdout:(25/152): rbd-fuse-19.0.0-3116.g1f81fb39.el9.x8 292 kB/s | 85 kB 00:00 2024-04-24T09:06:39.206 INFO:teuthology.orchestra.run.smithi114.stdout:(17/152): libradosstriper1-19.0.0-3116.g1f81fb3 5.3 MB/s | 501 kB 00:00 2024-04-24T09:06:39.456 INFO:teuthology.orchestra.run.smithi114.stdout:(18/152): libcephfs2-19.0.0-3116.g1f81fb39.el9. 1.4 MB/s | 746 kB 00:00 2024-04-24T09:06:39.564 INFO:teuthology.orchestra.run.smithi114.stdout:(19/152): python3-ceph-argparse-19.0.0-3116.g1f 417 kB/s | 45 kB 00:00 2024-04-24T09:06:39.717 INFO:teuthology.orchestra.run.smithi090.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 215 kB/s | 195 kB 00:00 2024-04-24T09:06:39.740 INFO:teuthology.orchestra.run.smithi114.stdout:(20/152): python3-ceph-common-19.0.0-3116.g1f81 769 kB/s | 135 kB 00:00 2024-04-24T09:06:39.823 INFO:teuthology.orchestra.run.smithi114.stdout:(21/152): librgw2-19.0.0-3116.g1f81fb39.el9.x86 8.7 MB/s | 5.4 MB 00:00 2024-04-24T09:06:39.875 INFO:teuthology.orchestra.run.smithi149.stdout:(26/152): rbd-mirror-19.0.0-3116.g1f81fb39.el9. 4.3 MB/s | 3.1 MB 00:00 2024-04-24T09:06:39.899 INFO:teuthology.orchestra.run.smithi114.stdout:(22/152): python3-cephfs-19.0.0-3116.g1f81fb39. 1.0 MB/s | 161 kB 00:00 2024-04-24T09:06:39.900 INFO:teuthology.orchestra.run.smithi090.stdout:(41/152): libgfortran-11.4.1-3.el9.x86_64.rpm 454 kB/s | 803 kB 00:01 2024-04-24T09:06:39.927 INFO:teuthology.orchestra.run.smithi114.stdout:(23/152): python3-rados-19.0.0-3116.g1f81fb39.e 3.1 MB/s | 320 kB 00:00 2024-04-24T09:06:39.933 INFO:teuthology.orchestra.run.smithi149.stdout:(27/152): rbd-nbd-19.0.0-3116.g1f81fb39.el9.x86 2.9 MB/s | 170 kB 00:00 2024-04-24T09:06:39.999 INFO:teuthology.orchestra.run.smithi114.stdout:(24/152): python3-rgw-19.0.0-3116.g1f81fb39.el9 1.3 MB/s | 99 kB 00:00 2024-04-24T09:06:40.000 INFO:teuthology.orchestra.run.smithi149.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3116.g 514 kB/s | 34 kB 00:00 2024-04-24T09:06:40.067 INFO:teuthology.orchestra.run.smithi149.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3116.g1f81fb3 2.1 MB/s | 142 kB 00:00 2024-04-24T09:06:40.091 INFO:teuthology.orchestra.run.smithi114.stdout:(25/152): python3-rbd-19.0.0-3116.g1f81fb39.el9 1.5 MB/s | 298 kB 00:00 2024-04-24T09:06:40.234 INFO:teuthology.orchestra.run.smithi090.stdout:(42/152): ceph-radosgw-19.0.0-3116.g1f81fb39.el 989 kB/s | 11 MB 00:10 2024-04-24T09:06:40.333 INFO:teuthology.orchestra.run.smithi114.stdout:(26/152): rbd-fuse-19.0.0-3116.g1f81fb39.el9.x8 255 kB/s | 85 kB 00:00 2024-04-24T09:06:40.425 INFO:teuthology.orchestra.run.smithi114.stdout:(27/152): rbd-nbd-19.0.0-3116.g1f81fb39.el9.x86 1.8 MB/s | 170 kB 00:00 2024-04-24T09:06:40.509 INFO:teuthology.orchestra.run.smithi114.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3116.g 411 kB/s | 34 kB 00:00 2024-04-24T09:06:40.592 INFO:teuthology.orchestra.run.smithi114.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3116.g1f81fb3 1.7 MB/s | 142 kB 00:00 2024-04-24T09:06:40.809 INFO:teuthology.orchestra.run.smithi149.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3116.g1f81f 4.8 MB/s | 3.5 MB 00:00 2024-04-24T09:06:40.859 INFO:teuthology.orchestra.run.smithi090.stdout:(43/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 222 kB/s | 253 kB 00:01 2024-04-24T09:06:40.984 INFO:teuthology.orchestra.run.smithi114.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3116.g1f81f 9.0 MB/s | 3.5 MB 00:00 2024-04-24T09:06:41.176 INFO:teuthology.orchestra.run.smithi090.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 122 kB/s | 106 kB 00:00 2024-04-24T09:06:41.726 INFO:teuthology.orchestra.run.smithi114.stdout:(31/152): ceph-test-19.0.0-3116.g1f81fb39.el9.x 13 MB/s | 49 MB 00:03 2024-04-24T09:06:41.757 INFO:teuthology.orchestra.run.smithi090.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 151 kB/s | 135 kB 00:00 2024-04-24T09:06:41.776 INFO:teuthology.orchestra.run.smithi114.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3116.g1f 4.9 MB/s | 249 kB 00:00 2024-04-24T09:06:41.860 INFO:teuthology.orchestra.run.smithi114.stdout:(33/152): ceph-mgr-diskprediction-local-19.0.0- 8.5 MB/s | 7.4 MB 00:00 2024-04-24T09:06:41.885 INFO:teuthology.orchestra.run.smithi114.stdout:(34/152): ceph-mgr-rook-19.0.0-3116.g1f81fb39.e 455 kB/s | 49 kB 00:00 2024-04-24T09:06:41.919 INFO:teuthology.orchestra.run.smithi114.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3116.g1 281 kB/s | 16 kB 00:00 2024-04-24T09:06:41.944 INFO:teuthology.orchestra.run.smithi114.stdout:(36/152): ceph-volume-19.0.0-3116.g1f81fb39.el9 4.5 MB/s | 273 kB 00:00 2024-04-24T09:06:41.995 INFO:teuthology.orchestra.run.smithi114.stdout:(37/152): rbd-mirror-19.0.0-3116.g1f81fb39.el9. 1.6 MB/s | 3.1 MB 00:01 2024-04-24T09:06:42.020 INFO:teuthology.orchestra.run.smithi114.stdout:(38/152): cephfs-top-19.0.0-3116.g1f81fb39.el9. 242 kB/s | 18 kB 00:00 2024-04-24T09:06:42.062 INFO:teuthology.orchestra.run.smithi114.stdout:(39/152): cephadm-19.0.0-3116.g1f81fb39.el9.noa 5.2 MB/s | 762 kB 00:00 2024-04-24T09:06:42.085 INFO:teuthology.orchestra.run.smithi090.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 138 kB/s | 125 kB 00:00 2024-04-24T09:06:42.170 INFO:teuthology.orchestra.run.smithi114.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 458 kB/s | 80 kB 00:00 2024-04-24T09:06:42.184 INFO:teuthology.orchestra.run.smithi149.stdout:(31/152): ceph-mgr-diskprediction-local-19.0.0- 5.4 MB/s | 7.4 MB 00:01 2024-04-24T09:06:42.251 INFO:teuthology.orchestra.run.smithi149.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3116.g1f 3.7 MB/s | 249 kB 00:00 2024-04-24T09:06:42.262 INFO:teuthology.orchestra.run.smithi114.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 976 kB/s | 195 kB 00:00 2024-04-24T09:06:42.287 INFO:teuthology.orchestra.run.smithi114.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.1 MB/s | 253 kB 00:00 2024-04-24T09:06:42.293 INFO:teuthology.orchestra.run.smithi149.stdout:(33/152): ceph-mgr-rook-19.0.0-3116.g1f81fb39.e 1.2 MB/s | 49 kB 00:00 2024-04-24T09:06:42.321 INFO:teuthology.orchestra.run.smithi114.stdout:(43/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.6 MB/s | 803 kB 00:00 2024-04-24T09:06:42.335 INFO:teuthology.orchestra.run.smithi149.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3116.g1 395 kB/s | 16 kB 00:00 2024-04-24T09:06:42.346 INFO:teuthology.orchestra.run.smithi114.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-24T09:06:42.360 INFO:teuthology.orchestra.run.smithi090.stdout:(47/152): python3-cryptography-36.0.1-4.el9.x86 511 kB/s | 1.2 MB 00:02 2024-04-24T09:06:42.397 INFO:teuthology.orchestra.run.smithi114.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-24T09:06:42.402 INFO:teuthology.orchestra.run.smithi149.stdout:(35/152): ceph-volume-19.0.0-3116.g1f81fb39.el9 4.0 MB/s | 273 kB 00:00 2024-04-24T09:06:42.422 INFO:teuthology.orchestra.run.smithi114.stdout:(46/152): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 135 kB 00:00 2024-04-24T09:06:42.481 INFO:teuthology.orchestra.run.smithi114.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-04-24T09:06:42.527 INFO:teuthology.orchestra.run.smithi149.stdout:(36/152): cephadm-19.0.0-3116.g1f81fb39.el9.noa 6.0 MB/s | 762 kB 00:00 2024-04-24T09:06:42.578 INFO:teuthology.orchestra.run.smithi149.stdout:(37/152): cephfs-top-19.0.0-3116.g1f81fb39.el9. 365 kB/s | 18 kB 00:00 2024-04-24T09:06:42.598 INFO:teuthology.orchestra.run.smithi114.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 3.8 MB/s | 689 kB 00:00 2024-04-24T09:06:42.631 INFO:teuthology.orchestra.run.smithi114.stdout:(49/152): python3-cryptography-36.0.1-4.el9.x86 3.3 MB/s | 1.2 MB 00:00 2024-04-24T09:06:42.656 INFO:teuthology.orchestra.run.smithi114.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-04-24T09:06:42.690 INFO:teuthology.orchestra.run.smithi114.stdout:(51/152): bison-3.7.4-5.el9.x86_64.rpm 4.4 MB/s | 943 kB 00:00 2024-04-24T09:06:42.713 INFO:teuthology.orchestra.run.smithi149.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 602 kB/s | 80 kB 00:00 2024-04-24T09:06:42.715 INFO:teuthology.orchestra.run.smithi114.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-04-24T09:06:42.719 INFO:teuthology.orchestra.run.smithi090.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 223 kB/s | 215 kB 00:00 2024-04-24T09:06:42.740 INFO:teuthology.orchestra.run.smithi114.stdout:(53/152): elfutils-libelf-devel-0.190-2.el9.x86 197 kB/s | 21 kB 00:00 2024-04-24T09:06:42.766 INFO:teuthology.orchestra.run.smithi114.stdout:(54/152): flexiblas-3.0.4-8.el9.x86_64.rpm 434 kB/s | 33 kB 00:00 2024-04-24T09:06:42.791 INFO:teuthology.orchestra.run.smithi114.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-04-24T09:06:42.816 INFO:teuthology.orchestra.run.smithi114.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-04-24T09:06:42.837 INFO:teuthology.orchestra.run.smithi149.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 6.3 MB/s | 803 kB 00:00 2024-04-24T09:06:42.844 INFO:teuthology.orchestra.run.smithi090.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 909 kB/s | 689 kB 00:00 2024-04-24T09:06:42.850 INFO:teuthology.orchestra.run.smithi114.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-24T09:06:42.875 INFO:teuthology.orchestra.run.smithi114.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 774 kB/s | 45 kB 00:00 2024-04-24T09:06:42.887 INFO:teuthology.orchestra.run.smithi149.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 195 kB 00:00 2024-04-24T09:06:42.917 INFO:teuthology.orchestra.run.smithi114.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-04-24T09:06:42.937 INFO:teuthology.orchestra.run.smithi149.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.0 MB/s | 253 kB 00:00 2024-04-24T09:06:42.953 INFO:teuthology.orchestra.run.smithi090.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 200 kB/s | 21 kB 00:00 2024-04-24T09:06:42.959 INFO:teuthology.orchestra.run.smithi114.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 613 kB/s | 25 kB 00:00 2024-04-24T09:06:43.021 INFO:teuthology.orchestra.run.smithi149.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.2 MB 00:00 2024-04-24T09:06:43.026 INFO:teuthology.orchestra.run.smithi114.stdout:(61/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 9.6 MB/s | 3.0 MB 00:00 2024-04-24T09:06:43.051 INFO:teuthology.orchestra.run.smithi114.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 517 kB/s | 47 kB 00:00 2024-04-24T09:06:43.070 INFO:teuthology.orchestra.run.smithi090.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-04-24T09:06:43.071 INFO:teuthology.orchestra.run.smithi149.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-04-24T09:06:43.076 INFO:teuthology.orchestra.run.smithi114.stdout:(63/152): libtool-2.4.6-45.el9.x86_64.rpm 2.8 MB/s | 582 kB 00:00 2024-04-24T09:06:43.102 INFO:teuthology.orchestra.run.smithi114.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-04-24T09:06:43.122 INFO:teuthology.orchestra.run.smithi149.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-04-24T09:06:43.127 INFO:teuthology.orchestra.run.smithi114.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 701 kB/s | 35 kB 00:00 2024-04-24T09:06:43.128 INFO:teuthology.orchestra.run.smithi090.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 1.2 MB/s | 943 kB 00:00 2024-04-24T09:06:43.152 INFO:teuthology.orchestra.run.smithi114.stdout:(66/152): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-04-24T09:06:43.171 INFO:teuthology.orchestra.run.smithi149.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-24T09:06:43.179 INFO:teuthology.orchestra.run.smithi090.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 302 kB/s | 33 kB 00:00 2024-04-24T09:06:43.222 INFO:teuthology.orchestra.run.smithi149.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-24T09:06:43.229 INFO:teuthology.orchestra.run.smithi090.stdout:(54/152): boost-program-options-1.75.0-8.el9.x8 210 kB/s | 107 kB 00:00 2024-04-24T09:06:43.236 INFO:teuthology.orchestra.run.smithi114.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-04-24T09:06:43.278 INFO:teuthology.orchestra.run.smithi114.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-04-24T09:06:43.288 INFO:teuthology.orchestra.run.smithi090.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 168 kB/s | 18 kB 00:00 2024-04-24T09:06:43.311 INFO:teuthology.orchestra.run.smithi114.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-04-24T09:06:43.337 INFO:teuthology.orchestra.run.smithi114.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-04-24T09:06:43.338 INFO:teuthology.orchestra.run.smithi090.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 103 kB/s | 11 kB 00:00 2024-04-24T09:06:43.404 INFO:teuthology.orchestra.run.smithi114.stdout:(71/152): openblas-openmp-0.3.21-2.el9.x86_64.r 16 MB/s | 4.7 MB 00:00 2024-04-24T09:06:43.414 INFO:teuthology.orchestra.run.smithi090.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-04-24T09:06:43.446 INFO:teuthology.orchestra.run.smithi114.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 9.3 MB/s | 1.0 MB 00:00 2024-04-24T09:06:43.447 INFO:teuthology.orchestra.run.smithi090.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 418 kB/s | 45 kB 00:00 2024-04-24T09:06:43.471 INFO:teuthology.orchestra.run.smithi114.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-04-24T09:06:43.513 INFO:teuthology.orchestra.run.smithi114.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 5.7 MB/s | 244 kB 00:00 2024-04-24T09:06:43.588 INFO:teuthology.orchestra.run.smithi114.stdout:(75/152): postgresql-server-13.14-1.el9.x86_64. 21 MB/s | 5.8 MB 00:00 2024-04-24T09:06:43.589 INFO:teuthology.orchestra.run.smithi149.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 1.8 MB/s | 689 kB 00:00 2024-04-24T09:06:43.613 INFO:teuthology.orchestra.run.smithi114.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-04-24T09:06:43.631 INFO:teuthology.orchestra.run.smithi090.stdout:(59/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.0 MB/s | 3.0 MB 00:00 2024-04-24T09:06:43.639 INFO:teuthology.orchestra.run.smithi114.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-04-24T09:06:43.664 INFO:teuthology.orchestra.run.smithi114.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-04-24T09:06:43.689 INFO:teuthology.orchestra.run.smithi114.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-04-24T09:06:43.698 INFO:teuthology.orchestra.run.smithi090.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 2.3 MB/s | 662 kB 00:00 2024-04-24T09:06:43.714 INFO:teuthology.orchestra.run.smithi114.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 694 kB/s | 35 kB 00:00 2024-04-24T09:06:43.740 INFO:teuthology.orchestra.run.smithi090.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 236 kB/s | 25 kB 00:00 2024-04-24T09:06:43.798 INFO:teuthology.orchestra.run.smithi114.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-04-24T09:06:43.807 INFO:teuthology.orchestra.run.smithi090.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 439 kB/s | 47 kB 00:00 2024-04-24T09:06:43.823 INFO:teuthology.orchestra.run.smithi114.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-04-24T09:06:43.848 INFO:teuthology.orchestra.run.smithi114.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-04-24T09:06:43.857 INFO:teuthology.orchestra.run.smithi090.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.5 MB/s | 292 kB 00:00 2024-04-24T09:06:43.874 INFO:teuthology.orchestra.run.smithi114.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-24T09:06:43.881 INFO:teuthology.orchestra.run.smithi149.stdout:(48/152): bison-3.7.4-5.el9.x86_64.rpm 3.2 MB/s | 943 kB 00:00 2024-04-24T09:06:43.899 INFO:teuthology.orchestra.run.smithi114.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 840 kB/s | 42 kB 00:00 2024-04-24T09:06:43.924 INFO:teuthology.orchestra.run.smithi114.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-04-24T09:06:43.931 INFO:teuthology.orchestra.run.smithi149.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-04-24T09:06:43.934 INFO:teuthology.orchestra.run.smithi090.stdout:(64/152): libtool-2.4.6-45.el9.x86_64.rpm 1.2 MB/s | 582 kB 00:00 2024-04-24T09:06:43.949 INFO:teuthology.orchestra.run.smithi114.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-24T09:06:43.967 INFO:teuthology.orchestra.run.smithi090.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 1.2 MB/s | 188 kB 00:00 2024-04-24T09:06:43.975 INFO:teuthology.orchestra.run.smithi114.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-04-24T09:06:43.981 INFO:teuthology.orchestra.run.smithi149.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 430 kB/s | 21 kB 00:00 2024-04-24T09:06:43.993 INFO:teuthology.orchestra.run.smithi090.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 261 kB/s | 35 kB 00:00 2024-04-24T09:06:44.000 INFO:teuthology.orchestra.run.smithi114.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-24T09:06:44.025 INFO:teuthology.orchestra.run.smithi114.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-24T09:06:44.101 INFO:teuthology.orchestra.run.smithi114.stdout:(91/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-04-24T09:06:44.126 INFO:teuthology.orchestra.run.smithi114.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 415 kB/s | 42 kB 00:00 2024-04-24T09:06:44.151 INFO:teuthology.orchestra.run.smithi114.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 886 kB/s | 44 kB 00:00 2024-04-24T09:06:44.176 INFO:teuthology.orchestra.run.smithi114.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-04-24T09:06:44.182 INFO:teuthology.orchestra.run.smithi149.stdout:(51/152): ceph-common-19.0.0-3116.g1f81fb39.el9 1.3 MB/s | 21 MB 00:16 2024-04-24T09:06:44.202 INFO:teuthology.orchestra.run.smithi114.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 715 kB/s | 36 kB 00:00 2024-04-24T09:06:44.207 INFO:teuthology.orchestra.run.smithi149.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 1.4 MB/s | 313 kB 00:00 2024-04-24T09:06:44.226 INFO:teuthology.orchestra.run.smithi090.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 6.8 MB/s | 1.6 MB 00:00 2024-04-24T09:06:44.227 INFO:teuthology.orchestra.run.smithi114.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-04-24T09:06:44.252 INFO:teuthology.orchestra.run.smithi114.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-04-24T09:06:44.278 INFO:teuthology.orchestra.run.smithi114.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-24T09:06:44.303 INFO:teuthology.orchestra.run.smithi114.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-24T09:06:44.307 INFO:teuthology.orchestra.run.smithi149.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 262 kB/s | 33 kB 00:00 2024-04-24T09:06:44.328 INFO:teuthology.orchestra.run.smithi114.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-24T09:06:44.357 INFO:teuthology.orchestra.run.smithi149.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 365 kB/s | 18 kB 00:00 2024-04-24T09:06:44.370 INFO:teuthology.orchestra.run.smithi114.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 670 kB/s | 45 kB 00:00 2024-04-24T09:06:44.377 INFO:teuthology.orchestra.run.smithi090.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 6.0 MB/s | 911 kB 00:00 2024-04-24T09:06:44.408 INFO:teuthology.orchestra.run.smithi149.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-04-24T09:06:44.485 INFO:teuthology.orchestra.run.smithi090.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 1.2 MB/s | 133 kB 00:00 2024-04-24T09:06:44.587 INFO:teuthology.orchestra.run.smithi114.stdout:(102/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2024-04-24T09:06:44.612 INFO:teuthology.orchestra.run.smithi114.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 79 kB/s | 22 kB 00:00 2024-04-24T09:06:44.636 INFO:teuthology.orchestra.run.smithi090.stdout:(70/152): openblas-openmp-0.3.21-2.el9.x86_64.r 6.8 MB/s | 4.7 MB 00:00 2024-04-24T09:06:44.683 INFO:teuthology.orchestra.run.smithi149.stdout:(56/152): ceph-osd-19.0.0-3116.g1f81fb39.el9.x8 1.2 MB/s | 17 MB 00:14 2024-04-24T09:06:44.708 INFO:teuthology.orchestra.run.smithi149.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 534 kB/s | 160 kB 00:00 2024-04-24T09:06:44.754 INFO:teuthology.orchestra.run.smithi114.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 2.2 MB/s | 863 kB 00:00 2024-04-24T09:06:44.800 INFO:teuthology.orchestra.run.smithi149.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 388 kB/s | 45 kB 00:00 2024-04-24T09:06:44.858 INFO:teuthology.orchestra.run.smithi149.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 662 kB 00:00 2024-04-24T09:06:44.909 INFO:teuthology.orchestra.run.smithi149.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 509 kB/s | 25 kB 00:00 2024-04-24T09:06:44.957 INFO:teuthology.orchestra.run.smithi114.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 903 kB/s | 309 kB 00:00 2024-04-24T09:06:44.959 INFO:teuthology.orchestra.run.smithi149.stdout:(61/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 953 kB/s | 47 kB 00:00 2024-04-24T09:06:44.980 INFO:teuthology.orchestra.run.smithi114.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 87 kB/s | 19 kB 00:00 2024-04-24T09:06:44.984 INFO:teuthology.orchestra.run.smithi149.stdout:(62/152): libtool-2.4.6-45.el9.x86_64.rpm 3.1 MB/s | 582 kB 00:00 2024-04-24T09:06:44.986 INFO:teuthology.orchestra.run.smithi090.stdout:(71/152): postgresql-server-13.14-1.el9.x86_64. 12 MB/s | 5.8 MB 00:00 2024-04-24T09:06:45.038 INFO:teuthology.orchestra.run.smithi114.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 446 kB/s | 26 kB 00:00 2024-04-24T09:06:45.043 INFO:teuthology.orchestra.run.smithi149.stdout:(63/152): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-04-24T09:06:45.068 INFO:teuthology.orchestra.run.smithi149.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-04-24T09:06:45.070 INFO:teuthology.orchestra.run.smithi090.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 2.3 MB/s | 1.0 MB 00:00 2024-04-24T09:06:45.095 INFO:teuthology.orchestra.run.smithi090.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 812 kB/s | 87 kB 00:00 2024-04-24T09:06:45.095 INFO:teuthology.orchestra.run.smithi149.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 675 kB/s | 35 kB 00:00 2024-04-24T09:06:45.105 INFO:teuthology.orchestra.run.smithi114.stdout:(108/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.0 MB/s | 548 kB 00:00 2024-04-24T09:06:45.131 INFO:teuthology.orchestra.run.smithi114.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 529 kB/s | 49 kB 00:00 2024-04-24T09:06:45.163 INFO:teuthology.orchestra.run.smithi149.stdout:(66/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.1 MB/s | 3.0 MB 00:00 2024-04-24T09:06:45.221 INFO:teuthology.orchestra.run.smithi090.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.9 MB/s | 244 kB 00:00 2024-04-24T09:06:45.256 INFO:teuthology.orchestra.run.smithi114.stdout:(110/152): libarrow-9.0.0-10.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-04-24T09:06:45.281 INFO:teuthology.orchestra.run.smithi114.stdout:(111/152): libunwind-1.6.2-1.el9.x86_64.rpm 449 kB/s | 67 kB 00:00 2024-04-24T09:06:45.315 INFO:teuthology.orchestra.run.smithi114.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 823 kB/s | 151 kB 00:00 2024-04-24T09:06:45.329 INFO:teuthology.orchestra.run.smithi090.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-04-24T09:06:45.348 INFO:teuthology.orchestra.run.smithi114.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 8.0 MB/s | 548 kB 00:00 2024-04-24T09:06:45.373 INFO:teuthology.orchestra.run.smithi114.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 744 kB/s | 43 kB 00:00 2024-04-24T09:06:45.407 INFO:teuthology.orchestra.run.smithi114.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 550 kB/s | 32 kB 00:00 2024-04-24T09:06:45.432 INFO:teuthology.orchestra.run.smithi114.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 256 kB/s | 15 kB 00:00 2024-04-24T09:06:45.463 INFO:teuthology.orchestra.run.smithi090.stdout:(76/152): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-04-24T09:06:45.466 INFO:teuthology.orchestra.run.smithi114.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-04-24T09:06:45.521 INFO:teuthology.orchestra.run.smithi149.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 4.4 MB/s | 1.6 MB 00:00 2024-04-24T09:06:45.522 INFO:teuthology.orchestra.run.smithi090.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 134 kB/s | 26 kB 00:00 2024-04-24T09:06:45.524 INFO:teuthology.orchestra.run.smithi114.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-04-24T09:06:45.550 INFO:teuthology.orchestra.run.smithi114.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-04-24T09:06:45.572 INFO:teuthology.orchestra.run.smithi090.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 178 kB/s | 19 kB 00:00 2024-04-24T09:06:45.583 INFO:teuthology.orchestra.run.smithi114.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 605 kB/s | 35 kB 00:00 2024-04-24T09:06:45.617 INFO:teuthology.orchestra.run.smithi114.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 3.0 MB/s | 206 kB 00:00 2024-04-24T09:06:45.630 INFO:teuthology.orchestra.run.smithi090.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.6 MB/s | 172 kB 00:00 2024-04-24T09:06:45.642 INFO:teuthology.orchestra.run.smithi114.stdout:(122/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.1 MB/s | 838 kB 00:00 2024-04-24T09:06:45.676 INFO:teuthology.orchestra.run.smithi114.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-04-24T09:06:45.681 INFO:teuthology.orchestra.run.smithi090.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 321 kB/s | 35 kB 00:00 2024-04-24T09:06:45.701 INFO:teuthology.orchestra.run.smithi114.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 183 kB/s | 11 kB 00:00 2024-04-24T09:06:45.735 INFO:teuthology.orchestra.run.smithi114.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 301 kB/s | 18 kB 00:00 2024-04-24T09:06:45.755 INFO:teuthology.orchestra.run.smithi149.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 3.8 MB/s | 911 kB 00:00 2024-04-24T09:06:45.802 INFO:teuthology.orchestra.run.smithi114.stdout:(126/152): python3-grpcio-1.46.7-10.el9.x86_64. 9.3 MB/s | 2.0 MB 00:00 2024-04-24T09:06:45.806 INFO:teuthology.orchestra.run.smithi090.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.0 MB/s | 250 kB 00:00 2024-04-24T09:06:45.818 INFO:teuthology.orchestra.run.smithi114.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 197 kB/s | 23 kB 00:00 2024-04-24T09:06:45.821 INFO:teuthology.orchestra.run.smithi149.stdout:(69/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 6.1 MB/s | 4.4 MB 00:00 2024-04-24T09:06:45.835 INFO:teuthology.orchestra.run.smithi114.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 194 kB/s | 19 kB 00:00 2024-04-24T09:06:45.878 INFO:teuthology.orchestra.run.smithi114.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 261 kB/s | 20 kB 00:00 2024-04-24T09:06:45.880 INFO:teuthology.orchestra.run.smithi149.stdout:(70/152): openblas-openmp-0.3.21-2.el9.x86_64.r 5.8 MB/s | 4.7 MB 00:00 2024-04-24T09:06:45.905 INFO:teuthology.orchestra.run.smithi149.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 886 kB/s | 133 kB 00:00 2024-04-24T09:06:45.911 INFO:teuthology.orchestra.run.smithi114.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-04-24T09:06:45.928 INFO:teuthology.orchestra.run.smithi114.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 501 kB/s | 46 kB 00:00 2024-04-24T09:06:45.953 INFO:teuthology.orchestra.run.smithi114.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-04-24T09:06:45.955 INFO:teuthology.orchestra.run.smithi149.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-04-24T09:06:45.970 INFO:teuthology.orchestra.run.smithi114.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 983 kB/s | 58 kB 00:00 2024-04-24T09:06:45.995 INFO:teuthology.orchestra.run.smithi090.stdout:(82/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 14 MB/s | 5.1 MB 00:00 2024-04-24T09:06:46.004 INFO:teuthology.orchestra.run.smithi114.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-04-24T09:06:46.021 INFO:teuthology.orchestra.run.smithi114.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 245 kB/s | 16 kB 00:00 2024-04-24T09:06:46.031 INFO:teuthology.orchestra.run.smithi090.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 990 kB/s | 222 kB 00:00 2024-04-24T09:06:46.046 INFO:teuthology.orchestra.run.smithi114.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-24T09:06:46.071 INFO:teuthology.orchestra.run.smithi114.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 422 kB/s | 28 kB 00:00 2024-04-24T09:06:46.081 INFO:teuthology.orchestra.run.smithi149.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 5.0 MB/s | 1.0 MB 00:00 2024-04-24T09:06:46.097 INFO:teuthology.orchestra.run.smithi114.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-04-24T09:06:46.099 INFO:teuthology.orchestra.run.smithi090.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 754 kB/s | 77 kB 00:00 2024-04-24T09:06:46.130 INFO:teuthology.orchestra.run.smithi114.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-04-24T09:06:46.140 INFO:teuthology.orchestra.run.smithi149.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.1 MB/s | 244 kB 00:00 2024-04-24T09:06:46.141 INFO:teuthology.orchestra.run.smithi090.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 388 kB/s | 42 kB 00:00 2024-04-24T09:06:46.155 INFO:teuthology.orchestra.run.smithi114.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 706 kB/s | 59 kB 00:00 2024-04-24T09:06:46.181 INFO:teuthology.orchestra.run.smithi114.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 427 kB/s | 36 kB 00:00 2024-04-24T09:06:46.206 INFO:teuthology.orchestra.run.smithi114.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 675 kB/s | 51 kB 00:00 2024-04-24T09:06:46.207 INFO:teuthology.orchestra.run.smithi149.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 3.7 MB/s | 248 kB 00:00 2024-04-24T09:06:46.208 INFO:teuthology.orchestra.run.smithi090.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 2.4 MB/s | 269 kB 00:00 2024-04-24T09:06:46.231 INFO:teuthology.orchestra.run.smithi114.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-24T09:06:46.250 INFO:teuthology.orchestra.run.smithi090.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-04-24T09:06:46.256 INFO:teuthology.orchestra.run.smithi114.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-24T09:06:46.257 INFO:teuthology.orchestra.run.smithi149.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 514 kB/s | 26 kB 00:00 2024-04-24T09:06:46.307 INFO:teuthology.orchestra.run.smithi149.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-04-24T09:06:46.326 INFO:teuthology.orchestra.run.smithi090.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 279 kB 00:00 2024-04-24T09:06:46.348 INFO:teuthology.orchestra.run.smithi114.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-04-24T09:06:46.360 INFO:teuthology.orchestra.run.smithi090.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 466 kB/s | 51 kB 00:00 2024-04-24T09:06:46.365 INFO:teuthology.orchestra.run.smithi149.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-24T09:06:46.390 INFO:teuthology.orchestra.run.smithi114.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 126 kB/s | 20 kB 00:00 2024-04-24T09:06:46.416 INFO:teuthology.orchestra.run.smithi114.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.2 MB/s | 192 kB 00:00 2024-04-24T09:06:46.424 INFO:teuthology.orchestra.run.smithi149.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 596 kB/s | 35 kB 00:00 2024-04-24T09:06:46.452 INFO:teuthology.orchestra.run.smithi090.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 430 kB/s | 54 kB 00:00 2024-04-24T09:06:46.458 INFO:teuthology.orchestra.run.smithi114.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-04-24T09:06:46.561 INFO:teuthology.orchestra.run.smithi090.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 386 kB/s | 42 kB 00:00 2024-04-24T09:06:46.670 INFO:teuthology.orchestra.run.smithi090.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 410 kB/s | 44 kB 00:00 2024-04-24T09:06:46.716 INFO:teuthology.orchestra.run.smithi149.stdout:(80/152): postgresql-server-13.14-1.el9.x86_64. 6.5 MB/s | 5.8 MB 00:00 2024-04-24T09:06:46.750 INFO:teuthology.orchestra.run.smithi114.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 4.6 MB/s | 1.3 MB 00:00 2024-04-24T09:06:46.770 INFO:teuthology.orchestra.run.smithi090.stdout:(93/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 1.6 MB/s | 4.4 MB 00:02 2024-04-24T09:06:46.774 INFO:teuthology.orchestra.run.smithi149.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-04-24T09:06:46.803 INFO:teuthology.orchestra.run.smithi090.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 1.6 MB/s | 221 kB 00:00 2024-04-24T09:06:46.842 INFO:teuthology.orchestra.run.smithi149.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.3 MB/s | 222 kB 00:00 2024-04-24T09:06:46.883 INFO:teuthology.orchestra.run.smithi114.stdout:(150/152): librados2-19.0.0-3116.g1f81fb39.el9. 6.9 MB/s | 3.4 MB 00:00 2024-04-24T09:06:46.900 INFO:teuthology.orchestra.run.smithi149.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-24T09:06:46.984 INFO:teuthology.orchestra.run.smithi149.stdout:(84/152): python3-babel-2.9.1-2.el9.noarch.rpm 5.8 MB/s | 6.0 MB 00:01 2024-04-24T09:06:47.020 INFO:teuthology.orchestra.run.smithi149.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 355 kB/s | 42 kB 00:00 2024-04-24T09:06:47.043 INFO:teuthology.orchestra.run.smithi149.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-04-24T09:06:47.071 INFO:teuthology.orchestra.run.smithi090.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 120 kB/s | 36 kB 00:00 2024-04-24T09:06:47.084 INFO:teuthology.orchestra.run.smithi149.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.4 MB/s | 159 kB 00:00 2024-04-24T09:06:47.104 INFO:teuthology.orchestra.run.smithi090.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 158 kB/s | 47 kB 00:00 2024-04-24T09:06:47.110 INFO:teuthology.orchestra.run.smithi149.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-04-24T09:06:47.116 INFO:teuthology.orchestra.run.smithi114.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 6.5 MB/s | 2.4 MB 00:00 2024-04-24T09:06:47.135 INFO:teuthology.orchestra.run.smithi149.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-24T09:06:47.169 INFO:teuthology.orchestra.run.smithi149.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 916 kB/s | 54 kB 00:00 2024-04-24T09:06:47.213 INFO:teuthology.orchestra.run.smithi090.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 536 kB/s | 58 kB 00:00 2024-04-24T09:06:47.269 INFO:teuthology.orchestra.run.smithi149.stdout:(91/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 6.0 MB/s | 5.1 MB 00:00 2024-04-24T09:06:47.294 INFO:teuthology.orchestra.run.smithi149.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 333 kB/s | 42 kB 00:00 2024-04-24T09:06:47.319 INFO:teuthology.orchestra.run.smithi149.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 880 kB/s | 44 kB 00:00 2024-04-24T09:06:47.353 INFO:teuthology.orchestra.run.smithi149.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-04-24T09:06:47.378 INFO:teuthology.orchestra.run.smithi149.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 614 kB/s | 36 kB 00:00 2024-04-24T09:06:47.420 INFO:teuthology.orchestra.run.smithi149.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 706 kB/s | 47 kB 00:00 2024-04-24T09:06:47.446 INFO:teuthology.orchestra.run.smithi149.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.4 MB/s | 305 kB 00:00 2024-04-24T09:06:47.479 INFO:teuthology.orchestra.run.smithi149.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 991 kB/s | 58 kB 00:00 2024-04-24T09:06:47.499 INFO:teuthology.orchestra.run.smithi090.stdout:(98/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 14 MB/s | 16 MB 00:01 2024-04-24T09:06:47.504 INFO:teuthology.orchestra.run.smithi149.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-24T09:06:47.531 INFO:teuthology.orchestra.run.smithi090.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 296 kB/s | 94 kB 00:00 2024-04-24T09:06:47.555 INFO:teuthology.orchestra.run.smithi149.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 847 kB/s | 64 kB 00:00 2024-04-24T09:06:47.556 INFO:teuthology.orchestra.run.smithi090.stdout:(100/152): socat-1.7.4.1-5.el9.x86_64.rpm 629 kB/s | 305 kB 00:00 2024-04-24T09:06:47.567 INFO:teuthology.orchestra.run.smithi114.stdout:(152/152): librbd1-19.0.0-3116.g1f81fb39.el9.x8 2.7 MB/s | 3.1 MB 00:01 2024-04-24T09:06:47.573 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-04-24T09:06:47.573 INFO:teuthology.orchestra.run.smithi114.stdout:Total 15 MB/s | 222 MB 00:14 2024-04-24T09:06:47.580 INFO:teuthology.orchestra.run.smithi149.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 594 kB/s | 45 kB 00:00 2024-04-24T09:06:47.607 INFO:teuthology.orchestra.run.smithi090.stdout:(101/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 602 kB/s | 64 kB 00:00 2024-04-24T09:06:47.632 INFO:teuthology.orchestra.run.smithi090.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 448 kB/s | 45 kB 00:00 2024-04-24T09:06:47.655 INFO:teuthology.orchestra.run.smithi149.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 222 kB/s | 22 kB 00:00 2024-04-24T09:06:47.690 INFO:teuthology.orchestra.run.smithi090.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 167 kB/s | 22 kB 00:00 2024-04-24T09:06:47.756 INFO:teuthology.orchestra.run.smithi149.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 4.8 MB/s | 863 kB 00:00 2024-04-24T09:06:47.939 INFO:teuthology.orchestra.run.smithi149.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.9 MB/s | 548 kB 00:00 2024-04-24T09:06:47.998 INFO:teuthology.orchestra.run.smithi149.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 333 kB/s | 19 kB 00:00 2024-04-24T09:06:48.023 INFO:teuthology.orchestra.run.smithi149.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.1 MB/s | 309 kB 00:00 2024-04-24T09:06:48.024 INFO:teuthology.orchestra.run.smithi090.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 2.0 MB/s | 863 kB 00:00 2024-04-24T09:06:48.082 INFO:teuthology.orchestra.run.smithi149.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 445 kB/s | 26 kB 00:00 2024-04-24T09:06:48.141 INFO:teuthology.orchestra.run.smithi149.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 832 kB/s | 49 kB 00:00 2024-04-24T09:06:48.233 INFO:teuthology.orchestra.run.smithi149.stdout:(109/152): libarrow-9.0.0-10.el9.x86_64.rpm 19 MB/s | 4.4 MB 00:00 2024-04-24T09:06:48.258 INFO:teuthology.orchestra.run.smithi149.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 577 kB/s | 67 kB 00:00 2024-04-24T09:06:48.316 INFO:teuthology.orchestra.run.smithi149.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-04-24T09:06:48.417 INFO:teuthology.orchestra.run.smithi149.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 5.4 MB/s | 548 kB 00:00 2024-04-24T09:06:48.567 INFO:teuthology.orchestra.run.smithi149.stdout:(113/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-04-24T09:06:48.591 INFO:teuthology.orchestra.run.smithi090.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 34 kB/s | 19 kB 00:00 2024-04-24T09:06:48.592 INFO:teuthology.orchestra.run.smithi149.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 248 kB/s | 43 kB 00:00 2024-04-24T09:06:48.617 INFO:teuthology.orchestra.run.smithi149.stdout:(115/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.3 MB/s | 838 kB 00:00 2024-04-24T09:06:48.643 INFO:teuthology.orchestra.run.smithi149.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 298 kB/s | 15 kB 00:00 2024-04-24T09:06:48.683 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-04-24T09:06:48.685 INFO:teuthology.orchestra.run.smithi149.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-04-24T09:06:48.708 INFO:teuthology.orchestra.run.smithi090.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 304 kB/s | 309 kB 00:01 2024-04-24T09:06:48.710 INFO:teuthology.orchestra.run.smithi149.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-04-24T09:06:48.727 INFO:teuthology.orchestra.run.smithi149.stdout:(119/152): python3-cachetools-4.2.4-1.el9.noarc 202 kB/s | 32 kB 00:00 2024-04-24T09:06:48.768 INFO:teuthology.orchestra.run.smithi149.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-04-24T09:06:48.785 INFO:teuthology.orchestra.run.smithi149.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 468 kB/s | 35 kB 00:00 2024-04-24T09:06:48.825 INFO:teuthology.orchestra.run.smithi090.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 224 kB/s | 26 kB 00:00 2024-04-24T09:06:48.827 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-04-24T09:06:48.827 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-04-24T09:06:48.853 INFO:teuthology.orchestra.run.smithi149.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-04-24T09:06:48.911 INFO:teuthology.orchestra.run.smithi149.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 184 kB/s | 11 kB 00:00 2024-04-24T09:06:48.936 INFO:teuthology.orchestra.run.smithi149.stdout:(124/152): python3-google-auth-2.29.0-1.el9.noa 983 kB/s | 206 kB 00:00 2024-04-24T09:06:48.959 INFO:teuthology.orchestra.run.smithi090.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 366 kB/s | 49 kB 00:00 2024-04-24T09:06:48.987 INFO:teuthology.orchestra.run.smithi149.stdout:(125/152): python3-grpcio-1.46.7-10.el9.x86_64. 9.4 MB/s | 2.0 MB 00:00 2024-04-24T09:06:49.004 INFO:teuthology.orchestra.run.smithi149.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 192 kB/s | 18 kB 00:00 2024-04-24T09:06:49.021 INFO:teuthology.orchestra.run.smithi149.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 275 kB/s | 23 kB 00:00 2024-04-24T09:06:49.026 INFO:teuthology.orchestra.run.smithi090.stdout:(109/152): abseil-cpp-20211102.0-3.el9.x86_64.r 393 kB/s | 548 kB 00:01 2024-04-24T09:06:49.046 INFO:teuthology.orchestra.run.smithi149.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 331 kB/s | 19 kB 00:00 2024-04-24T09:06:49.079 INFO:teuthology.orchestra.run.smithi149.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 337 kB/s | 20 kB 00:00 2024-04-24T09:06:49.113 INFO:teuthology.orchestra.run.smithi149.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 693 kB/s | 46 kB 00:00 2024-04-24T09:06:49.155 INFO:teuthology.orchestra.run.smithi149.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-04-24T09:06:49.176 INFO:teuthology.orchestra.run.smithi090.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 311 kB/s | 67 kB 00:00 2024-04-24T09:06:49.188 INFO:teuthology.orchestra.run.smithi149.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 767 kB/s | 58 kB 00:00 2024-04-24T09:06:49.247 INFO:teuthology.orchestra.run.smithi149.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-04-24T09:06:49.272 INFO:teuthology.orchestra.run.smithi149.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 196 kB/s | 16 kB 00:00 2024-04-24T09:06:49.306 INFO:teuthology.orchestra.run.smithi149.stdout:(135/152): python3-kubernetes-26.1.0-2.el9.noar 3.6 MB/s | 1.0 MB 00:00 2024-04-24T09:06:49.340 INFO:teuthology.orchestra.run.smithi149.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 977 kB/s | 90 kB 00:00 2024-04-24T09:06:49.365 INFO:teuthology.orchestra.run.smithi149.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 306 kB/s | 28 kB 00:00 2024-04-24T09:06:49.394 INFO:teuthology.orchestra.run.smithi090.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 411 kB/s | 151 kB 00:00 2024-04-24T09:06:49.398 INFO:teuthology.orchestra.run.smithi149.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 334 kB/s | 31 kB 00:00 2024-04-24T09:06:49.465 INFO:teuthology.orchestra.run.smithi149.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 590 kB/s | 59 kB 00:00 2024-04-24T09:06:49.491 INFO:teuthology.orchestra.run.smithi149.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 388 kB/s | 36 kB 00:00 2024-04-24T09:06:49.541 INFO:teuthology.orchestra.run.smithi149.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 677 kB/s | 51 kB 00:00 2024-04-24T09:06:49.641 INFO:teuthology.orchestra.run.smithi149.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.5 MB/s | 230 kB 00:00 2024-04-24T09:06:49.667 INFO:teuthology.orchestra.run.smithi149.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 716 kB/s | 90 kB 00:00 2024-04-24T09:06:49.717 INFO:teuthology.orchestra.run.smithi149.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-04-24T09:06:49.742 INFO:teuthology.orchestra.run.smithi149.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 265 kB/s | 20 kB 00:00 2024-04-24T09:06:49.775 INFO:teuthology.orchestra.run.smithi149.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 3.2 MB/s | 192 kB 00:00 2024-04-24T09:06:49.843 INFO:teuthology.orchestra.run.smithi149.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-04-24T09:06:49.876 INFO:teuthology.orchestra.run.smithi149.stdout:(148/152): python3-routes-2.5.1-5.el9.noarch.rp 351 kB/s | 188 kB 00:00 2024-04-24T09:06:50.110 INFO:teuthology.orchestra.run.smithi149.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 5.8 MB/s | 1.3 MB 00:00 2024-04-24T09:06:50.235 INFO:teuthology.orchestra.run.smithi149.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 19 MB/s | 2.4 MB 00:00 2024-04-24T09:06:50.261 INFO:teuthology.orchestra.run.smithi090.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 633 kB/s | 548 kB 00:00 2024-04-24T09:06:50.302 INFO:teuthology.orchestra.run.smithi149.stdout:(151/152): librbd1-19.0.0-3116.g1f81fb39.el9.x8 6.8 MB/s | 3.1 MB 00:00 2024-04-24T09:06:50.403 INFO:teuthology.orchestra.run.smithi090.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 307 kB/s | 43 kB 00:00 2024-04-24T09:06:50.453 INFO:teuthology.orchestra.run.smithi090.stdout:(114/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 657 kB/s | 838 kB 00:01 2024-04-24T09:06:50.520 INFO:teuthology.orchestra.run.smithi090.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 276 kB/s | 32 kB 00:00 2024-04-24T09:06:50.571 INFO:teuthology.orchestra.run.smithi090.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 128 kB/s | 15 kB 00:00 2024-04-24T09:06:50.805 INFO:teuthology.orchestra.run.smithi090.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 606 kB/s | 172 kB 00:00 2024-04-24T09:06:50.838 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-04-24T09:06:50.839 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-04-24T09:06:50.952 INFO:teuthology.orchestra.run.smithi149.stdout:(152/152): librados2-19.0.0-3116.g1f81fb39.el9. 2.9 MB/s | 3.4 MB 00:01 2024-04-24T09:06:50.955 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-04-24T09:06:50.955 INFO:teuthology.orchestra.run.smithi149.stdout:Total 9.3 MB/s | 222 MB 00:23 2024-04-24T09:06:51.197 INFO:teuthology.orchestra.run.smithi090.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 574 kB/s | 358 kB 00:00 2024-04-24T09:06:51.339 INFO:teuthology.orchestra.run.smithi090.stdout:(119/152): python3-colorama-0.4.4-12.el9.noarch 250 kB/s | 35 kB 00:00 2024-04-24T09:06:51.389 INFO:teuthology.orchestra.run.smithi090.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 519 kB/s | 303 kB 00:00 2024-04-24T09:06:51.739 INFO:teuthology.orchestra.run.smithi090.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 514 kB/s | 206 kB 00:00 2024-04-24T09:06:52.015 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-04-24T09:06:52.156 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-04-24T09:06:52.157 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-04-24T09:06:52.165 INFO:teuthology.orchestra.run.smithi090.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 340 kB/s | 144 kB 00:00 2024-04-24T09:06:52.290 INFO:teuthology.orchestra.run.smithi090.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 86 kB/s | 11 kB 00:00 2024-04-24T09:06:52.399 INFO:teuthology.orchestra.run.smithi090.stdout:(124/152): python3-jaraco-classes-3.2.1-5.el9.n 164 kB/s | 18 kB 00:00 2024-04-24T09:06:52.507 INFO:teuthology.orchestra.run.smithi090.stdout:(125/152): python3-jaraco-collections-3.0.0-8.e 215 kB/s | 23 kB 00:00 2024-04-24T09:06:52.616 INFO:teuthology.orchestra.run.smithi090.stdout:(126/152): python3-jaraco-functools-3.5.0-2.el9 181 kB/s | 19 kB 00:00 2024-04-24T09:06:52.733 INFO:teuthology.orchestra.run.smithi090.stdout:(127/152): python3-jaraco-text-3.2.0-6.el9.noar 170 kB/s | 20 kB 00:00 2024-04-24T09:06:53.925 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-04-24T09:06:53.925 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-04-24T09:06:54.134 INFO:teuthology.orchestra.run.smithi090.stdout:(128/152): python3-kubernetes-26.1.0-2.el9.noar 747 kB/s | 1.0 MB 00:01 2024-04-24T09:06:54.200 INFO:teuthology.orchestra.run.smithi090.stdout:(129/152): python3-grpcio-1.46.7-10.el9.x86_64. 742 kB/s | 2.0 MB 00:02 2024-04-24T09:06:54.251 INFO:teuthology.orchestra.run.smithi090.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 398 kB/s | 46 kB 00:00 2024-04-24T09:06:54.430 INFO:teuthology.orchestra.run.smithi090.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 345 kB/s | 79 kB 00:00 2024-04-24T09:06:54.460 INFO:teuthology.orchestra.run.smithi090.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 277 kB/s | 58 kB 00:00 2024-04-24T09:06:54.644 INFO:teuthology.orchestra.run.smithi090.stdout:(133/152): libarrow-9.0.0-10.el9.x86_64.rpm 747 kB/s | 4.4 MB 00:06 2024-04-24T09:06:54.670 INFO:teuthology.orchestra.run.smithi090.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 78 kB/s | 16 kB 00:00 2024-04-24T09:06:54.796 INFO:teuthology.orchestra.run.smithi090.stdout:(135/152): python3-pyperclip-1.8.2-2.el9.noarch 227 kB/s | 28 kB 00:00 2024-04-24T09:06:54.910 INFO:teuthology.orchestra.run.smithi090.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 339 kB/s | 90 kB 00:00 2024-04-24T09:06:54.955 INFO:teuthology.orchestra.run.smithi090.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 194 kB/s | 31 kB 00:00 2024-04-24T09:06:54.988 INFO:teuthology.orchestra.run.smithi090.stdout:(138/152): python3-pecan-1.4.2-3.el9.noarch.rpm 488 kB/s | 272 kB 00:00 2024-04-24T09:06:55.130 INFO:teuthology.orchestra.run.smithi090.stdout:(139/152): python3-tempora-5.0.0-2.el9.noarch.r 253 kB/s | 36 kB 00:00 2024-04-24T09:06:55.214 INFO:teuthology.orchestra.run.smithi090.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 229 kB/s | 59 kB 00:00 2024-04-24T09:06:55.264 INFO:teuthology.orchestra.run.smithi090.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 383 kB/s | 51 kB 00:00 2024-04-24T09:06:55.298 INFO:teuthology.orchestra.run.smithi090.stdout:(142/152): python3-routes-2.5.1-5.el9.noarch.rp 487 kB/s | 188 kB 00:00 2024-04-24T09:06:55.531 INFO:teuthology.orchestra.run.smithi090.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 336 kB/s | 90 kB 00:00 2024-04-24T09:06:55.682 INFO:teuthology.orchestra.run.smithi090.stdout:(144/152): python3-zc-lockfile-2.0-10.el9.noarc 134 kB/s | 20 kB 00:00 2024-04-24T09:06:55.999 INFO:teuthology.orchestra.run.smithi090.stdout:(145/152): re2-20211101-3.el9.x86_64.rpm 606 kB/s | 192 kB 00:00 2024-04-24T09:06:56.116 INFO:teuthology.orchestra.run.smithi090.stdout:(146/152): python3-webob-1.8.7-6.el9.noarch.rpm 255 kB/s | 230 kB 00:00 2024-04-24T09:06:56.308 INFO:teuthology.orchestra.run.smithi090.stdout:(147/152): python3-werkzeug-2.0.3-3.el9.1.noarc 423 kB/s | 427 kB 00:01 2024-04-24T09:06:56.642 INFO:teuthology.orchestra.run.smithi090.stdout:(148/152): librbd1-19.0.0-3116.g1f81fb39.el9.x8 9.4 MB/s | 3.1 MB 00:00 2024-04-24T09:06:56.935 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-04-24T09:06:57.012 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-24T09:06:57.050 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-24T09:06:57.104 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-24T09:06:57.138 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-24T09:06:57.164 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-24T09:06:57.206 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-24T09:06:57.525 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-24T09:06:57.639 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : librados2-2:19.0.0-3116.g1f81fb39.el9.x86_64 8/156 2024-04-24T09:06:57.760 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librados2-2:19.0.0-3116.g1f81fb39.el9.x86_64 8/156 2024-04-24T09:06:57.782 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libcephfs2-2:19.0.0-3116.g1f81fb39.el9.x86_64 9/156 2024-04-24T09:06:57.834 INFO:teuthology.orchestra.run.smithi090.stdout:(149/152): librados2-19.0.0-3116.g1f81fb39.el9. 2.0 MB/s | 3.4 MB 00:01 2024-04-24T09:06:57.865 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: libcephfs2-2:19.0.0-3116.g1f81fb39.el9.x86_64 9/156 2024-04-24T09:06:57.903 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-rados-2:19.0.0-3116.g1f81fb39.el9.x86_64 10/156 2024-04-24T09:06:58.051 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-24T09:06:58.102 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-24T09:06:58.130 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libcephsqlite-2:19.0.0-3116.g1f81fb39.el9.x86_64 13/156 2024-04-24T09:06:58.234 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3116.g1f81fb39.el9.x86_64 13/156 2024-04-24T09:06:58.256 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libradosstriper1-2:19.0.0-3116.g1f81fb39.el9.x86 14/156 2024-04-24T09:06:58.318 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3116.g1f81fb39.el9.x86 14/156 2024-04-24T09:06:58.340 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T09:06:58.358 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T09:06:58.515 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T09:06:58.548 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T09:06:58.671 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-24T09:06:58.766 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-24T09:06:58.805 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-24T09:06:58.839 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-24T09:06:58.903 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-24T09:06:58.955 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-24T09:06:59.000 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-24T09:06:59.032 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-24T09:06:59.060 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-24T09:06:59.196 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-24T09:06:59.201 INFO:teuthology.orchestra.run.smithi090.stdout:(150/152): thrift-0.15.0-2.el9.x86_64.rpm 506 kB/s | 1.6 MB 00:03 2024-04-24T09:06:59.324 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-24T09:06:59.354 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-24T09:06:59.389 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-24T09:06:59.468 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-24T09:06:59.500 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-ceph-common-2:19.0.0-3116.g1f81fb39.el9. 31/156 2024-04-24T09:06:59.526 INFO:teuthology.orchestra.run.smithi090.stdout:(151/152): openssl-3.2.1-1.el9.x86_64.rpm 479 kB/s | 1.3 MB 00:02 2024-04-24T09:06:59.543 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-ceph-argparse-2:19.0.0-3116.g1f81fb39.el 32/156 2024-04-24T09:06:59.572 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cephfs-2:19.0.0-3116.g1f81fb39.el9.x86_6 33/156 2024-04-24T09:06:59.642 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-24T09:06:59.686 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-24T09:06:59.760 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-24T09:06:59.894 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-24T09:06:59.938 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-24T09:07:00.086 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-04-24T09:07:00.190 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-24T09:07:00.228 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-24T09:07:00.282 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-24T09:07:00.315 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-24T09:07:00.341 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-24T09:07:00.384 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-24T09:07:00.453 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-24T09:07:00.534 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-24T09:07:00.563 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-24T09:07:00.606 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-24T09:07:00.644 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : librados-devel-2:19.0.0-3116.g1f81fb39.el9.x86_6 43/156 2024-04-24T09:07:00.686 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-24T09:07:00.711 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-24T09:07:00.826 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : librados2-2:19.0.0-3116.g1f81fb39.el9.x86_64 8/156 2024-04-24T09:07:00.846 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-24T09:07:00.888 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-24T09:07:00.937 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librados2-2:19.0.0-3116.g1f81fb39.el9.x86_64 8/156 2024-04-24T09:07:00.942 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-24T09:07:00.967 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephfs2-2:19.0.0-3116.g1f81fb39.el9.x86_64 9/156 2024-04-24T09:07:01.008 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-24T09:07:01.039 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-24T09:07:01.059 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libcephfs2-2:19.0.0-3116.g1f81fb39.el9.x86_64 9/156 2024-04-24T09:07:01.076 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-24T09:07:01.088 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rados-2:19.0.0-3116.g1f81fb39.el9.x86_64 10/156 2024-04-24T09:07:01.122 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-24T09:07:01.158 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-24T09:07:01.201 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-24T09:07:01.237 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-24T09:07:01.244 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-24T09:07:01.286 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-24T09:07:01.314 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephsqlite-2:19.0.0-3116.g1f81fb39.el9.x86_64 13/156 2024-04-24T09:07:01.380 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-24T09:07:01.424 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-24T09:07:01.454 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-24T09:07:01.496 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-24T09:07:01.583 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3116.g1f81fb39.el9.x86_64 13/156 2024-04-24T09:07:01.588 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-24T09:07:01.618 INFO:teuthology.orchestra.run.smithi090.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 648 kB/s | 2.4 MB 00:03 2024-04-24T09:07:01.629 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-24T09:07:01.631 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libradosstriper1-2:19.0.0-3116.g1f81fb39.el9.x86 14/156 2024-04-24T09:07:01.633 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-24T09:07:01.633 INFO:teuthology.orchestra.run.smithi090.stdout:Total 6.3 MB/s | 222 MB 00:34 2024-04-24T09:07:01.669 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-24T09:07:01.685 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3116.g1f81fb39.el9.x86 14/156 2024-04-24T09:07:01.695 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-24T09:07:01.707 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T09:07:01.719 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-24T09:07:01.732 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T09:07:01.794 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-24T09:07:01.921 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-24T09:07:01.934 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T09:07:01.952 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-24T09:07:01.965 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T09:07:01.979 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-24T09:07:02.014 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-24T09:07:02.053 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-24T09:07:02.073 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-24T09:07:02.084 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-24T09:07:02.114 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-24T09:07:02.149 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-24T09:07:02.177 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-24T09:07:02.182 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-24T09:07:02.214 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-24T09:07:02.224 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-24T09:07:02.249 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-24T09:07:02.258 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-24T09:07:02.314 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-24T09:07:02.345 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-24T09:07:02.348 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-24T09:07:02.389 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-24T09:07:02.400 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-24T09:07:02.420 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-24T09:07:02.432 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-24T09:07:02.460 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-24T09:07:02.467 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-24T09:07:02.530 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-24T09:07:02.566 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-24T09:07:02.590 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-24T09:07:02.613 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-24T09:07:02.621 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-24T09:07:02.657 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-24T09:07:02.732 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-24T09:07:02.767 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-ceph-common-2:19.0.0-3116.g1f81fb39.el9. 31/156 2024-04-24T09:07:02.811 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-ceph-argparse-2:19.0.0-3116.g1f81fb39.el 32/156 2024-04-24T09:07:02.840 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cephfs-2:19.0.0-3116.g1f81fb39.el9.x86_6 33/156 2024-04-24T09:07:02.909 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-24T09:07:02.954 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-24T09:07:03.037 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-24T09:07:03.177 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-24T09:07:03.231 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-24T09:07:03.331 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-24T09:07:03.377 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-24T09:07:03.410 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-24T09:07:03.520 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-24T09:07:03.577 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-24T09:07:03.665 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-24T09:07:03.721 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-24T09:07:03.762 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-24T09:07:03.844 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-24T09:07:03.873 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-24T09:07:03.914 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-24T09:07:03.941 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-24T09:07:03.961 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-24T09:07:03.979 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librados-devel-2:19.0.0-3116.g1f81fb39.el9.x86_6 43/156 2024-04-24T09:07:04.021 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-24T09:07:04.039 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-24T09:07:04.190 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-24T09:07:04.249 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-24T09:07:04.304 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-24T09:07:04.376 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-24T09:07:04.408 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-24T09:07:04.444 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-24T09:07:04.490 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-24T09:07:04.516 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-24T09:07:04.526 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-24T09:07:04.570 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-24T09:07:04.612 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-24T09:07:04.688 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-24T09:07:04.748 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-24T09:07:04.760 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-24T09:07:04.760 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-24T09:07:04.784 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-24T09:07:04.814 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-24T09:07:04.864 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-24T09:07:04.910 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-24T09:07:04.957 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-24T09:07:04.989 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-24T09:07:05.029 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-24T09:07:05.054 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-24T09:07:05.079 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-24T09:07:05.154 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-24T09:07:05.311 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-24T09:07:05.345 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-24T09:07:05.373 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-24T09:07:05.410 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-24T09:07:05.455 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-24T09:07:05.486 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-24T09:07:05.516 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-24T09:07:05.550 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-24T09:07:05.584 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-24T09:07:05.625 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-24T09:07:05.660 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-24T09:07:05.747 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-24T09:07:05.791 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-24T09:07:05.831 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-24T09:07:05.877 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-24T09:07:05.975 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-24T09:07:06.023 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-24T09:07:06.135 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-24T09:07:06.207 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-24T09:07:06.234 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-24T09:07:06.267 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-24T09:07:06.582 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-24T09:07:06.622 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : librbd1-2:19.0.0-3116.g1f81fb39.el9.x86_64 99/156 2024-04-24T09:07:06.705 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librbd1-2:19.0.0-3116.g1f81fb39.el9.x86_64 99/156 2024-04-24T09:07:06.735 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-rbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 100/156 2024-04-24T09:07:06.749 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-24T09:07:06.794 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-24T09:07:06.795 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-24T09:07:06.828 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-24T09:07:06.918 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-24T09:07:06.978 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-24T09:07:07.066 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-24T09:07:07.098 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-24T09:07:07.277 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-24T09:07:07.312 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-24T09:07:07.389 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-24T09:07:07.439 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-24T09:07:07.475 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : librgw2-2:19.0.0-3116.g1f81fb39.el9.x86_64 103/156 2024-04-24T09:07:07.546 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librgw2-2:19.0.0-3116.g1f81fb39.el9.x86_64 103/156 2024-04-24T09:07:07.568 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-rgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 104/156 2024-04-24T09:07:07.701 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-24T09:07:07.701 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-24T09:07:08.030 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-24T09:07:08.144 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-24T09:07:09.386 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-24T09:07:09.474 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-24T09:07:09.511 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-24T09:07:09.543 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-24T09:07:09.856 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-24T09:07:09.876 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 105/156 2024-04-24T09:07:09.906 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : librbd1-2:19.0.0-3116.g1f81fb39.el9.x86_64 99/156 2024-04-24T09:07:09.907 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 105/156 2024-04-24T09:07:09.997 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librbd1-2:19.0.0-3116.g1f81fb39.el9.x86_64 99/156 2024-04-24T09:07:10.027 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 100/156 2024-04-24T09:07:10.064 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-24T09:07:10.445 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 105/156 2024-04-24T09:07:10.485 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-base-2:19.0.0-3116.g1f81fb39.el9.x86_64 106/156 2024-04-24T09:07:10.545 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-24T09:07:10.574 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-base-2:19.0.0-3116.g1f81fb39.el9.x86_64 106/156 2024-04-24T09:07:10.574 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T09:07:10.574 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T09:07:10.575 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-24T09:07:10.575 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librgw2-2:19.0.0-3116.g1f81fb39.el9.x86_64 103/156 2024-04-24T09:07:10.602 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-selinux-2:19.0.0-3116.g1f81fb39.el9.x86_64 107/156 2024-04-24T09:07:10.647 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librgw2-2:19.0.0-3116.g1f81fb39.el9.x86_64 103/156 2024-04-24T09:07:10.669 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 104/156 2024-04-24T09:07:12.965 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 105/156 2024-04-24T09:07:13.007 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 105/156 2024-04-24T09:07:13.550 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 105/156 2024-04-24T09:07:13.593 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-base-2:19.0.0-3116.g1f81fb39.el9.x86_64 106/156 2024-04-24T09:07:13.669 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-base-2:19.0.0-3116.g1f81fb39.el9.x86_64 106/156 2024-04-24T09:07:13.669 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T09:07:13.669 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T09:07:13.669 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T09:07:13.703 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-selinux-2:19.0.0-3116.g1f81fb39.el9.x86_64 107/156 2024-04-24T09:07:14.152 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-24T09:07:14.313 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-24T09:07:14.408 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-24T09:07:14.526 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-24T09:07:14.592 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-24T09:07:14.629 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-24T09:07:14.793 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-24T09:07:15.232 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-24T09:07:15.369 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : librados2-2:19.0.0-3116.g1f81fb39.el9.x86_64 8/156 2024-04-24T09:07:15.548 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librados2-2:19.0.0-3116.g1f81fb39.el9.x86_64 8/156 2024-04-24T09:07:15.587 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libcephfs2-2:19.0.0-3116.g1f81fb39.el9.x86_64 9/156 2024-04-24T09:07:15.717 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libcephfs2-2:19.0.0-3116.g1f81fb39.el9.x86_64 9/156 2024-04-24T09:07:15.754 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rados-2:19.0.0-3116.g1f81fb39.el9.x86_64 10/156 2024-04-24T09:07:16.110 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-24T09:07:16.176 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-24T09:07:16.210 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libcephsqlite-2:19.0.0-3116.g1f81fb39.el9.x86_64 13/156 2024-04-24T09:07:16.375 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3116.g1f81fb39.el9.x86_64 13/156 2024-04-24T09:07:16.403 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libradosstriper1-2:19.0.0-3116.g1f81fb39.el9.x86 14/156 2024-04-24T09:07:16.531 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3116.g1f81fb39.el9.x86 14/156 2024-04-24T09:07:16.562 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T09:07:16.635 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T09:07:16.966 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T09:07:17.023 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T09:07:17.219 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-24T09:07:17.339 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-24T09:07:17.423 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-24T09:07:17.475 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-24T09:07:17.583 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-24T09:07:17.628 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-24T09:07:17.701 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-24T09:07:17.759 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-24T09:07:17.800 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-24T09:07:17.932 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-24T09:07:18.003 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-24T09:07:18.042 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-24T09:07:18.109 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-24T09:07:18.229 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-24T09:07:18.272 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-ceph-common-2:19.0.0-3116.g1f81fb39.el9. 31/156 2024-04-24T09:07:18.366 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-ceph-argparse-2:19.0.0-3116.g1f81fb39.el 32/156 2024-04-24T09:07:18.404 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cephfs-2:19.0.0-3116.g1f81fb39.el9.x86_6 33/156 2024-04-24T09:07:18.528 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-24T09:07:18.592 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-24T09:07:18.704 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-24T09:07:18.883 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-24T09:07:18.952 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-24T09:07:19.636 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-24T09:07:19.787 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-24T09:07:19.828 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-24T09:07:19.877 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-24T09:07:19.923 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librados-devel-2:19.0.0-3116.g1f81fb39.el9.x86_6 43/156 2024-04-24T09:07:19.996 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-24T09:07:20.321 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-24T09:07:20.375 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-24T09:07:20.455 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-24T09:07:20.541 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-24T09:07:20.583 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-24T09:07:20.669 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-24T09:07:20.752 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-24T09:07:20.800 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-24T09:07:20.861 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-24T09:07:20.928 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-24T09:07:21.133 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-24T09:07:21.188 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-24T09:07:21.226 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-24T09:07:21.274 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-24T09:07:21.412 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-24T09:07:21.459 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-24T09:07:21.511 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-24T09:07:21.568 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-24T09:07:21.602 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-24T09:07:21.757 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3116.g1f81fb39.el9.x86_64 107/156 2024-04-24T09:07:21.758 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /sys 2024-04-24T09:07:21.758 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /proc 2024-04-24T09:07:21.758 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /mnt 2024-04-24T09:07:21.758 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /var/tmp 2024-04-24T09:07:21.758 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /home 2024-04-24T09:07:21.758 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /root 2024-04-24T09:07:21.758 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /tmp 2024-04-24T09:07:21.759 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-24T09:07:21.788 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-24T09:07:21.948 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-volume-2:19.0.0-3116.g1f81fb39.el9.noarch 108/156 2024-04-24T09:07:21.964 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-volume-2:19.0.0-3116.g1f81fb39.el9.noarch 108/156 2024-04-24T09:07:21.964 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T09:07:21.964 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-24T09:07:21.964 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-24T09:07:21.969 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-24T09:07:22.033 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-24T09:07:22.071 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-24T09:07:22.117 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-24T09:07:22.182 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-24T09:07:22.249 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-24T09:07:22.298 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-24T09:07:22.345 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-24T09:07:22.394 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-24T09:07:22.527 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-24T09:07:22.585 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-24T09:07:22.739 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-24T09:07:22.811 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-24T09:07:22.858 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-24T09:07:22.930 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-24T09:07:23.088 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-24T09:07:23.155 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-24T09:07:23.451 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-osd-2:19.0.0-3116.g1f81fb39.el9.x86_64 109/156 2024-04-24T09:07:23.476 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-osd-2:19.0.0-3116.g1f81fb39.el9.x86_64 109/156 2024-04-24T09:07:23.476 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T09:07:23.476 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T09:07:23.476 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T09:07:23.477 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T09:07:23.477 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-24T09:07:23.701 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mds-2:19.0.0-3116.g1f81fb39.el9.x86_64 110/156 2024-04-24T09:07:23.724 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mds-2:19.0.0-3116.g1f81fb39.el9.x86_64 110/156 2024-04-24T09:07:23.724 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T09:07:23.724 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T09:07:23.724 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T09:07:23.724 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T09:07:23.724 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-24T09:07:24.117 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-24T09:07:24.169 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mon-2:19.0.0-3116.g1f81fb39.el9.x86_64 111/156 2024-04-24T09:07:24.179 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-24T09:07:24.191 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mon-2:19.0.0-3116.g1f81fb39.el9.x86_64 111/156 2024-04-24T09:07:24.192 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T09:07:24.192 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T09:07:24.192 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T09:07:24.192 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T09:07:24.192 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-24T09:07:24.213 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-24T09:07:24.298 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-24T09:07:24.338 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-24T09:07:24.347 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-24T09:07:24.402 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-24T09:07:24.409 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-24T09:07:24.517 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-24T09:07:24.558 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-24T09:07:24.589 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-24T09:07:24.644 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-24T09:07:24.713 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-24T09:07:24.792 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-24T09:07:24.801 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T09:07:24.822 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-24T09:07:24.941 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T09:07:24.953 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-24T09:07:24.978 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-24T09:07:25.018 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-24T09:07:25.046 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-24T09:07:25.086 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3116.g1f81fb39.el9.x86_64 107/156 2024-04-24T09:07:25.087 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /sys 2024-04-24T09:07:25.087 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /proc 2024-04-24T09:07:25.087 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /mnt 2024-04-24T09:07:25.087 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /var/tmp 2024-04-24T09:07:25.087 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /home 2024-04-24T09:07:25.087 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /root 2024-04-24T09:07:25.087 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /tmp 2024-04-24T09:07:25.087 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T09:07:25.264 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-volume-2:19.0.0-3116.g1f81fb39.el9.noarch 108/156 2024-04-24T09:07:25.279 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-volume-2:19.0.0-3116.g1f81fb39.el9.noarch 108/156 2024-04-24T09:07:25.279 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T09:07:25.279 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-24T09:07:25.280 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T09:07:25.481 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: cephadm-2:19.0.0-3116.g1f81fb39.el9.noarch 123/156 2024-04-24T09:07:25.511 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : cephadm-2:19.0.0-3116.g1f81fb39.el9.noarch 123/156 2024-04-24T09:07:25.534 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3116.g1f81fb39.e 124/156 2024-04-24T09:07:25.604 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3116.g1f81fb39. 125/156 2024-04-24T09:07:25.634 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3116.g1f81fb39.el9.noa 126/156 2024-04-24T09:07:25.776 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-24T09:07:25.931 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-24T09:07:26.678 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3116.g1f81fb39.el9.noa 126/156 2024-04-24T09:07:26.719 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3116.g1f81fb39.el9.n 127/156 2024-04-24T09:07:26.750 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-osd-2:19.0.0-3116.g1f81fb39.el9.x86_64 109/156 2024-04-24T09:07:26.776 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-osd-2:19.0.0-3116.g1f81fb39.el9.x86_64 109/156 2024-04-24T09:07:26.776 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T09:07:26.776 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T09:07:26.776 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T09:07:26.776 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T09:07:26.776 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T09:07:27.001 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mds-2:19.0.0-3116.g1f81fb39.el9.x86_64 110/156 2024-04-24T09:07:27.023 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mds-2:19.0.0-3116.g1f81fb39.el9.x86_64 110/156 2024-04-24T09:07:27.023 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T09:07:27.023 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T09:07:27.023 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T09:07:27.023 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T09:07:27.023 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T09:07:27.468 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mon-2:19.0.0-3116.g1f81fb39.el9.x86_64 111/156 2024-04-24T09:07:27.491 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mon-2:19.0.0-3116.g1f81fb39.el9.x86_64 111/156 2024-04-24T09:07:27.491 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T09:07:27.491 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T09:07:27.491 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T09:07:27.491 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T09:07:27.491 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T09:07:27.492 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-24T09:07:27.590 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-24T09:07:27.593 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-24T09:07:27.638 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-24T09:07:27.653 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-24T09:07:27.689 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3116.g1f81fb39.el9.n 127/156 2024-04-24T09:07:27.691 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-24T09:07:27.702 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-24T09:07:27.719 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3116.g1f8 128/156 2024-04-24T09:07:27.821 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3116.g1f8 128/156 2024-04-24T09:07:27.888 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-24T09:07:27.935 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-24T09:07:27.983 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3116.g1f81fb39.el 129/156 2024-04-24T09:07:28.005 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-2:19.0.0-3116.g1f81fb39.el9.x86_64 130/156 2024-04-24T09:07:28.013 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-24T09:07:28.028 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3116.g1f81fb39.el9.x86_64 130/156 2024-04-24T09:07:28.028 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T09:07:28.028 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T09:07:28.028 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T09:07:28.028 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T09:07:28.028 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-24T09:07:28.072 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-rook-2:19.0.0-3116.g1f81fb39.el9.noarch 131/156 2024-04-24T09:07:28.083 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3116.g1f81fb39.el9.noarch 131/156 2024-04-24T09:07:28.094 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-24T09:07:28.110 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T09:07:28.148 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : librbd1-2:19.0.0-3116.g1f81fb39.el9.x86_64 99/156 2024-04-24T09:07:28.172 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-2:19.0.0-3116.g1f81fb39.el9.x86_64 132/156 2024-04-24T09:07:28.221 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 133/156 2024-04-24T09:07:28.250 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T09:07:28.272 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librbd1-2:19.0.0-3116.g1f81fb39.el9.x86_64 99/156 2024-04-24T09:07:28.294 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-24T09:07:28.314 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 100/156 2024-04-24T09:07:28.326 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-24T09:07:28.354 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-24T09:07:28.374 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-24T09:07:28.747 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: cephadm-2:19.0.0-3116.g1f81fb39.el9.noarch 123/156 2024-04-24T09:07:28.777 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephadm-2:19.0.0-3116.g1f81fb39.el9.noarch 123/156 2024-04-24T09:07:28.809 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3116.g1f81fb39.e 124/156 2024-04-24T09:07:28.871 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3116.g1f81fb39. 125/156 2024-04-24T09:07:28.900 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3116.g1f81fb39.el9.noa 126/156 2024-04-24T09:07:28.962 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-24T09:07:28.993 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librgw2-2:19.0.0-3116.g1f81fb39.el9.x86_64 103/156 2024-04-24T09:07:29.094 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librgw2-2:19.0.0-3116.g1f81fb39.el9.x86_64 103/156 2024-04-24T09:07:29.122 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 104/156 2024-04-24T09:07:29.140 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-24T09:07:29.167 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-radosgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 135/156 2024-04-24T09:07:29.189 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 135/156 2024-04-24T09:07:29.190 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T09:07:29.190 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T09:07:29.190 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T09:07:29.190 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T09:07:29.190 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-24T09:07:29.235 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3116.g1f81f 136/156 2024-04-24T09:07:29.256 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3116.g1f81f 136/156 2024-04-24T09:07:29.256 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T09:07:29.257 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T09:07:29.257 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-24T09:07:29.301 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : cephfs-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 137/156 2024-04-24T09:07:29.324 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 137/156 2024-04-24T09:07:29.324 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T09:07:29.324 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-24T09:07:29.324 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T09:07:29.324 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T09:07:29.324 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-24T09:07:29.626 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : rbd-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 138/156 2024-04-24T09:07:29.649 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 138/156 2024-04-24T09:07:29.649 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T09:07:29.649 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T09:07:29.649 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T09:07:29.649 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T09:07:29.649 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-24T09:07:29.947 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3116.g1f81fb39.el9.noa 126/156 2024-04-24T09:07:29.994 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3116.g1f81fb39.el9.n 127/156 2024-04-24T09:07:30.965 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3116.g1f81fb39.el9.n 127/156 2024-04-24T09:07:30.994 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3116.g1f8 128/156 2024-04-24T09:07:31.097 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3116.g1f8 128/156 2024-04-24T09:07:31.283 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3116.g1f81fb39.el 129/156 2024-04-24T09:07:31.313 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-2:19.0.0-3116.g1f81fb39.el9.x86_64 130/156 2024-04-24T09:07:31.336 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3116.g1f81fb39.el9.x86_64 130/156 2024-04-24T09:07:31.336 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T09:07:31.336 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T09:07:31.336 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T09:07:31.336 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T09:07:31.336 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T09:07:31.380 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-rook-2:19.0.0-3116.g1f81fb39.el9.noarch 131/156 2024-04-24T09:07:31.391 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3116.g1f81fb39.el9.noarch 131/156 2024-04-24T09:07:31.489 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-2:19.0.0-3116.g1f81fb39.el9.x86_64 132/156 2024-04-24T09:07:31.537 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 133/156 2024-04-24T09:07:31.820 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 105/156 2024-04-24T09:07:31.863 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 105/156 2024-04-24T09:07:32.466 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-24T09:07:32.505 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-radosgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 135/156 2024-04-24T09:07:32.511 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 105/156 2024-04-24T09:07:32.521 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 135/156 2024-04-24T09:07:32.521 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T09:07:32.521 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T09:07:32.522 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T09:07:32.522 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T09:07:32.522 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T09:07:32.567 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3116.g1f81f 136/156 2024-04-24T09:07:32.586 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-base-2:19.0.0-3116.g1f81fb39.el9.x86_64 106/156 2024-04-24T09:07:32.592 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3116.g1f81f 136/156 2024-04-24T09:07:32.592 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T09:07:32.593 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T09:07:32.593 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T09:07:32.634 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephfs-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 137/156 2024-04-24T09:07:32.656 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 137/156 2024-04-24T09:07:32.656 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T09:07:32.656 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-24T09:07:32.656 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T09:07:32.656 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T09:07:32.657 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T09:07:32.750 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-base-2:19.0.0-3116.g1f81fb39.el9.x86_64 106/156 2024-04-24T09:07:32.750 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T09:07:32.750 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T09:07:32.750 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:07:32.787 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-selinux-2:19.0.0-3116.g1f81fb39.el9.x86_64 107/156 2024-04-24T09:07:32.959 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 138/156 2024-04-24T09:07:32.985 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 138/156 2024-04-24T09:07:32.985 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T09:07:32.985 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T09:07:32.985 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T09:07:32.986 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T09:07:32.986 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T09:07:34.247 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-test-2:19.0.0-3116.g1f81fb39.el9.x86_64 139/156 2024-04-24T09:07:34.287 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : rbd-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 140/156 2024-04-24T09:07:34.329 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : rbd-nbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 141/156 2024-04-24T09:07:34.361 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-24T09:07:34.388 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-24T09:07:34.574 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : cephfs-shell-2:19.0.0-3116.g1f81fb39.el9.x86_64 144/156 2024-04-24T09:07:34.626 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-24T09:07:34.656 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libcephfs-devel-2:19.0.0-3116.g1f81fb39.el9.x86_ 146/156 2024-04-24T09:07:35.244 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T09:07:35.278 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T09:07:35.298 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T09:07:36.119 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : cephfs-top-2:19.0.0-3116.g1f81fb39.el9.noarch 148/156 2024-04-24T09:07:36.199 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-24T09:07:36.240 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-24T09:07:36.321 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-24T09:07:36.353 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-24T09:07:36.353 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T09:07:36.437 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T09:07:36.470 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T09:07:36.470 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T09:07:36.559 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T09:07:36.625 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T09:07:37.596 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-test-2:19.0.0-3116.g1f81fb39.el9.x86_64 139/156 2024-04-24T09:07:37.637 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 140/156 2024-04-24T09:07:37.678 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-nbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 141/156 2024-04-24T09:07:37.710 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-24T09:07:37.738 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-24T09:07:37.922 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephfs-shell-2:19.0.0-3116.g1f81fb39.el9.x86_64 144/156 2024-04-24T09:07:37.967 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-24T09:07:37.996 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephfs-devel-2:19.0.0-3116.g1f81fb39.el9.x86_ 146/156 2024-04-24T09:07:38.637 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T09:07:38.669 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T09:07:38.689 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T09:07:39.531 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephfs-top-2:19.0.0-3116.g1f81fb39.el9.noarch 148/156 2024-04-24T09:07:39.624 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-24T09:07:39.664 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-24T09:07:39.745 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-24T09:07:39.777 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-24T09:07:39.778 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T09:07:39.861 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T09:07:39.903 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T09:07:39.903 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T09:07:39.990 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T09:07:40.060 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T09:07:42.593 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T09:07:42.593 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-2:19.0.0-3116.g1f81fb39.el9.x86_64 1/156 2024-04-24T09:07:42.593 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-base-2:19.0.0-3116.g1f81fb39.el9.x86_64 2/156 2024-04-24T09:07:42.593 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 3/156 2024-04-24T09:07:42.593 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 4/156 2024-04-24T09:07:42.594 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3116.g1f81f 5/156 2024-04-24T09:07:42.594 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mds-2:19.0.0-3116.g1f81fb39.el9.x86_64 6/156 2024-04-24T09:07:42.594 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-2:19.0.0-3116.g1f81fb39.el9.x86_64 7/156 2024-04-24T09:07:42.594 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mon-2:19.0.0-3116.g1f81fb39.el9.x86_64 8/156 2024-04-24T09:07:42.594 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-osd-2:19.0.0-3116.g1f81fb39.el9.x86_64 9/156 2024-04-24T09:07:42.594 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-radosgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 10/156 2024-04-24T09:07:42.594 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-selinux-2:19.0.0-3116.g1f81fb39.el9.x86_64 11/156 2024-04-24T09:07:42.594 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-test-2:19.0.0-3116.g1f81fb39.el9.x86_64 12/156 2024-04-24T09:07:42.594 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : cephfs-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 13/156 2024-04-24T09:07:42.594 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : cephfs-shell-2:19.0.0-3116.g1f81fb39.el9.x86_64 14/156 2024-04-24T09:07:42.595 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libcephfs-devel-2:19.0.0-3116.g1f81fb39.el9.x86_ 15/156 2024-04-24T09:07:42.595 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libcephfs2-2:19.0.0-3116.g1f81fb39.el9.x86_64 16/156 2024-04-24T09:07:42.595 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libcephsqlite-2:19.0.0-3116.g1f81fb39.el9.x86_64 17/156 2024-04-24T09:07:42.595 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librados-devel-2:19.0.0-3116.g1f81fb39.el9.x86_6 18/156 2024-04-24T09:07:42.595 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libradosstriper1-2:19.0.0-3116.g1f81fb39.el9.x86 19/156 2024-04-24T09:07:42.595 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librgw2-2:19.0.0-3116.g1f81fb39.el9.x86_64 20/156 2024-04-24T09:07:42.595 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3116.g1f81fb39.el 21/156 2024-04-24T09:07:42.595 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-ceph-common-2:19.0.0-3116.g1f81fb39.el9. 22/156 2024-04-24T09:07:42.595 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cephfs-2:19.0.0-3116.g1f81fb39.el9.x86_6 23/156 2024-04-24T09:07:42.596 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-rados-2:19.0.0-3116.g1f81fb39.el9.x86_64 24/156 2024-04-24T09:07:42.596 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-rbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 25/156 2024-04-24T09:07:42.596 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-rgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 26/156 2024-04-24T09:07:42.596 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : rbd-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 27/156 2024-04-24T09:07:42.596 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : rbd-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 28/156 2024-04-24T09:07:42.596 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : rbd-nbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 29/156 2024-04-24T09:07:42.596 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3116.g1f81fb39. 30/156 2024-04-24T09:07:42.609 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3116.g1f81fb39.el9.noa 31/156 2024-04-24T09:07:42.609 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3116.g1f81fb39.el9.n 32/156 2024-04-24T09:07:42.609 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3116.g1f8 33/156 2024-04-24T09:07:42.610 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3116.g1f81fb39.el 34/156 2024-04-24T09:07:42.610 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3116.g1f81fb39.el9.noarch 35/156 2024-04-24T09:07:42.610 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3116.g1f81fb39.e 36/156 2024-04-24T09:07:42.610 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-volume-2:19.0.0-3116.g1f81fb39.el9.noarch 37/156 2024-04-24T09:07:42.610 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : cephadm-2:19.0.0-3116.g1f81fb39.el9.noarch 38/156 2024-04-24T09:07:42.610 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : cephfs-top-2:19.0.0-3116.g1f81fb39.el9.noarch 39/156 2024-04-24T09:07:42.610 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-24T09:07:42.610 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-24T09:07:42.614 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-24T09:07:42.614 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-24T09:07:42.614 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-24T09:07:42.614 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-24T09:07:42.614 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-24T09:07:42.614 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-24T09:07:42.614 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-24T09:07:42.614 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-24T09:07:42.615 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-24T09:07:42.615 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-24T09:07:42.615 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-24T09:07:42.615 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-24T09:07:42.615 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-24T09:07:42.615 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-24T09:07:42.615 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-24T09:07:42.615 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-24T09:07:42.615 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-24T09:07:42.615 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-24T09:07:42.615 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-24T09:07:42.616 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-24T09:07:42.616 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-24T09:07:42.616 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-24T09:07:42.616 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-24T09:07:42.616 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-24T09:07:42.616 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-24T09:07:42.616 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-24T09:07:42.616 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-24T09:07:42.616 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-24T09:07:42.616 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-24T09:07:42.616 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-24T09:07:42.616 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-24T09:07:42.616 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-24T09:07:42.616 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-24T09:07:42.617 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-24T09:07:42.617 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-24T09:07:42.617 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-24T09:07:42.617 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-24T09:07:42.617 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-24T09:07:42.617 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-24T09:07:42.617 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-24T09:07:42.617 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-24T09:07:42.617 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-24T09:07:42.617 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-24T09:07:42.617 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-24T09:07:42.617 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-24T09:07:42.617 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-24T09:07:42.618 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-24T09:07:42.618 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-24T09:07:42.618 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-24T09:07:42.618 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-24T09:07:42.618 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-24T09:07:42.618 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-24T09:07:42.618 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-24T09:07:42.618 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-24T09:07:42.618 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-24T09:07:42.618 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-24T09:07:42.618 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-24T09:07:42.618 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-24T09:07:42.618 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-24T09:07:42.619 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-24T09:07:42.619 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-24T09:07:42.619 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-24T09:07:42.619 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-24T09:07:42.619 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-24T09:07:42.619 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-24T09:07:42.619 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-24T09:07:42.619 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-24T09:07:42.619 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-24T09:07:42.619 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-24T09:07:42.619 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-24T09:07:42.619 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-24T09:07:42.619 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-24T09:07:42.619 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-24T09:07:42.620 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-24T09:07:42.620 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-24T09:07:42.620 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-24T09:07:42.620 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T09:07:42.620 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T09:07:42.620 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-24T09:07:42.620 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-24T09:07:42.620 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-24T09:07:42.620 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-24T09:07:42.620 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-24T09:07:42.620 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-24T09:07:42.620 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-24T09:07:42.620 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-24T09:07:42.620 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-24T09:07:42.620 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-24T09:07:42.620 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-24T09:07:42.621 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-24T09:07:42.621 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-24T09:07:42.621 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-24T09:07:42.621 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-24T09:07:42.621 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-24T09:07:42.621 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-24T09:07:42.621 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-24T09:07:42.621 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-24T09:07:42.621 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-24T09:07:42.621 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-24T09:07:42.621 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-24T09:07:42.621 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-24T09:07:42.621 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-24T09:07:42.621 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-24T09:07:42.622 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-24T09:07:42.622 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-24T09:07:42.622 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-24T09:07:42.622 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-24T09:07:42.622 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librados2-2:19.0.0-3116.g1f81fb39.el9.x86_64 149/156 2024-04-24T09:07:42.622 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-24T09:07:42.622 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librbd1-2:19.0.0-3116.g1f81fb39.el9.x86_64 151/156 2024-04-24T09:07:42.622 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-24T09:07:42.622 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-24T09:07:42.622 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T09:07:42.622 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-24T09:07:44.729 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T09:07:44.730 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-24T09:07:44.730 INFO:teuthology.orchestra.run.smithi114.stdout:Upgraded: 2024-04-24T09:07:44.730 INFO:teuthology.orchestra.run.smithi114.stdout: librados2-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:44.730 INFO:teuthology.orchestra.run.smithi114.stdout: librbd1-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:44.730 INFO:teuthology.orchestra.run.smithi114.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-24T09:07:44.730 INFO:teuthology.orchestra.run.smithi114.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-24T09:07:44.730 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-04-24T09:07:44.731 INFO:teuthology.orchestra.run.smithi114.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-24T09:07:44.731 INFO:teuthology.orchestra.run.smithi114.stdout: automake-1.16.2-8.el9.noarch 2024-04-24T09:07:44.731 INFO:teuthology.orchestra.run.smithi114.stdout: bison-3.7.4-5.el9.x86_64 2024-04-24T09:07:44.731 INFO:teuthology.orchestra.run.smithi114.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-24T09:07:44.731 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:44.731 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-base-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:44.731 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:44.731 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:44.731 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-grafana-dashboards-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T09:07:44.731 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-immutable-object-cache-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:44.731 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mds-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:44.731 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:44.731 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-cephadm-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T09:07:44.732 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-dashboard-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T09:07:44.732 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T09:07:44.732 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-modules-core-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T09:07:44.732 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-rook-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T09:07:44.732 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mon-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:44.732 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-osd-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:44.732 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-prometheus-alerts-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T09:07:44.732 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-radosgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:44.732 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-selinux-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:44.732 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-test-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:44.732 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-volume-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T09:07:44.732 INFO:teuthology.orchestra.run.smithi114.stdout: cephadm-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T09:07:44.732 INFO:teuthology.orchestra.run.smithi114.stdout: cephfs-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:44.733 INFO:teuthology.orchestra.run.smithi114.stdout: cephfs-shell-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:44.733 INFO:teuthology.orchestra.run.smithi114.stdout: cephfs-top-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T09:07:44.733 INFO:teuthology.orchestra.run.smithi114.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-24T09:07:44.733 INFO:teuthology.orchestra.run.smithi114.stdout: flex-2.6.4-9.el9.x86_64 2024-04-24T09:07:44.733 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T09:07:44.733 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T09:07:44.733 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T09:07:44.733 INFO:teuthology.orchestra.run.smithi114.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-24T09:07:44.733 INFO:teuthology.orchestra.run.smithi114.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T09:07:44.733 INFO:teuthology.orchestra.run.smithi114.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-24T09:07:44.733 INFO:teuthology.orchestra.run.smithi114.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-24T09:07:44.733 INFO:teuthology.orchestra.run.smithi114.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-24T09:07:44.733 INFO:teuthology.orchestra.run.smithi114.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-24T09:07:44.733 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs-devel-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:44.734 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs2-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:44.734 INFO:teuthology.orchestra.run.smithi114.stdout: libcephsqlite-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:44.734 INFO:teuthology.orchestra.run.smithi114.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T09:07:44.734 INFO:teuthology.orchestra.run.smithi114.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T09:07:44.734 INFO:teuthology.orchestra.run.smithi114.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T09:07:44.734 INFO:teuthology.orchestra.run.smithi114.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T09:07:44.734 INFO:teuthology.orchestra.run.smithi114.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T09:07:44.734 INFO:teuthology.orchestra.run.smithi114.stdout: librados-devel-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:44.734 INFO:teuthology.orchestra.run.smithi114.stdout: libradosstriper1-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:44.734 INFO:teuthology.orchestra.run.smithi114.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T09:07:44.734 INFO:teuthology.orchestra.run.smithi114.stdout: librgw2-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:44.734 INFO:teuthology.orchestra.run.smithi114.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-24T09:07:44.735 INFO:teuthology.orchestra.run.smithi114.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T09:07:44.735 INFO:teuthology.orchestra.run.smithi114.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-24T09:07:44.735 INFO:teuthology.orchestra.run.smithi114.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-24T09:07:44.735 INFO:teuthology.orchestra.run.smithi114.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T09:07:44.735 INFO:teuthology.orchestra.run.smithi114.stdout: lua-5.4.4-4.el9.x86_64 2024-04-24T09:07:44.735 INFO:teuthology.orchestra.run.smithi114.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-24T09:07:44.735 INFO:teuthology.orchestra.run.smithi114.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-24T09:07:44.735 INFO:teuthology.orchestra.run.smithi114.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T09:07:44.735 INFO:teuthology.orchestra.run.smithi114.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T09:07:44.735 INFO:teuthology.orchestra.run.smithi114.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-24T09:07:44.735 INFO:teuthology.orchestra.run.smithi114.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-24T09:07:44.736 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-13.14-1.el9.x86_64 2024-04-24T09:07:44.736 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-24T09:07:44.736 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-24T09:07:44.736 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-24T09:07:44.736 INFO:teuthology.orchestra.run.smithi114.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-24T09:07:44.736 INFO:teuthology.orchestra.run.smithi114.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-24T09:07:44.736 INFO:teuthology.orchestra.run.smithi114.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T09:07:44.736 INFO:teuthology.orchestra.run.smithi114.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-24T09:07:44.736 INFO:teuthology.orchestra.run.smithi114.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T09:07:44.736 INFO:teuthology.orchestra.run.smithi114.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T09:07:44.736 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T09:07:44.736 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-argparse-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:44.736 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:44.736 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cephfs-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:44.737 INFO:teuthology.orchestra.run.smithi114.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T09:07:44.737 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T09:07:44.737 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T09:07:44.737 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T09:07:44.737 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-24T09:07:44.737 INFO:teuthology.orchestra.run.smithi114.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-24T09:07:44.737 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T09:07:44.737 INFO:teuthology.orchestra.run.smithi114.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T09:07:44.737 INFO:teuthology.orchestra.run.smithi114.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T09:07:44.737 INFO:teuthology.orchestra.run.smithi114.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-24T09:07:44.737 INFO:teuthology.orchestra.run.smithi114.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-24T09:07:44.737 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T09:07:44.737 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T09:07:44.738 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T09:07:44.738 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T09:07:44.738 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T09:07:44.738 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T09:07:44.738 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T09:07:44.738 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T09:07:44.738 INFO:teuthology.orchestra.run.smithi114.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T09:07:44.738 INFO:teuthology.orchestra.run.smithi114.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T09:07:44.738 INFO:teuthology.orchestra.run.smithi114.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T09:07:44.738 INFO:teuthology.orchestra.run.smithi114.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T09:07:44.738 INFO:teuthology.orchestra.run.smithi114.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T09:07:44.738 INFO:teuthology.orchestra.run.smithi114.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T09:07:44.738 INFO:teuthology.orchestra.run.smithi114.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T09:07:44.738 INFO:teuthology.orchestra.run.smithi114.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T09:07:44.739 INFO:teuthology.orchestra.run.smithi114.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T09:07:44.739 INFO:teuthology.orchestra.run.smithi114.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-24T09:07:44.739 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T09:07:44.739 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T09:07:44.739 INFO:teuthology.orchestra.run.smithi114.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T09:07:44.739 INFO:teuthology.orchestra.run.smithi114.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T09:07:44.739 INFO:teuthology.orchestra.run.smithi114.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-24T09:07:44.739 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T09:07:44.739 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T09:07:44.739 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T09:07:44.739 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T09:07:44.739 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-24T09:07:44.739 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T09:07:44.740 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rados-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:44.740 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:44.740 INFO:teuthology.orchestra.run.smithi114.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T09:07:44.740 INFO:teuthology.orchestra.run.smithi114.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T09:07:44.740 INFO:teuthology.orchestra.run.smithi114.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T09:07:44.740 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:44.740 INFO:teuthology.orchestra.run.smithi114.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T09:07:44.740 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T09:07:44.740 INFO:teuthology.orchestra.run.smithi114.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T09:07:44.740 INFO:teuthology.orchestra.run.smithi114.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T09:07:44.740 INFO:teuthology.orchestra.run.smithi114.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T09:07:44.740 INFO:teuthology.orchestra.run.smithi114.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T09:07:44.740 INFO:teuthology.orchestra.run.smithi114.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T09:07:44.740 INFO:teuthology.orchestra.run.smithi114.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-24T09:07:44.741 INFO:teuthology.orchestra.run.smithi114.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T09:07:44.741 INFO:teuthology.orchestra.run.smithi114.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T09:07:44.741 INFO:teuthology.orchestra.run.smithi114.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T09:07:44.741 INFO:teuthology.orchestra.run.smithi114.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T09:07:44.741 INFO:teuthology.orchestra.run.smithi114.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-24T09:07:44.741 INFO:teuthology.orchestra.run.smithi114.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-24T09:07:44.741 INFO:teuthology.orchestra.run.smithi114.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-24T09:07:44.741 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:44.741 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:44.742 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-nbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:44.742 INFO:teuthology.orchestra.run.smithi114.stdout: re2-1:20211101-3.el9.x86_64 2024-04-24T09:07:44.742 INFO:teuthology.orchestra.run.smithi114.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-24T09:07:44.742 INFO:teuthology.orchestra.run.smithi114.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T09:07:44.742 INFO:teuthology.orchestra.run.smithi114.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-24T09:07:44.742 INFO:teuthology.orchestra.run.smithi114.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-24T09:07:44.742 INFO:teuthology.orchestra.run.smithi114.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T09:07:44.742 INFO:teuthology.orchestra.run.smithi114.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-24T09:07:44.743 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-04-24T09:07:44.743 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-04-24T09:07:45.155 DEBUG:teuthology.parallel:result is None 2024-04-24T09:07:45.922 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T09:07:45.922 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-2:19.0.0-3116.g1f81fb39.el9.x86_64 1/156 2024-04-24T09:07:45.922 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-base-2:19.0.0-3116.g1f81fb39.el9.x86_64 2/156 2024-04-24T09:07:45.922 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 3/156 2024-04-24T09:07:45.922 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 4/156 2024-04-24T09:07:45.922 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3116.g1f81f 5/156 2024-04-24T09:07:45.922 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mds-2:19.0.0-3116.g1f81fb39.el9.x86_64 6/156 2024-04-24T09:07:45.922 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-2:19.0.0-3116.g1f81fb39.el9.x86_64 7/156 2024-04-24T09:07:45.922 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mon-2:19.0.0-3116.g1f81fb39.el9.x86_64 8/156 2024-04-24T09:07:45.922 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-osd-2:19.0.0-3116.g1f81fb39.el9.x86_64 9/156 2024-04-24T09:07:45.922 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-radosgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 10/156 2024-04-24T09:07:45.923 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-selinux-2:19.0.0-3116.g1f81fb39.el9.x86_64 11/156 2024-04-24T09:07:45.923 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-test-2:19.0.0-3116.g1f81fb39.el9.x86_64 12/156 2024-04-24T09:07:45.923 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephfs-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 13/156 2024-04-24T09:07:45.923 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephfs-shell-2:19.0.0-3116.g1f81fb39.el9.x86_64 14/156 2024-04-24T09:07:45.923 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephfs-devel-2:19.0.0-3116.g1f81fb39.el9.x86_ 15/156 2024-04-24T09:07:45.923 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephfs2-2:19.0.0-3116.g1f81fb39.el9.x86_64 16/156 2024-04-24T09:07:45.923 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephsqlite-2:19.0.0-3116.g1f81fb39.el9.x86_64 17/156 2024-04-24T09:07:45.923 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados-devel-2:19.0.0-3116.g1f81fb39.el9.x86_6 18/156 2024-04-24T09:07:45.923 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libradosstriper1-2:19.0.0-3116.g1f81fb39.el9.x86 19/156 2024-04-24T09:07:45.923 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librgw2-2:19.0.0-3116.g1f81fb39.el9.x86_64 20/156 2024-04-24T09:07:45.923 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3116.g1f81fb39.el 21/156 2024-04-24T09:07:45.923 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-ceph-common-2:19.0.0-3116.g1f81fb39.el9. 22/156 2024-04-24T09:07:45.923 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cephfs-2:19.0.0-3116.g1f81fb39.el9.x86_6 23/156 2024-04-24T09:07:45.924 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rados-2:19.0.0-3116.g1f81fb39.el9.x86_64 24/156 2024-04-24T09:07:45.924 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 25/156 2024-04-24T09:07:45.924 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 26/156 2024-04-24T09:07:45.924 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 27/156 2024-04-24T09:07:45.924 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 28/156 2024-04-24T09:07:45.924 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-nbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 29/156 2024-04-24T09:07:45.924 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3116.g1f81fb39. 30/156 2024-04-24T09:07:45.924 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3116.g1f81fb39.el9.noa 31/156 2024-04-24T09:07:45.924 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3116.g1f81fb39.el9.n 32/156 2024-04-24T09:07:45.924 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3116.g1f8 33/156 2024-04-24T09:07:45.924 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3116.g1f81fb39.el 34/156 2024-04-24T09:07:45.924 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3116.g1f81fb39.el9.noarch 35/156 2024-04-24T09:07:45.924 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3116.g1f81fb39.e 36/156 2024-04-24T09:07:45.925 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-volume-2:19.0.0-3116.g1f81fb39.el9.noarch 37/156 2024-04-24T09:07:45.925 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephadm-2:19.0.0-3116.g1f81fb39.el9.noarch 38/156 2024-04-24T09:07:45.925 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephfs-top-2:19.0.0-3116.g1f81fb39.el9.noarch 39/156 2024-04-24T09:07:45.925 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-24T09:07:45.925 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-24T09:07:45.925 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-24T09:07:45.925 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-24T09:07:45.925 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-24T09:07:45.925 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-24T09:07:45.925 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-24T09:07:45.925 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-24T09:07:45.925 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-24T09:07:45.925 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-24T09:07:45.925 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-24T09:07:45.925 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-24T09:07:45.926 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-24T09:07:45.926 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-24T09:07:45.926 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-24T09:07:45.926 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-24T09:07:45.926 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-24T09:07:45.926 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-24T09:07:45.926 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-24T09:07:45.926 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-24T09:07:45.926 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-24T09:07:45.926 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-24T09:07:45.926 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-24T09:07:45.926 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-24T09:07:45.927 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-24T09:07:45.927 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-24T09:07:45.927 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-24T09:07:45.927 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-24T09:07:45.927 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-24T09:07:45.927 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-24T09:07:45.927 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-24T09:07:45.927 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-24T09:07:45.927 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-24T09:07:45.927 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-24T09:07:45.927 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-24T09:07:45.927 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-24T09:07:45.927 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-24T09:07:45.927 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-24T09:07:45.927 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-24T09:07:45.928 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-24T09:07:45.928 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-24T09:07:45.928 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-24T09:07:45.928 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-24T09:07:45.928 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-24T09:07:45.928 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-24T09:07:45.928 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-24T09:07:45.928 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-24T09:07:45.928 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-24T09:07:45.928 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-24T09:07:45.928 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-24T09:07:45.928 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-24T09:07:45.928 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-24T09:07:45.928 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-24T09:07:45.929 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-24T09:07:45.929 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-24T09:07:45.929 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-24T09:07:45.929 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-24T09:07:45.929 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-24T09:07:45.929 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-24T09:07:45.929 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-24T09:07:45.929 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-24T09:07:45.929 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-24T09:07:45.929 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-24T09:07:45.929 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-24T09:07:45.929 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-24T09:07:45.930 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-24T09:07:45.930 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-24T09:07:45.930 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-24T09:07:45.930 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-24T09:07:45.930 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-24T09:07:45.930 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-24T09:07:45.930 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-24T09:07:45.930 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-24T09:07:45.930 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-24T09:07:45.930 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-24T09:07:45.930 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-24T09:07:45.930 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-24T09:07:45.930 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-24T09:07:45.930 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T09:07:45.931 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T09:07:45.931 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-24T09:07:45.931 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-24T09:07:45.931 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-24T09:07:45.931 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-24T09:07:45.931 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-24T09:07:45.931 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-24T09:07:45.931 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-24T09:07:45.931 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-24T09:07:45.931 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-24T09:07:45.931 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-24T09:07:45.931 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-24T09:07:45.931 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-24T09:07:45.931 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-24T09:07:45.931 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-24T09:07:45.932 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-24T09:07:45.932 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-24T09:07:45.932 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-24T09:07:45.932 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-24T09:07:45.932 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-24T09:07:45.932 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-24T09:07:45.932 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-24T09:07:45.932 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-24T09:07:45.932 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-24T09:07:45.932 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-24T09:07:45.932 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-24T09:07:45.932 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-24T09:07:45.932 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-24T09:07:45.933 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-24T09:07:45.933 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-24T09:07:45.933 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados2-2:19.0.0-3116.g1f81fb39.el9.x86_64 149/156 2024-04-24T09:07:45.933 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-24T09:07:45.933 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librbd1-2:19.0.0-3116.g1f81fb39.el9.x86_64 151/156 2024-04-24T09:07:45.933 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-24T09:07:45.933 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-24T09:07:45.933 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T09:07:45.933 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-24T09:07:46.597 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3116.g1f81fb39.el9.x86_64 107/156 2024-04-24T09:07:46.597 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /sys 2024-04-24T09:07:46.597 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /proc 2024-04-24T09:07:46.597 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /mnt 2024-04-24T09:07:46.597 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /var/tmp 2024-04-24T09:07:46.597 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /home 2024-04-24T09:07:46.597 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /root 2024-04-24T09:07:46.597 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /tmp 2024-04-24T09:07:46.598 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:07:46.828 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-volume-2:19.0.0-3116.g1f81fb39.el9.noarch 108/156 2024-04-24T09:07:46.868 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-volume-2:19.0.0-3116.g1f81fb39.el9.noarch 108/156 2024-04-24T09:07:46.868 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T09:07:46.868 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-24T09:07:46.868 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:07:47.971 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T09:07:47.972 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T09:07:47.972 INFO:teuthology.orchestra.run.smithi149.stdout:Upgraded: 2024-04-24T09:07:47.972 INFO:teuthology.orchestra.run.smithi149.stdout: librados2-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:47.972 INFO:teuthology.orchestra.run.smithi149.stdout: librbd1-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:47.972 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-24T09:07:47.972 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-24T09:07:47.972 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-04-24T09:07:47.972 INFO:teuthology.orchestra.run.smithi149.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-24T09:07:47.972 INFO:teuthology.orchestra.run.smithi149.stdout: automake-1.16.2-8.el9.noarch 2024-04-24T09:07:47.972 INFO:teuthology.orchestra.run.smithi149.stdout: bison-3.7.4-5.el9.x86_64 2024-04-24T09:07:47.972 INFO:teuthology.orchestra.run.smithi149.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-24T09:07:47.972 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:47.972 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-base-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:47.973 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:47.973 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:47.973 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-grafana-dashboards-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T09:07:47.973 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-immutable-object-cache-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:47.973 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mds-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:47.973 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:47.973 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-cephadm-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T09:07:47.973 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-dashboard-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T09:07:47.973 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T09:07:47.973 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-modules-core-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T09:07:47.973 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-rook-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T09:07:47.973 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mon-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:47.973 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-osd-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:47.973 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-prometheus-alerts-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T09:07:47.974 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-radosgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:47.974 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-selinux-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:47.974 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-test-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:47.974 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-volume-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T09:07:47.974 INFO:teuthology.orchestra.run.smithi149.stdout: cephadm-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T09:07:47.974 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:47.974 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-shell-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:47.974 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-top-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T09:07:47.974 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-24T09:07:47.974 INFO:teuthology.orchestra.run.smithi149.stdout: flex-2.6.4-9.el9.x86_64 2024-04-24T09:07:47.974 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T09:07:47.974 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T09:07:47.974 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T09:07:47.975 INFO:teuthology.orchestra.run.smithi149.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-24T09:07:47.975 INFO:teuthology.orchestra.run.smithi149.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T09:07:47.975 INFO:teuthology.orchestra.run.smithi149.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-24T09:07:47.975 INFO:teuthology.orchestra.run.smithi149.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-24T09:07:47.975 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-24T09:07:47.975 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-24T09:07:47.975 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs-devel-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:47.975 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs2-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:47.975 INFO:teuthology.orchestra.run.smithi149.stdout: libcephsqlite-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:47.975 INFO:teuthology.orchestra.run.smithi149.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T09:07:47.975 INFO:teuthology.orchestra.run.smithi149.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T09:07:47.975 INFO:teuthology.orchestra.run.smithi149.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T09:07:47.975 INFO:teuthology.orchestra.run.smithi149.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T09:07:47.975 INFO:teuthology.orchestra.run.smithi149.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T09:07:47.976 INFO:teuthology.orchestra.run.smithi149.stdout: librados-devel-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:47.976 INFO:teuthology.orchestra.run.smithi149.stdout: libradosstriper1-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:47.976 INFO:teuthology.orchestra.run.smithi149.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T09:07:47.976 INFO:teuthology.orchestra.run.smithi149.stdout: librgw2-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:47.976 INFO:teuthology.orchestra.run.smithi149.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-24T09:07:47.976 INFO:teuthology.orchestra.run.smithi149.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T09:07:47.976 INFO:teuthology.orchestra.run.smithi149.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-24T09:07:47.976 INFO:teuthology.orchestra.run.smithi149.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-24T09:07:47.976 INFO:teuthology.orchestra.run.smithi149.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T09:07:47.976 INFO:teuthology.orchestra.run.smithi149.stdout: lua-5.4.4-4.el9.x86_64 2024-04-24T09:07:47.976 INFO:teuthology.orchestra.run.smithi149.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-24T09:07:47.976 INFO:teuthology.orchestra.run.smithi149.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-24T09:07:47.976 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T09:07:47.976 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T09:07:47.977 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-24T09:07:47.977 INFO:teuthology.orchestra.run.smithi149.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-24T09:07:47.977 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-13.14-1.el9.x86_64 2024-04-24T09:07:47.977 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-24T09:07:47.977 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-24T09:07:47.977 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-24T09:07:47.977 INFO:teuthology.orchestra.run.smithi149.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-24T09:07:47.977 INFO:teuthology.orchestra.run.smithi149.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-24T09:07:47.977 INFO:teuthology.orchestra.run.smithi149.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T09:07:47.977 INFO:teuthology.orchestra.run.smithi149.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-24T09:07:47.977 INFO:teuthology.orchestra.run.smithi149.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T09:07:47.977 INFO:teuthology.orchestra.run.smithi149.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T09:07:47.977 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T09:07:47.977 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-argparse-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:47.977 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:47.978 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cephfs-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:47.978 INFO:teuthology.orchestra.run.smithi149.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T09:07:47.978 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T09:07:47.978 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T09:07:47.978 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T09:07:47.978 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-24T09:07:47.978 INFO:teuthology.orchestra.run.smithi149.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-24T09:07:47.978 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T09:07:47.978 INFO:teuthology.orchestra.run.smithi149.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T09:07:47.978 INFO:teuthology.orchestra.run.smithi149.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T09:07:47.978 INFO:teuthology.orchestra.run.smithi149.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-24T09:07:47.978 INFO:teuthology.orchestra.run.smithi149.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-24T09:07:47.978 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T09:07:47.979 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T09:07:47.979 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T09:07:47.979 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T09:07:47.979 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T09:07:47.979 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T09:07:47.979 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T09:07:47.979 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T09:07:47.979 INFO:teuthology.orchestra.run.smithi149.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T09:07:47.979 INFO:teuthology.orchestra.run.smithi149.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T09:07:47.979 INFO:teuthology.orchestra.run.smithi149.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T09:07:47.979 INFO:teuthology.orchestra.run.smithi149.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T09:07:47.979 INFO:teuthology.orchestra.run.smithi149.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T09:07:47.979 INFO:teuthology.orchestra.run.smithi149.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T09:07:47.979 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T09:07:47.980 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T09:07:47.980 INFO:teuthology.orchestra.run.smithi149.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T09:07:47.980 INFO:teuthology.orchestra.run.smithi149.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-24T09:07:47.980 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T09:07:47.980 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T09:07:47.980 INFO:teuthology.orchestra.run.smithi149.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T09:07:47.980 INFO:teuthology.orchestra.run.smithi149.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T09:07:47.980 INFO:teuthology.orchestra.run.smithi149.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-24T09:07:47.980 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T09:07:47.980 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T09:07:47.980 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T09:07:47.980 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T09:07:47.980 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-24T09:07:47.980 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T09:07:47.980 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rados-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:47.981 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:47.981 INFO:teuthology.orchestra.run.smithi149.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T09:07:47.981 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T09:07:47.981 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T09:07:47.981 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:47.981 INFO:teuthology.orchestra.run.smithi149.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T09:07:47.981 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T09:07:47.981 INFO:teuthology.orchestra.run.smithi149.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T09:07:47.981 INFO:teuthology.orchestra.run.smithi149.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T09:07:47.981 INFO:teuthology.orchestra.run.smithi149.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T09:07:47.981 INFO:teuthology.orchestra.run.smithi149.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T09:07:47.981 INFO:teuthology.orchestra.run.smithi149.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T09:07:47.981 INFO:teuthology.orchestra.run.smithi149.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-24T09:07:47.981 INFO:teuthology.orchestra.run.smithi149.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T09:07:47.981 INFO:teuthology.orchestra.run.smithi149.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T09:07:47.982 INFO:teuthology.orchestra.run.smithi149.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T09:07:47.982 INFO:teuthology.orchestra.run.smithi149.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T09:07:47.982 INFO:teuthology.orchestra.run.smithi149.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-24T09:07:47.982 INFO:teuthology.orchestra.run.smithi149.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-24T09:07:47.982 INFO:teuthology.orchestra.run.smithi149.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-24T09:07:47.982 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:47.982 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:47.982 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-nbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:07:47.982 INFO:teuthology.orchestra.run.smithi149.stdout: re2-1:20211101-3.el9.x86_64 2024-04-24T09:07:47.982 INFO:teuthology.orchestra.run.smithi149.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-24T09:07:47.982 INFO:teuthology.orchestra.run.smithi149.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T09:07:47.982 INFO:teuthology.orchestra.run.smithi149.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-24T09:07:47.982 INFO:teuthology.orchestra.run.smithi149.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-24T09:07:47.982 INFO:teuthology.orchestra.run.smithi149.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T09:07:47.983 INFO:teuthology.orchestra.run.smithi149.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-24T09:07:47.983 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T09:07:47.983 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-04-24T09:07:48.363 DEBUG:teuthology.parallel:result is None 2024-04-24T09:07:48.499 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-osd-2:19.0.0-3116.g1f81fb39.el9.x86_64 109/156 2024-04-24T09:07:48.578 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-osd-2:19.0.0-3116.g1f81fb39.el9.x86_64 109/156 2024-04-24T09:07:48.578 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T09:07:48.578 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T09:07:48.578 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T09:07:48.578 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T09:07:48.578 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:07:48.849 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mds-2:19.0.0-3116.g1f81fb39.el9.x86_64 110/156 2024-04-24T09:07:48.912 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mds-2:19.0.0-3116.g1f81fb39.el9.x86_64 110/156 2024-04-24T09:07:48.912 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T09:07:48.912 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T09:07:48.912 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T09:07:48.912 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T09:07:48.912 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:07:49.408 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mon-2:19.0.0-3116.g1f81fb39.el9.x86_64 111/156 2024-04-24T09:07:49.482 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mon-2:19.0.0-3116.g1f81fb39.el9.x86_64 111/156 2024-04-24T09:07:49.482 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T09:07:49.482 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T09:07:49.482 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T09:07:49.482 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T09:07:49.482 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:07:49.609 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-24T09:07:49.668 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-24T09:07:49.744 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-24T09:07:49.980 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-24T09:07:50.048 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-24T09:07:50.132 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-24T09:07:50.250 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T09:07:50.417 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T09:07:50.474 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-24T09:07:50.514 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-24T09:07:50.548 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-24T09:07:51.263 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: cephadm-2:19.0.0-3116.g1f81fb39.el9.noarch 123/156 2024-04-24T09:07:51.303 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : cephadm-2:19.0.0-3116.g1f81fb39.el9.noarch 123/156 2024-04-24T09:07:51.388 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3116.g1f81fb39.e 124/156 2024-04-24T09:07:51.506 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3116.g1f81fb39. 125/156 2024-04-24T09:07:51.538 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3116.g1f81fb39.el9.noa 126/156 2024-04-24T09:07:52.858 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3116.g1f81fb39.el9.noa 126/156 2024-04-24T09:07:52.925 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3116.g1f81fb39.el9.n 127/156 2024-04-24T09:07:54.021 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3116.g1f81fb39.el9.n 127/156 2024-04-24T09:07:54.058 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3116.g1f8 128/156 2024-04-24T09:07:54.204 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3116.g1f8 128/156 2024-04-24T09:07:54.423 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3116.g1f81fb39.el 129/156 2024-04-24T09:07:54.452 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-2:19.0.0-3116.g1f81fb39.el9.x86_64 130/156 2024-04-24T09:07:54.522 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3116.g1f81fb39.el9.x86_64 130/156 2024-04-24T09:07:54.522 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T09:07:54.522 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T09:07:54.522 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T09:07:54.522 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T09:07:54.522 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:07:54.578 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-rook-2:19.0.0-3116.g1f81fb39.el9.noarch 131/156 2024-04-24T09:07:54.616 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3116.g1f81fb39.el9.noarch 131/156 2024-04-24T09:07:54.721 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-2:19.0.0-3116.g1f81fb39.el9.x86_64 132/156 2024-04-24T09:07:54.800 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 133/156 2024-04-24T09:07:55.811 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-24T09:07:55.848 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-radosgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 135/156 2024-04-24T09:07:55.918 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 135/156 2024-04-24T09:07:55.918 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T09:07:55.918 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T09:07:55.918 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T09:07:55.918 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T09:07:55.918 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:07:55.967 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3116.g1f81f 136/156 2024-04-24T09:07:56.038 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3116.g1f81f 136/156 2024-04-24T09:07:56.038 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T09:07:56.038 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T09:07:56.038 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:07:56.134 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : cephfs-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 137/156 2024-04-24T09:07:56.208 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 137/156 2024-04-24T09:07:56.208 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T09:07:56.208 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-24T09:07:56.208 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T09:07:56.208 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T09:07:56.208 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:07:56.618 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : rbd-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 138/156 2024-04-24T09:07:56.689 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 138/156 2024-04-24T09:07:56.689 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T09:07:56.689 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T09:07:56.689 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T09:07:56.689 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T09:07:56.689 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:08:01.759 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-test-2:19.0.0-3116.g1f81fb39.el9.x86_64 139/156 2024-04-24T09:08:01.820 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : rbd-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 140/156 2024-04-24T09:08:01.897 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : rbd-nbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 141/156 2024-04-24T09:08:01.942 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-24T09:08:01.990 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-24T09:08:02.331 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : cephfs-shell-2:19.0.0-3116.g1f81fb39.el9.x86_64 144/156 2024-04-24T09:08:02.396 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-24T09:08:02.424 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libcephfs-devel-2:19.0.0-3116.g1f81fb39.el9.x86_ 146/156 2024-04-24T09:08:03.358 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T09:08:03.408 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T09:08:03.462 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T09:08:04.839 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : cephfs-top-2:19.0.0-3116.g1f81fb39.el9.noarch 148/156 2024-04-24T09:08:04.953 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-24T09:08:05.034 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-24T09:08:05.188 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-24T09:08:05.234 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-24T09:08:05.234 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T09:08:05.378 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T09:08:05.426 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T09:08:05.426 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T09:08:05.582 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T09:08:05.715 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T09:08:06.023 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T09:08:06.023 INFO:teuthology.orchestra.run.smithi090.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-24T09:08:06.023 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:08:13.374 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-2:19.0.0-3116.g1f81fb39.el9.x86_64 1/156 2024-04-24T09:08:13.374 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-base-2:19.0.0-3116.g1f81fb39.el9.x86_64 2/156 2024-04-24T09:08:13.374 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 3/156 2024-04-24T09:08:13.374 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 4/156 2024-04-24T09:08:13.374 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3116.g1f81f 5/156 2024-04-24T09:08:13.375 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mds-2:19.0.0-3116.g1f81fb39.el9.x86_64 6/156 2024-04-24T09:08:13.375 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-2:19.0.0-3116.g1f81fb39.el9.x86_64 7/156 2024-04-24T09:08:13.375 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mon-2:19.0.0-3116.g1f81fb39.el9.x86_64 8/156 2024-04-24T09:08:13.375 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-osd-2:19.0.0-3116.g1f81fb39.el9.x86_64 9/156 2024-04-24T09:08:13.375 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-radosgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 10/156 2024-04-24T09:08:13.375 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-selinux-2:19.0.0-3116.g1f81fb39.el9.x86_64 11/156 2024-04-24T09:08:13.375 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-test-2:19.0.0-3116.g1f81fb39.el9.x86_64 12/156 2024-04-24T09:08:13.375 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : cephfs-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 13/156 2024-04-24T09:08:13.375 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : cephfs-shell-2:19.0.0-3116.g1f81fb39.el9.x86_64 14/156 2024-04-24T09:08:13.375 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libcephfs-devel-2:19.0.0-3116.g1f81fb39.el9.x86_ 15/156 2024-04-24T09:08:13.375 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libcephfs2-2:19.0.0-3116.g1f81fb39.el9.x86_64 16/156 2024-04-24T09:08:13.375 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libcephsqlite-2:19.0.0-3116.g1f81fb39.el9.x86_64 17/156 2024-04-24T09:08:13.375 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librados-devel-2:19.0.0-3116.g1f81fb39.el9.x86_6 18/156 2024-04-24T09:08:13.375 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libradosstriper1-2:19.0.0-3116.g1f81fb39.el9.x86 19/156 2024-04-24T09:08:13.376 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librgw2-2:19.0.0-3116.g1f81fb39.el9.x86_64 20/156 2024-04-24T09:08:13.376 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3116.g1f81fb39.el 21/156 2024-04-24T09:08:13.376 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-ceph-common-2:19.0.0-3116.g1f81fb39.el9. 22/156 2024-04-24T09:08:13.376 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cephfs-2:19.0.0-3116.g1f81fb39.el9.x86_6 23/156 2024-04-24T09:08:13.376 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rados-2:19.0.0-3116.g1f81fb39.el9.x86_64 24/156 2024-04-24T09:08:13.376 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 25/156 2024-04-24T09:08:13.376 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 26/156 2024-04-24T09:08:13.376 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : rbd-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 27/156 2024-04-24T09:08:13.388 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : rbd-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 28/156 2024-04-24T09:08:13.389 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : rbd-nbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 29/156 2024-04-24T09:08:13.389 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3116.g1f81fb39. 30/156 2024-04-24T09:08:13.389 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3116.g1f81fb39.el9.noa 31/156 2024-04-24T09:08:13.389 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3116.g1f81fb39.el9.n 32/156 2024-04-24T09:08:13.389 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3116.g1f8 33/156 2024-04-24T09:08:13.389 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3116.g1f81fb39.el 34/156 2024-04-24T09:08:13.389 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3116.g1f81fb39.el9.noarch 35/156 2024-04-24T09:08:13.389 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3116.g1f81fb39.e 36/156 2024-04-24T09:08:13.389 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-volume-2:19.0.0-3116.g1f81fb39.el9.noarch 37/156 2024-04-24T09:08:13.389 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : cephadm-2:19.0.0-3116.g1f81fb39.el9.noarch 38/156 2024-04-24T09:08:13.389 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : cephfs-top-2:19.0.0-3116.g1f81fb39.el9.noarch 39/156 2024-04-24T09:08:13.389 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-24T09:08:13.389 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-24T09:08:13.390 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-24T09:08:13.390 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-24T09:08:13.390 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-24T09:08:13.390 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-24T09:08:13.390 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-24T09:08:13.390 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-24T09:08:13.390 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-24T09:08:13.390 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-24T09:08:13.390 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-24T09:08:13.390 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-24T09:08:13.390 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-24T09:08:13.390 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-24T09:08:13.391 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-24T09:08:13.391 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-24T09:08:13.391 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-24T09:08:13.391 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-24T09:08:13.391 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-24T09:08:13.391 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-24T09:08:13.391 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-24T09:08:13.391 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-24T09:08:13.391 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-24T09:08:13.391 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-24T09:08:13.391 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-24T09:08:13.391 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-24T09:08:13.391 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-24T09:08:13.391 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-24T09:08:13.392 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-24T09:08:13.392 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-24T09:08:13.392 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-24T09:08:13.392 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-24T09:08:13.392 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-24T09:08:13.392 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-24T09:08:13.392 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-24T09:08:13.392 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-24T09:08:13.392 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-24T09:08:13.392 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-24T09:08:13.392 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-24T09:08:13.392 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-24T09:08:13.392 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-24T09:08:13.392 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-24T09:08:13.393 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-24T09:08:13.393 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-24T09:08:13.393 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-24T09:08:13.393 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-24T09:08:13.393 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-24T09:08:13.393 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-24T09:08:13.393 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-24T09:08:13.393 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-24T09:08:13.393 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-24T09:08:13.393 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-24T09:08:13.393 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-24T09:08:13.394 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-24T09:08:13.394 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-24T09:08:13.394 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-24T09:08:13.394 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-24T09:08:13.394 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-24T09:08:13.394 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-24T09:08:13.394 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-24T09:08:13.394 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-24T09:08:13.394 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-24T09:08:13.394 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-24T09:08:13.394 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-24T09:08:13.395 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-24T09:08:13.395 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-24T09:08:13.395 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-24T09:08:13.395 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-24T09:08:13.395 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-24T09:08:13.395 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-24T09:08:13.395 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-24T09:08:13.395 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-24T09:08:13.395 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-24T09:08:13.395 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-24T09:08:13.396 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-24T09:08:13.396 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-24T09:08:13.396 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-24T09:08:13.396 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-24T09:08:13.396 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T09:08:13.396 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T09:08:13.396 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-24T09:08:13.396 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-24T09:08:13.396 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-24T09:08:13.396 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-24T09:08:13.396 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-24T09:08:13.396 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-24T09:08:13.396 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-24T09:08:13.396 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-24T09:08:13.396 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-24T09:08:13.397 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-24T09:08:13.397 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-24T09:08:13.397 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-24T09:08:13.397 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-24T09:08:13.397 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-24T09:08:13.397 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-24T09:08:13.397 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-24T09:08:13.397 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-24T09:08:13.397 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-24T09:08:13.397 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-24T09:08:13.397 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-24T09:08:13.398 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-24T09:08:13.398 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-24T09:08:13.398 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-24T09:08:13.398 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-24T09:08:13.398 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-24T09:08:13.398 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-24T09:08:13.398 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-24T09:08:13.398 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-24T09:08:13.398 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-24T09:08:13.398 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librados2-2:19.0.0-3116.g1f81fb39.el9.x86_64 149/156 2024-04-24T09:08:13.398 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-24T09:08:13.398 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librbd1-2:19.0.0-3116.g1f81fb39.el9.x86_64 151/156 2024-04-24T09:08:13.398 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-24T09:08:13.398 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-24T09:08:13.399 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T09:08:13.399 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-24T09:08:15.830 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T09:08:15.831 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:08:15.831 INFO:teuthology.orchestra.run.smithi090.stdout:Upgraded: 2024-04-24T09:08:15.831 INFO:teuthology.orchestra.run.smithi090.stdout: librados2-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:08:15.831 INFO:teuthology.orchestra.run.smithi090.stdout: librbd1-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:08:15.831 INFO:teuthology.orchestra.run.smithi090.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-24T09:08:15.831 INFO:teuthology.orchestra.run.smithi090.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-24T09:08:15.831 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-04-24T09:08:15.831 INFO:teuthology.orchestra.run.smithi090.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-24T09:08:15.831 INFO:teuthology.orchestra.run.smithi090.stdout: automake-1.16.2-8.el9.noarch 2024-04-24T09:08:15.831 INFO:teuthology.orchestra.run.smithi090.stdout: bison-3.7.4-5.el9.x86_64 2024-04-24T09:08:15.831 INFO:teuthology.orchestra.run.smithi090.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-24T09:08:15.831 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:08:15.831 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-base-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:08:15.831 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:08:15.832 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:08:15.832 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-grafana-dashboards-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T09:08:15.832 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-immutable-object-cache-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:08:15.832 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mds-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:08:15.832 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:08:15.832 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-cephadm-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T09:08:15.832 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-dashboard-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T09:08:15.832 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T09:08:15.832 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-modules-core-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T09:08:15.832 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-rook-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T09:08:15.832 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mon-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:08:15.833 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-osd-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:08:15.833 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-prometheus-alerts-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T09:08:15.833 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-radosgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:08:15.833 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-selinux-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:08:15.833 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-test-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:08:15.833 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-volume-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T09:08:15.833 INFO:teuthology.orchestra.run.smithi090.stdout: cephadm-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T09:08:15.833 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:08:15.833 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-shell-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:08:15.833 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-top-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T09:08:15.833 INFO:teuthology.orchestra.run.smithi090.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-24T09:08:15.833 INFO:teuthology.orchestra.run.smithi090.stdout: flex-2.6.4-9.el9.x86_64 2024-04-24T09:08:15.833 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T09:08:15.833 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T09:08:15.833 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T09:08:15.834 INFO:teuthology.orchestra.run.smithi090.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-24T09:08:15.834 INFO:teuthology.orchestra.run.smithi090.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T09:08:15.834 INFO:teuthology.orchestra.run.smithi090.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-24T09:08:15.834 INFO:teuthology.orchestra.run.smithi090.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-24T09:08:15.834 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-24T09:08:15.834 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-24T09:08:15.834 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs-devel-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:08:15.834 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs2-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:08:15.834 INFO:teuthology.orchestra.run.smithi090.stdout: libcephsqlite-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:08:15.835 INFO:teuthology.orchestra.run.smithi090.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T09:08:15.835 INFO:teuthology.orchestra.run.smithi090.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T09:08:15.835 INFO:teuthology.orchestra.run.smithi090.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T09:08:15.835 INFO:teuthology.orchestra.run.smithi090.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T09:08:15.835 INFO:teuthology.orchestra.run.smithi090.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T09:08:15.835 INFO:teuthology.orchestra.run.smithi090.stdout: librados-devel-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:08:15.835 INFO:teuthology.orchestra.run.smithi090.stdout: libradosstriper1-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:08:15.835 INFO:teuthology.orchestra.run.smithi090.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T09:08:15.835 INFO:teuthology.orchestra.run.smithi090.stdout: librgw2-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:08:15.835 INFO:teuthology.orchestra.run.smithi090.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-24T09:08:15.835 INFO:teuthology.orchestra.run.smithi090.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T09:08:15.835 INFO:teuthology.orchestra.run.smithi090.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-24T09:08:15.835 INFO:teuthology.orchestra.run.smithi090.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-24T09:08:15.835 INFO:teuthology.orchestra.run.smithi090.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T09:08:15.835 INFO:teuthology.orchestra.run.smithi090.stdout: lua-5.4.4-4.el9.x86_64 2024-04-24T09:08:15.836 INFO:teuthology.orchestra.run.smithi090.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-24T09:08:15.836 INFO:teuthology.orchestra.run.smithi090.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-24T09:08:15.836 INFO:teuthology.orchestra.run.smithi090.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T09:08:15.836 INFO:teuthology.orchestra.run.smithi090.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T09:08:15.836 INFO:teuthology.orchestra.run.smithi090.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-24T09:08:15.836 INFO:teuthology.orchestra.run.smithi090.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-24T09:08:15.836 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-13.14-1.el9.x86_64 2024-04-24T09:08:15.836 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-24T09:08:15.836 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-24T09:08:15.836 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-24T09:08:15.836 INFO:teuthology.orchestra.run.smithi090.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-24T09:08:15.836 INFO:teuthology.orchestra.run.smithi090.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-24T09:08:15.836 INFO:teuthology.orchestra.run.smithi090.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T09:08:15.836 INFO:teuthology.orchestra.run.smithi090.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-24T09:08:15.836 INFO:teuthology.orchestra.run.smithi090.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T09:08:15.837 INFO:teuthology.orchestra.run.smithi090.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T09:08:15.837 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T09:08:15.837 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-argparse-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:08:15.837 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:08:15.837 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cephfs-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:08:15.837 INFO:teuthology.orchestra.run.smithi090.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T09:08:15.837 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T09:08:15.837 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T09:08:15.837 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T09:08:15.837 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-24T09:08:15.837 INFO:teuthology.orchestra.run.smithi090.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-24T09:08:15.837 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T09:08:15.837 INFO:teuthology.orchestra.run.smithi090.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T09:08:15.837 INFO:teuthology.orchestra.run.smithi090.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T09:08:15.837 INFO:teuthology.orchestra.run.smithi090.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-24T09:08:15.837 INFO:teuthology.orchestra.run.smithi090.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-24T09:08:15.838 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T09:08:15.838 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T09:08:15.838 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T09:08:15.838 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T09:08:15.838 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T09:08:15.838 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T09:08:15.838 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T09:08:15.838 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T09:08:15.838 INFO:teuthology.orchestra.run.smithi090.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T09:08:15.838 INFO:teuthology.orchestra.run.smithi090.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T09:08:15.838 INFO:teuthology.orchestra.run.smithi090.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T09:08:15.838 INFO:teuthology.orchestra.run.smithi090.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T09:08:15.838 INFO:teuthology.orchestra.run.smithi090.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T09:08:15.839 INFO:teuthology.orchestra.run.smithi090.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T09:08:15.839 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T09:08:15.839 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T09:08:15.839 INFO:teuthology.orchestra.run.smithi090.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T09:08:15.839 INFO:teuthology.orchestra.run.smithi090.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-24T09:08:15.839 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T09:08:15.839 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T09:08:15.839 INFO:teuthology.orchestra.run.smithi090.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T09:08:15.839 INFO:teuthology.orchestra.run.smithi090.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T09:08:15.839 INFO:teuthology.orchestra.run.smithi090.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-24T09:08:15.839 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T09:08:15.839 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T09:08:15.840 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T09:08:15.840 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T09:08:15.840 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-24T09:08:15.840 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T09:08:15.840 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rados-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:08:15.840 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:08:15.840 INFO:teuthology.orchestra.run.smithi090.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T09:08:15.840 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T09:08:15.840 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T09:08:15.840 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:08:15.840 INFO:teuthology.orchestra.run.smithi090.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T09:08:15.840 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T09:08:15.840 INFO:teuthology.orchestra.run.smithi090.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T09:08:15.840 INFO:teuthology.orchestra.run.smithi090.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T09:08:15.841 INFO:teuthology.orchestra.run.smithi090.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T09:08:15.841 INFO:teuthology.orchestra.run.smithi090.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T09:08:15.841 INFO:teuthology.orchestra.run.smithi090.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T09:08:15.841 INFO:teuthology.orchestra.run.smithi090.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-24T09:08:15.841 INFO:teuthology.orchestra.run.smithi090.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T09:08:15.841 INFO:teuthology.orchestra.run.smithi090.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T09:08:15.841 INFO:teuthology.orchestra.run.smithi090.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T09:08:15.841 INFO:teuthology.orchestra.run.smithi090.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T09:08:15.841 INFO:teuthology.orchestra.run.smithi090.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-24T09:08:15.841 INFO:teuthology.orchestra.run.smithi090.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-24T09:08:15.841 INFO:teuthology.orchestra.run.smithi090.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-24T09:08:15.841 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:08:15.841 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:08:15.841 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-nbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T09:08:15.842 INFO:teuthology.orchestra.run.smithi090.stdout: re2-1:20211101-3.el9.x86_64 2024-04-24T09:08:15.842 INFO:teuthology.orchestra.run.smithi090.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-24T09:08:15.842 INFO:teuthology.orchestra.run.smithi090.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T09:08:15.842 INFO:teuthology.orchestra.run.smithi090.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-24T09:08:15.842 INFO:teuthology.orchestra.run.smithi090.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-24T09:08:15.842 INFO:teuthology.orchestra.run.smithi090.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T09:08:15.842 INFO:teuthology.orchestra.run.smithi090.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-24T09:08:15.842 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:08:15.843 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-24T09:08:16.497 DEBUG:teuthology.parallel:result is None 2024-04-24T09:08:16.498 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T09:08:16.715 DEBUG:teuthology.orchestra.run.smithi090:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T09:08:16.771 INFO:teuthology.orchestra.run.smithi090.stdout:19.0.0-3116.g1f81fb39.el9 2024-04-24T09:08:16.772 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3116.g1f81fb39.el9 2024-04-24T09:08:16.772 INFO:teuthology.task.install:The correct ceph version 19.0.0-3116.g1f81fb39 is installed. 2024-04-24T09:08:16.774 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T09:08:16.911 DEBUG:teuthology.orchestra.run.smithi114:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T09:08:16.937 INFO:teuthology.orchestra.run.smithi114.stdout:19.0.0-3116.g1f81fb39.el9 2024-04-24T09:08:16.937 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3116.g1f81fb39.el9 2024-04-24T09:08:16.937 INFO:teuthology.task.install:The correct ceph version 19.0.0-3116.g1f81fb39 is installed. 2024-04-24T09:08:16.939 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T09:08:17.160 DEBUG:teuthology.orchestra.run.smithi149:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T09:08:17.186 INFO:teuthology.orchestra.run.smithi149.stdout:19.0.0-3116.g1f81fb39.el9 2024-04-24T09:08:17.186 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3116.g1f81fb39.el9 2024-04-24T09:08:17.186 INFO:teuthology.task.install:The correct ceph version 19.0.0-3116.g1f81fb39 is installed. 2024-04-24T09:08:17.188 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-24T09:08:17.188 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-24T09:08:17.188 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T09:08:17.270 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-24T09:08:17.270 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T09:08:17.302 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-04-24T09:08:17.302 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T09:08:17.333 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-24T09:08:17.333 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-24T09:08:17.333 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T09:08:17.397 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T09:08:17.502 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-24T09:08:17.503 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T09:08:17.531 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T09:08:17.599 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-04-24T09:08:17.599 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T09:08:17.634 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T09:08:17.699 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-24T09:08:17.699 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-24T09:08:17.699 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T09:08:17.768 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T09:08:17.878 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-24T09:08:17.878 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T09:08:17.902 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T09:08:17.968 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-04-24T09:08:17.968 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T09:08:17.993 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T09:08:18.057 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-24T09:08:18.057 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-24T09:08:18.057 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T09:08:18.123 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T09:08:18.231 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-24T09:08:18.231 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T09:08:18.255 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T09:08:18.319 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-04-24T09:08:18.319 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T09:08:18.343 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T09:08:18.406 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-24T09:08:18.511 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': '1f81fb397ae98da7563d451a78c61574c8f4e6e0', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-04-24T09:08:18.511 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T09:08:18.512 INFO:tasks.cephadm:Cluster fsid is 30ee9be2-021a-11ef-bc93-c7b262605968 2024-04-24T09:08:18.512 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-24T09:08:18.512 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.90', 'mon.b': '172.21.15.114', 'mon.c': '172.21.15.149'} 2024-04-24T09:08:18.512 INFO:tasks.cephadm:First mon is mon.a on smithi090 2024-04-24T09:08:18.513 INFO:tasks.cephadm:First mgr is x 2024-04-24T09:08:18.513 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-24T09:08:18.513 DEBUG:teuthology.orchestra.run.smithi090:> sudo hostname $(hostname -s) 2024-04-24T09:08:18.586 DEBUG:teuthology.orchestra.run.smithi114:> sudo hostname $(hostname -s) 2024-04-24T09:08:18.610 DEBUG:teuthology.orchestra.run.smithi149:> sudo hostname $(hostname -s) 2024-04-24T09:08:18.634 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-24T09:08:18.634 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T09:08:18.853 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '1f81fb397ae98da7563d451a78c61574c8f4e6e0', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78823/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3116-g1f81fb39', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3116.g1f81fb39'}, 'url': 'https://4.chacra.ceph.com/r/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-24 01:00:13.634707', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'rishabh-main-apr17-a654945', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-24T09:08:18.991 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref rishabh-main-apr17-a654945, sha1 1f81fb397ae98da7563d451a78c61574c8f4e6e0 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T09:08:18.993 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/x86_64/flavors/default/cephadm 2024-04-24T09:08:18.993 DEBUG:teuthology.orchestra.run.smithi090:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-24T09:08:19.967 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-r--r--. 1 ubuntu ubuntu 781486 Apr 24 09:08 /home/ubuntu/cephtest/cephadm 2024-04-24T09:08:19.968 DEBUG:teuthology.orchestra.run.smithi114:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-24T09:08:20.857 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-r--r--. 1 ubuntu ubuntu 781486 Apr 24 09:08 /home/ubuntu/cephtest/cephadm 2024-04-24T09:08:20.858 DEBUG:teuthology.orchestra.run.smithi149:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-24T09:08:21.639 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r--. 1 ubuntu ubuntu 781486 Apr 24 09:08 /home/ubuntu/cephtest/cephadm 2024-04-24T09:08:21.639 DEBUG:teuthology.orchestra.run.smithi090:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-24T09:08:21.685 DEBUG:teuthology.orchestra.run.smithi114:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-24T09:08:21.703 DEBUG:teuthology.orchestra.run.smithi149:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-24T09:08:21.738 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 on all hosts... 2024-04-24T09:08:21.738 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 pull 2024-04-24T09:08:21.742 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 pull 2024-04-24T09:08:21.745 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 pull 2024-04-24T09:08:23.297 INFO:teuthology.orchestra.run.smithi114.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0... 2024-04-24T09:08:23.310 INFO:teuthology.orchestra.run.smithi149.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0... 2024-04-24T09:08:25.253 INFO:teuthology.orchestra.run.smithi090.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0... 2024-04-24T09:09:09.197 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T09:09:09.197 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_version": "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)", 2024-04-24T09:09:09.197 INFO:teuthology.orchestra.run.smithi090.stdout: "image_id": "b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02", 2024-04-24T09:09:09.197 INFO:teuthology.orchestra.run.smithi090.stdout: "repo_digests": [ 2024-04-24T09:09:09.197 INFO:teuthology.orchestra.run.smithi090.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644" 2024-04-24T09:09:09.197 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-04-24T09:09:09.197 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T09:09:22.592 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-04-24T09:09:22.592 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_version": "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)", 2024-04-24T09:09:22.592 INFO:teuthology.orchestra.run.smithi114.stdout: "image_id": "b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02", 2024-04-24T09:09:22.592 INFO:teuthology.orchestra.run.smithi114.stdout: "repo_digests": [ 2024-04-24T09:09:22.592 INFO:teuthology.orchestra.run.smithi114.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644" 2024-04-24T09:09:22.592 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-04-24T09:09:22.592 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-04-24T09:09:24.488 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-04-24T09:09:24.488 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_version": "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)", 2024-04-24T09:09:24.488 INFO:teuthology.orchestra.run.smithi149.stdout: "image_id": "b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02", 2024-04-24T09:09:24.488 INFO:teuthology.orchestra.run.smithi149.stdout: "repo_digests": [ 2024-04-24T09:09:24.488 INFO:teuthology.orchestra.run.smithi149.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644" 2024-04-24T09:09:24.489 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-04-24T09:09:24.489 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-04-24T09:09:24.514 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /etc/ceph 2024-04-24T09:09:24.611 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /etc/ceph 2024-04-24T09:09:24.642 DEBUG:teuthology.orchestra.run.smithi149:> sudo mkdir -p /etc/ceph 2024-04-24T09:09:24.670 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 777 /etc/ceph 2024-04-24T09:09:24.758 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 777 /etc/ceph 2024-04-24T09:09:24.785 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod 777 /etc/ceph 2024-04-24T09:09:24.809 INFO:tasks.cephadm:Writing seed config... 2024-04-24T09:09:24.810 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-24T09:09:24.811 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-24T09:09:24.811 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-24T09:09:24.811 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-24T09:09:24.811 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-24T09:09:24.811 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-24T09:09:24.811 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-24T09:09:24.811 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-24T09:09:24.811 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-24T09:09:24.811 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-24T09:09:24.811 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-24T09:09:24.811 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-24T09:09:24.811 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-24T09:09:24.811 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-24T09:09:24.811 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-24T09:09:24.811 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-24T09:09:24.811 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-24T09:09:24.812 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-04-24T09:09:24.812 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-04-24T09:09:24.812 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-24T09:09:24.812 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-24T09:09:24.812 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-24T09:09:24.812 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-24T09:09:24.812 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-24T09:09:24.812 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-24T09:09:24.812 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-24T09:09:24.812 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-24T09:09:24.812 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-24T09:09:24.812 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-24T09:09:24.812 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-24T09:09:24.812 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-24T09:09:24.812 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-24T09:09:24.812 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-04-24T09:09:24.813 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-24T09:09:24.813 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-24T09:09:24.813 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-24T09:09:24.813 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-24T09:09:24.813 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-24T09:09:24.813 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-24T09:09:24.813 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-24T09:09:24.813 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-24T09:09:24.813 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-24T09:09:24.813 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-24T09:09:24.813 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-24T09:09:24.813 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-24T09:09:24.813 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-24T09:09:24.813 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-24T09:09:24.814 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-24T09:09:24.814 DEBUG:teuthology.orchestra.run.smithi090:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-24T09:09:24.869 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 30ee9be2-021a-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-24T09:09:24.869 DEBUG:teuthology.orchestra.run.smithi090:mon.a> sudo journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mon.a.service 2024-04-24T09:09:24.917 DEBUG:teuthology.orchestra.run.smithi090:mgr.x> sudo journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mgr.x.service 2024-04-24T09:09:24.963 INFO:tasks.cephadm:Bootstrapping... 2024-04-24T09:09:24.963 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 -v bootstrap --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.90 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-24T09:09:25.235 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-24T09:09:25.235 INFO:teuthology.orchestra.run.smithi090.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0', '-v', 'bootstrap', '--fsid', '30ee9be2-021a-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.90', '--skip-admin-label'] 2024-04-24T09:09:25.235 INFO:teuthology.orchestra.run.smithi090.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-24T09:09:25.235 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying podman|docker is present... 2024-04-24T09:09:25.289 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-24T09:09:25.290 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying lvm2 is present... 2024-04-24T09:09:25.290 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying time synchronization is in place... 2024-04-24T09:09:25.306 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-24T09:09:25.306 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-24T09:09:25.321 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-24T09:09:25.321 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout inactive 2024-04-24T09:09:25.337 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout enabled 2024-04-24T09:09:25.353 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout active 2024-04-24T09:09:25.353 INFO:teuthology.orchestra.run.smithi090.stdout:Unit chronyd.service is enabled and running 2024-04-24T09:09:25.354 INFO:teuthology.orchestra.run.smithi090.stdout:Repeating the final host check... 2024-04-24T09:09:25.412 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-24T09:09:25.412 INFO:teuthology.orchestra.run.smithi090.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-24T09:09:25.412 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl is present 2024-04-24T09:09:25.412 INFO:teuthology.orchestra.run.smithi090.stdout:lvcreate is present 2024-04-24T09:09:25.426 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-24T09:09:25.426 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-24T09:09:25.439 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-24T09:09:25.440 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout inactive 2024-04-24T09:09:25.454 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout enabled 2024-04-24T09:09:25.467 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout active 2024-04-24T09:09:25.467 INFO:teuthology.orchestra.run.smithi090.stdout:Unit chronyd.service is enabled and running 2024-04-24T09:09:25.467 INFO:teuthology.orchestra.run.smithi090.stdout:Host looks OK 2024-04-24T09:09:25.467 INFO:teuthology.orchestra.run.smithi090.stdout:Cluster fsid: 30ee9be2-021a-11ef-bc93-c7b262605968 2024-04-24T09:09:25.468 INFO:teuthology.orchestra.run.smithi090.stdout:Acquiring lock 139631810145440 on /run/cephadm/30ee9be2-021a-11ef-bc93-c7b262605968.lock 2024-04-24T09:09:25.468 INFO:teuthology.orchestra.run.smithi090.stdout:Lock 139631810145440 acquired on /run/cephadm/30ee9be2-021a-11ef-bc93-c7b262605968.lock 2024-04-24T09:09:25.468 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying IP 172.21.15.90 port 3300 ... 2024-04-24T09:09:25.469 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying IP 172.21.15.90 port 6789 ... 2024-04-24T09:09:25.469 INFO:teuthology.orchestra.run.smithi090.stdout:Base mon IP(s) is [172.21.15.90:3300, 172.21.15.90:6789], mon addrv is [v2:172.21.15.90:3300,v1:172.21.15.90:6789] 2024-04-24T09:09:25.476 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.90 metric 100 2024-04-24T09:09:25.476 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.90 metric 100 2024-04-24T09:09:25.483 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-24T09:09:25.483 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-04-24T09:09:25.483 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1725sec hoplimit 64 pref medium 2024-04-24T09:09:25.489 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-24T09:09:25.489 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-24T09:09:25.489 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-24T09:09:25.489 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-24T09:09:25.489 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d767/64 scope link 2024-04-24T09:09:25.489 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-24T09:09:25.490 INFO:teuthology.orchestra.run.smithi090.stdout:Mon IP `172.21.15.90` is in CIDR network `172.21.0.0/20` 2024-04-24T09:09:25.490 INFO:teuthology.orchestra.run.smithi090.stdout:Mon IP `172.21.15.90` is in CIDR network `172.21.0.0/20` 2024-04-24T09:09:25.490 INFO:teuthology.orchestra.run.smithi090.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-24T09:09:25.491 INFO:teuthology.orchestra.run.smithi090.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-24T09:09:25.492 INFO:teuthology.orchestra.run.smithi090.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0... 2024-04-24T09:09:26.129 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stdout b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 2024-04-24T09:09:26.129 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0... 2024-04-24T09:09:26.129 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stderr Getting image source signatures 2024-04-24T09:09:26.129 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stderr Copying blob sha256:b727accf024e55f70f86e348fc139074bb9b3de1cca3400f8b76b2ed7d35ac83 2024-04-24T09:09:26.129 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stderr Copying blob sha256:3d665f4c032644be9cae748af04bee69f2bfe96cdc3616dabe0ab17899633ab5 2024-04-24T09:09:26.129 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stderr Copying config sha256:b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 2024-04-24T09:09:26.129 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-24T09:09:27.421 INFO:teuthology.orchestra.run.smithi090.stdout:ceph: stdout ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev) 2024-04-24T09:09:27.421 INFO:teuthology.orchestra.run.smithi090.stdout:Ceph version: ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev) 2024-04-24T09:09:27.421 INFO:teuthology.orchestra.run.smithi090.stdout:Extracting ceph user uid/gid from container image... 2024-04-24T09:09:28.498 INFO:teuthology.orchestra.run.smithi090.stdout:stat: stdout 167 167 2024-04-24T09:09:28.499 INFO:teuthology.orchestra.run.smithi090.stdout:Creating initial keys... 2024-04-24T09:09:41.097 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph-authtool: stdout AQBUzChmgbyXHhAActjC6R6g6r4G5sja/F8ohA== 2024-04-24T09:09:42.323 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph-authtool: stdout AQBVzChmzhpbJRAA33XGEvJ4Z/Vc4q5mwdQ2+Q== 2024-04-24T09:09:43.437 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph-authtool: stdout AQBWzChmW0blLxAAzzGzEHXRP+iuuImPu8mv+Q== 2024-04-24T09:09:43.438 INFO:teuthology.orchestra.run.smithi090.stdout:Creating initial monmap... 2024-04-24T09:09:45.339 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-24T09:09:45.339 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-24T09:09:45.339 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 30ee9be2-021a-11ef-bc93-c7b262605968 2024-04-24T09:09:45.340 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-24T09:09:45.340 INFO:teuthology.orchestra.run.smithi090.stdout:monmaptool for a [v2:172.21.15.90:3300,v1:172.21.15.90:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-24T09:09:45.340 INFO:teuthology.orchestra.run.smithi090.stdout:setting min_mon_release = quincy 2024-04-24T09:09:45.340 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/monmaptool: set fsid to 30ee9be2-021a-11ef-bc93-c7b262605968 2024-04-24T09:09:45.340 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-24T09:09:45.340 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:09:45.340 INFO:teuthology.orchestra.run.smithi090.stdout:Creating mon... 2024-04-24T09:09:47.035 INFO:teuthology.orchestra.run.smithi090.stdout:create mon.a on 2024-04-24T09:09:47.430 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-24T09:09:47.671 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-24T09:09:47.963 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-30ee9be2-021a-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-30ee9be2-021a-11ef-bc93-c7b262605968.target. 2024-04-24T09:09:47.964 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-30ee9be2-021a-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-30ee9be2-021a-11ef-bc93-c7b262605968.target. 2024-04-24T09:09:48.355 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mon.a 2024-04-24T09:09:48.355 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Failed to reset failed state of unit ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-24T09:09:48.652 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-30ee9be2-021a-11ef-bc93-c7b262605968.target.wants/ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-30ee9be2-021a-11ef-bc93-c7b262605968@.service. 2024-04-24T09:09:49.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:49 smithi090 podman[27438]: 2024-04-24 09:09:49.09764481 +0000 UTC m=+0.129689815 container create 1aea72b6c62be5084954525e3a4e80615edd7870324a41e25e2321830087ec1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424) 2024-04-24T09:09:49.530 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T09:09:49.530 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout disabled 2024-04-24T09:09:49.544 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T09:09:49.544 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout inactive 2024-04-24T09:09:49.544 INFO:teuthology.orchestra.run.smithi090.stdout:firewalld.service is not enabled 2024-04-24T09:09:49.544 INFO:teuthology.orchestra.run.smithi090.stdout:Not possible to enable service . firewalld.service is not available 2024-04-24T09:09:49.545 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for mon to start... 2024-04-24T09:09:49.545 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for mon... 2024-04-24T09:09:49.692 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:49 smithi090 podman[27438]: 2024-04-24 09:09:49.410438775 +0000 UTC m=+0.442483803 container init 1aea72b6c62be5084954525e3a4e80615edd7870324a41e25e2321830087ec1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mon-a, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-24T09:09:49.692 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:49 smithi090 podman[27438]: 2024-04-24 09:09:49.420590299 +0000 UTC m=+0.452635327 container start 1aea72b6c62be5084954525e3a4e80615edd7870324a41e25e2321830087ec1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True, GIT_CLEAN=True) 2024-04-24T09:09:49.693 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:49 smithi090 ceph-mon[27456]: mkfs 30ee9be2-021a-11ef-bc93-c7b262605968 2024-04-24T09:09:49.693 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:49 smithi090 bash[27438]: 1aea72b6c62be5084954525e3a4e80615edd7870324a41e25e2321830087ec1d 2024-04-24T09:09:49.693 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:49 smithi090 ceph-mon[27456]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-24T09:09:49.693 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:49 smithi090 systemd[1]: Started Ceph mon.a for 30ee9be2-021a-11ef-bc93-c7b262605968. 2024-04-24T09:09:50.851 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:50 smithi090 ceph-mon[27456]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-24T09:09:50.851 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:50 smithi090 ceph-mon[27456]: monmap epoch 1 2024-04-24T09:09:50.851 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:50 smithi090 ceph-mon[27456]: fsid 30ee9be2-021a-11ef-bc93-c7b262605968 2024-04-24T09:09:50.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:50 smithi090 ceph-mon[27456]: last_changed 2024-04-24T09:09:44.652569+0000 2024-04-24T09:09:50.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:50 smithi090 ceph-mon[27456]: created 2024-04-24T09:09:44.652569+0000 2024-04-24T09:09:50.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:50 smithi090 ceph-mon[27456]: min_mon_release 19 (squid) 2024-04-24T09:09:50.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:50 smithi090 ceph-mon[27456]: election_strategy: 1 2024-04-24T09:09:50.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:50 smithi090 ceph-mon[27456]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-04-24T09:09:50.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:50 smithi090 ceph-mon[27456]: fsmap 2024-04-24T09:09:50.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:50 smithi090 ceph-mon[27456]: osdmap e1: 0 total, 0 up, 0 in 2024-04-24T09:09:50.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:50 smithi090 ceph-mon[27456]: mgrmap e1: no daemons active 2024-04-24T09:09:50.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:50 smithi090 ceph-mon[27456]: from='client.? 172.21.15.90:0/3148701574' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-24T09:09:50.861 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout cluster: 2024-04-24T09:09:50.867 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout id: 30ee9be2-021a-11ef-bc93-c7b262605968 2024-04-24T09:09:50.867 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-24T09:09:50.867 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-24T09:09:50.867 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout services: 2024-04-24T09:09:50.867 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.750502s) 2024-04-24T09:09:50.867 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-24T09:09:50.867 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-24T09:09:50.867 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-24T09:09:50.867 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout data: 2024-04-24T09:09:50.867 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-24T09:09:50.867 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-24T09:09:50.868 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-24T09:09:50.868 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout pgs: 2024-04-24T09:09:50.868 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-24T09:09:50.868 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:50.152+0000 7f4e55e00700 1 Processor -- start 2024-04-24T09:09:50.868 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:50.153+0000 7f4e55e00700 1 -- start start 2024-04-24T09:09:50.868 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:50.153+0000 7f4e55e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e5010f0a0 0x7f4e5010f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.868 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:50.153+0000 7f4e55e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5010fa90 con 0x7f4e5010f0a0 2024-04-24T09:09:50.868 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:50.154+0000 7f4e4ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e5010f0a0 0x7f4e5010f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.868 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:50.154+0000 7f4e4ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e5010f0a0 0x7f4e5010f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54632/0 (socket says 172.21.15.90:54632) 2024-04-24T09:09:50.868 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:50.154+0000 7f4e4ea00700 1 -- 172.21.15.90:0/3952397338 learned_addr learned my addr 172.21.15.90:0/3952397338 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:09:50.868 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:50.155+0000 7f4e4ea00700 1 -- 172.21.15.90:0/3952397338 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e5010fd70 con 0x7f4e5010f0a0 2024-04-24T09:09:50.868 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:50.155+0000 7f4e4ea00700 1 --2- 172.21.15.90:0/3952397338 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e5010f0a0 0x7f4e5010f4c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4e400098a0 tx=0x7f4e40009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f8024bb0dd7a733 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:50.868 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:50.155+0000 7f4e4d600700 1 -- 172.21.15.90:0/3952397338 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e40004160 con 0x7f4e5010f0a0 2024-04-24T09:09:50.868 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:50.155+0000 7f4e4d600700 1 -- 172.21.15.90:0/3952397338 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e40004300 con 0x7f4e5010f0a0 2024-04-24T09:09:50.869 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:50.155+0000 7f4e4d600700 1 -- 172.21.15.90:0/3952397338 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e400036a0 con 0x7f4e5010f0a0 2024-04-24T09:09:50.869 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:50.156+0000 7f4e55e00700 1 -- 172.21.15.90:0/3952397338 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e5010f0a0 msgr2=0x7f4e5010f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:50.869 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:50.156+0000 7f4e55e00700 1 --2- 172.21.15.90:0/3952397338 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e5010f0a0 0x7f4e5010f4c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4e400098a0 tx=0x7f4e40009bb0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.869 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:50.156+0000 7f4e55e00700 1 -- 172.21.15.90:0/3952397338 shutdown_connections 2024-04-24T09:09:50.869 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:50.156+0000 7f4e55e00700 1 --2- 172.21.15.90:0/3952397338 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e5010f0a0 0x7f4e5010f4c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.869 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:50.156+0000 7f4e55e00700 1 -- 172.21.15.90:0/3952397338 >> 172.21.15.90:0/3952397338 conn(0x7f4e5007ddc0 msgr2=0x7f4e500801e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:50.869 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:50.156+0000 7f4e55e00700 1 -- 172.21.15.90:0/3952397338 shutdown_connections 2024-04-24T09:09:50.869 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:50.156+0000 7f4e55e00700 1 -- 172.21.15.90:0/3952397338 wait complete. 2024-04-24T09:09:50.869 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:50.157+0000 7f4e55e00700 1 Processor -- start 2024-04-24T09:09:50.869 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:50.157+0000 7f4e55e00700 1 -- start start 2024-04-24T09:09:50.869 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:50.158+0000 7f4e55e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e5010f0a0 0x7f4e501a2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.869 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:50.158+0000 7f4e55e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4e501102a0 con 0x7f4e5010f0a0 2024-04-24T09:09:50.869 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:50.158+0000 7f4e4ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e5010f0a0 0x7f4e501a2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.869 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:50.158+0000 7f4e4ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e5010f0a0 0x7f4e501a2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54634/0 (socket says 172.21.15.90:54634) 2024-04-24T09:09:50.869 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:50.158+0000 7f4e4ea00700 1 -- 172.21.15.90:0/3148701574 learned_addr learned my addr 172.21.15.90:0/3148701574 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:09:50.869 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:50.158+0000 7f4e4ea00700 1 -- 172.21.15.90:0/3148701574 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e501a27d0 con 0x7f4e5010f0a0 2024-04-24T09:09:50.870 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:50.159+0000 7f4e4ea00700 1 --2- 172.21.15.90:0/3148701574 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e5010f0a0 0x7f4e501a2290 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4e40000c00 tx=0x7f4e40004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:50.870 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:50.159+0000 7f4e54a00700 1 -- 172.21.15.90:0/3148701574 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e40003db0 con 0x7f4e5010f0a0 2024-04-24T09:09:50.870 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:50.159+0000 7f4e54a00700 1 -- 172.21.15.90:0/3148701574 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e40011630 con 0x7f4e5010f0a0 2024-04-24T09:09:50.870 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:50.159+0000 7f4e54a00700 1 -- 172.21.15.90:0/3148701574 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e40011950 con 0x7f4e5010f0a0 2024-04-24T09:09:50.870 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:50.159+0000 7f4e55e00700 1 -- 172.21.15.90:0/3148701574 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e501a2a60 con 0x7f4e5010f0a0 2024-04-24T09:09:50.870 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:50.159+0000 7f4e55e00700 1 -- 172.21.15.90:0/3148701574 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e501a5750 con 0x7f4e5010f0a0 2024-04-24T09:09:50.870 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:50.159+0000 7f4e54a00700 1 -- 172.21.15.90:0/3148701574 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4e40011af0 con 0x7f4e5010f0a0 2024-04-24T09:09:50.870 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:50.159+0000 7f4e54a00700 1 -- 172.21.15.90:0/3148701574 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4e4001c450 con 0x7f4e5010f0a0 2024-04-24T09:09:50.870 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:50.160+0000 7f4e55e00700 1 -- 172.21.15.90:0/3148701574 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e50064150 con 0x7f4e5010f0a0 2024-04-24T09:09:50.870 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:50.164+0000 7f4e54a00700 1 -- 172.21.15.90:0/3148701574 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f4e40032020 con 0x7f4e5010f0a0 2024-04-24T09:09:50.870 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:50.243+0000 7f4e55e00700 1 -- 172.21.15.90:0/3148701574 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f4e5002e160 con 0x7f4e5010f0a0 2024-04-24T09:09:50.870 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:50.244+0000 7f4e54a00700 1 -- 172.21.15.90:0/3148701574 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f4e40004260 con 0x7f4e5010f0a0 2024-04-24T09:09:50.870 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:50.246+0000 7f4e55e00700 1 -- 172.21.15.90:0/3148701574 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e5010f0a0 msgr2=0x7f4e501a2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:50.870 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:50.246+0000 7f4e55e00700 1 --2- 172.21.15.90:0/3148701574 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e5010f0a0 0x7f4e501a2290 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4e40000c00 tx=0x7f4e40004750 comp rx=0 tx=0).stop 2024-04-24T09:09:50.870 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:50.246+0000 7f4e55e00700 1 -- 172.21.15.90:0/3148701574 shutdown_connections 2024-04-24T09:09:50.871 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:50.246+0000 7f4e55e00700 1 --2- 172.21.15.90:0/3148701574 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e5010f0a0 0x7f4e501a2290 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.871 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:50.246+0000 7f4e55e00700 1 -- 172.21.15.90:0/3148701574 >> 172.21.15.90:0/3148701574 conn(0x7f4e5007ddc0 msgr2=0x7f4e50198f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:50.871 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:50.246+0000 7f4e55e00700 1 -- 172.21.15.90:0/3148701574 shutdown_connections 2024-04-24T09:09:50.871 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:50.246+0000 7f4e55e00700 1 -- 172.21.15.90:0/3148701574 wait complete. 2024-04-24T09:09:50.871 INFO:teuthology.orchestra.run.smithi090.stdout:mon is available 2024-04-24T09:09:50.871 INFO:teuthology.orchestra.run.smithi090.stdout:Assimilating anything we can from ceph.conf... 2024-04-24T09:09:51.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:51 smithi090 ceph-mon[27456]: from='client.? 172.21.15.90:0/3230077476' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-24T09:09:51.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:51 smithi090 ceph-mon[27456]: from='client.? 172.21.15.90:0/3230077476' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-24T09:09:52.307 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-24T09:09:52.307 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout [global] 2024-04-24T09:09:52.307 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout fsid = 30ee9be2-021a-11ef-bc93-c7b262605968 2024-04-24T09:09:52.307 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.90:3300,v1:172.21.15.90:6789] 2024-04-24T09:09:52.307 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-24T09:09:52.313 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-24T09:09:52.313 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-24T09:09:52.313 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-24T09:09:52.313 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-24T09:09:52.313 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-24T09:09:52.314 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-24T09:09:52.314 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-24T09:09:52.314 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout [osd] 2024-04-24T09:09:52.314 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-24T09:09:52.314 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-24T09:09:52.314 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-24T09:09:52.314 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-24T09:09:52.314 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-24T09:09:52.314 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:51.512+0000 7f1769000700 1 Processor -- start 2024-04-24T09:09:52.314 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:51.512+0000 7f1769000700 1 -- start start 2024-04-24T09:09:52.314 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:51.513+0000 7f1769000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f176410a600 0x7f176410aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:52.314 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:51.513+0000 7f1769000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f176410aff0 con 0x7f176410a600 2024-04-24T09:09:52.314 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:51.513+0000 7f1763400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f176410a600 0x7f176410aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:52.315 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:51.513+0000 7f1763400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f176410a600 0x7f176410aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54648/0 (socket says 172.21.15.90:54648) 2024-04-24T09:09:52.315 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:51.513+0000 7f1763400700 1 -- 172.21.15.90:0/1441015174 learned_addr learned my addr 172.21.15.90:0/1441015174 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:09:52.315 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:51.514+0000 7f1763400700 1 -- 172.21.15.90:0/1441015174 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f176410b7b0 con 0x7f176410a600 2024-04-24T09:09:52.315 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:51.514+0000 7f1763400700 1 --2- 172.21.15.90:0/1441015174 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f176410a600 0x7f176410aa20 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f17540098a0 tx=0x7f1754009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e3b5af99cb8a81e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:52.315 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:51.514+0000 7f1762000700 1 -- 172.21.15.90:0/1441015174 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1754004160 con 0x7f176410a600 2024-04-24T09:09:52.315 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:51.514+0000 7f1762000700 1 -- 172.21.15.90:0/1441015174 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1754004300 con 0x7f176410a600 2024-04-24T09:09:52.315 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:51.514+0000 7f1762000700 1 -- 172.21.15.90:0/1441015174 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17540036a0 con 0x7f176410a600 2024-04-24T09:09:52.315 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:51.515+0000 7f1769000700 1 -- 172.21.15.90:0/1441015174 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f176410a600 msgr2=0x7f176410aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:52.315 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:51.515+0000 7f1769000700 1 --2- 172.21.15.90:0/1441015174 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f176410a600 0x7f176410aa20 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f17540098a0 tx=0x7f1754009bb0 comp rx=0 tx=0).stop 2024-04-24T09:09:52.315 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:51.515+0000 7f1769000700 1 -- 172.21.15.90:0/1441015174 shutdown_connections 2024-04-24T09:09:52.315 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:51.515+0000 7f1769000700 1 --2- 172.21.15.90:0/1441015174 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f176410a600 0x7f176410aa20 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:52.315 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:51.515+0000 7f1769000700 1 -- 172.21.15.90:0/1441015174 >> 172.21.15.90:0/1441015174 conn(0x7f176407dbb0 msgr2=0x7f17641084c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:52.315 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:51.515+0000 7f1769000700 1 -- 172.21.15.90:0/1441015174 shutdown_connections 2024-04-24T09:09:52.316 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:51.515+0000 7f1769000700 1 -- 172.21.15.90:0/1441015174 wait complete. 2024-04-24T09:09:52.316 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:51.516+0000 7f1769000700 1 Processor -- start 2024-04-24T09:09:52.316 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:51.517+0000 7f1769000700 1 -- start start 2024-04-24T09:09:52.316 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:51.517+0000 7f1769000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f176410a600 0x7f176419dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:52.316 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:51.517+0000 7f1769000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f176410bce0 con 0x7f176410a600 2024-04-24T09:09:52.316 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:51.517+0000 7f1763400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f176410a600 0x7f176419dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:52.316 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:51.517+0000 7f1763400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f176410a600 0x7f176419dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54660/0 (socket says 172.21.15.90:54660) 2024-04-24T09:09:52.316 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:51.517+0000 7f1763400700 1 -- 172.21.15.90:0/3230077476 learned_addr learned my addr 172.21.15.90:0/3230077476 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:09:52.316 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:51.517+0000 7f1763400700 1 -- 172.21.15.90:0/3230077476 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f176419e1f0 con 0x7f176410a600 2024-04-24T09:09:52.316 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:51.518+0000 7f1763400700 1 --2- 172.21.15.90:0/3230077476 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f176410a600 0x7f176419dcb0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f1754000c00 tx=0x7f1754004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:52.316 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:51.518+0000 7f175be00700 1 -- 172.21.15.90:0/3230077476 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1754003db0 con 0x7f176410a600 2024-04-24T09:09:52.316 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:51.518+0000 7f175be00700 1 -- 172.21.15.90:0/3230077476 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1754011630 con 0x7f176410a600 2024-04-24T09:09:52.316 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:51.518+0000 7f175be00700 1 -- 172.21.15.90:0/3230077476 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1754011950 con 0x7f176410a600 2024-04-24T09:09:52.317 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:51.518+0000 7f1769000700 1 -- 172.21.15.90:0/3230077476 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f176419e480 con 0x7f176410a600 2024-04-24T09:09:52.317 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:51.518+0000 7f1769000700 1 -- 172.21.15.90:0/3230077476 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17641a1170 con 0x7f176410a600 2024-04-24T09:09:52.317 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:51.519+0000 7f175be00700 1 -- 172.21.15.90:0/3230077476 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1754011af0 con 0x7f176410a600 2024-04-24T09:09:52.317 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:51.519+0000 7f175be00700 1 -- 172.21.15.90:0/3230077476 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f175401c450 con 0x7f176410a600 2024-04-24T09:09:52.317 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:51.519+0000 7f1769000700 1 -- 172.21.15.90:0/3230077476 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1764064150 con 0x7f176410a600 2024-04-24T09:09:52.317 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:51.522+0000 7f175be00700 1 -- 172.21.15.90:0/3230077476 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f1754032020 con 0x7f176410a600 2024-04-24T09:09:52.317 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:51.599+0000 7f1769000700 1 -- 172.21.15.90:0/3230077476 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f17641a1340 con 0x7f176410a600 2024-04-24T09:09:52.317 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:51.607+0000 7f175be00700 1 -- 172.21.15.90:0/3230077476 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f175401b450 con 0x7f176410a600 2024-04-24T09:09:52.317 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:51.608+0000 7f1769000700 1 -- 172.21.15.90:0/3230077476 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f176410a600 msgr2=0x7f176419dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:52.317 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:51.608+0000 7f1769000700 1 --2- 172.21.15.90:0/3230077476 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f176410a600 0x7f176419dcb0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f1754000c00 tx=0x7f1754004750 comp rx=0 tx=0).stop 2024-04-24T09:09:52.317 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:51.608+0000 7f1769000700 1 -- 172.21.15.90:0/3230077476 shutdown_connections 2024-04-24T09:09:52.317 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:51.608+0000 7f1769000700 1 --2- 172.21.15.90:0/3230077476 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f176410a600 0x7f176419dcb0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:52.317 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:51.608+0000 7f1769000700 1 -- 172.21.15.90:0/3230077476 >> 172.21.15.90:0/3230077476 conn(0x7f176407dbb0 msgr2=0x7f1764191fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:52.317 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:51.608+0000 7f1769000700 1 -- 172.21.15.90:0/3230077476 shutdown_connections 2024-04-24T09:09:52.317 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:51.609+0000 7f1769000700 1 -- 172.21.15.90:0/3230077476 wait complete. 2024-04-24T09:09:52.317 INFO:teuthology.orchestra.run.smithi090.stdout:Generating new minimal ceph.conf... 2024-04-24T09:09:53.274 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:53 smithi090 ceph-mon[27456]: from='client.? 172.21.15.90:0/2370752338' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:09:53.566 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:52.862+0000 7fa26fe00700 1 Processor -- start 2024-04-24T09:09:53.566 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:52.862+0000 7fa26fe00700 1 -- start start 2024-04-24T09:09:53.566 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:52.863+0000 7fa26fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa27010a600 0x7fa27010aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:53.567 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:52.863+0000 7fa26fe00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa27010aff0 con 0x7fa27010a600 2024-04-24T09:09:53.567 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:52.864+0000 7fa26ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa27010a600 0x7fa27010aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:53.568 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:52.864+0000 7fa26ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa27010a600 0x7fa27010aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54664/0 (socket says 172.21.15.90:54664) 2024-04-24T09:09:53.568 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:52.864+0000 7fa26ea00700 1 -- 172.21.15.90:0/1008844360 learned_addr learned my addr 172.21.15.90:0/1008844360 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:09:53.568 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:52.864+0000 7fa26ea00700 1 -- 172.21.15.90:0/1008844360 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa27010b7b0 con 0x7fa27010a600 2024-04-24T09:09:53.568 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:52.865+0000 7fa26ea00700 1 --2- 172.21.15.90:0/1008844360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa27010a600 0x7fa27010aa20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa2600098a0 tx=0x7fa260009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4ccb882659a91cf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:53.568 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:52.865+0000 7fa26d600700 1 -- 172.21.15.90:0/1008844360 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa260004160 con 0x7fa27010a600 2024-04-24T09:09:53.568 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:52.865+0000 7fa26d600700 1 -- 172.21.15.90:0/1008844360 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa260004300 con 0x7fa27010a600 2024-04-24T09:09:53.568 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:52.865+0000 7fa26d600700 1 -- 172.21.15.90:0/1008844360 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa260011b50 con 0x7fa27010a600 2024-04-24T09:09:53.568 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:52.866+0000 7fa26fe00700 1 -- 172.21.15.90:0/1008844360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa27010a600 msgr2=0x7fa27010aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:53.568 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:52.866+0000 7fa26fe00700 1 --2- 172.21.15.90:0/1008844360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa27010a600 0x7fa27010aa20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa2600098a0 tx=0x7fa260009bb0 comp rx=0 tx=0).stop 2024-04-24T09:09:53.571 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:52.866+0000 7fa26fe00700 1 -- 172.21.15.90:0/1008844360 shutdown_connections 2024-04-24T09:09:53.571 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:52.866+0000 7fa26fe00700 1 --2- 172.21.15.90:0/1008844360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa27010a600 0x7fa27010aa20 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:53.571 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:52.866+0000 7fa26fe00700 1 -- 172.21.15.90:0/1008844360 >> 172.21.15.90:0/1008844360 conn(0x7fa27007dbb0 msgr2=0x7fa2701084c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:53.571 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:52.866+0000 7fa26fe00700 1 -- 172.21.15.90:0/1008844360 shutdown_connections 2024-04-24T09:09:53.572 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:52.866+0000 7fa26fe00700 1 -- 172.21.15.90:0/1008844360 wait complete. 2024-04-24T09:09:53.572 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:52.867+0000 7fa26fe00700 1 Processor -- start 2024-04-24T09:09:53.572 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:52.867+0000 7fa26fe00700 1 -- start start 2024-04-24T09:09:53.572 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:52.868+0000 7fa26fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa27010a600 0x7fa27019e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:53.572 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:52.868+0000 7fa26fe00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa27010eb60 con 0x7fa27010a600 2024-04-24T09:09:53.572 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:52.868+0000 7fa26ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa27010a600 0x7fa27019e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:53.572 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:52.868+0000 7fa26ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa27010a600 0x7fa27019e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54666/0 (socket says 172.21.15.90:54666) 2024-04-24T09:09:53.572 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:52.868+0000 7fa26ea00700 1 -- 172.21.15.90:0/2370752338 learned_addr learned my addr 172.21.15.90:0/2370752338 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:09:53.572 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:52.868+0000 7fa26ea00700 1 -- 172.21.15.90:0/2370752338 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa27019eb10 con 0x7fa27010a600 2024-04-24T09:09:53.572 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:52.869+0000 7fa26ea00700 1 --2- 172.21.15.90:0/2370752338 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa27010a600 0x7fa27019e5d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa260009fb0 tx=0x7fa260011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:53.572 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:52.869+0000 7fa267400700 1 -- 172.21.15.90:0/2370752338 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2600038b0 con 0x7fa27010a600 2024-04-24T09:09:53.572 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:52.869+0000 7fa267400700 1 -- 172.21.15.90:0/2370752338 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa260011e50 con 0x7fa27010a600 2024-04-24T09:09:53.572 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:52.869+0000 7fa26fe00700 1 -- 172.21.15.90:0/2370752338 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa27019eda0 con 0x7fa27010a600 2024-04-24T09:09:53.573 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:52.869+0000 7fa267400700 1 -- 172.21.15.90:0/2370752338 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa260019550 con 0x7fa27010a600 2024-04-24T09:09:53.573 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:52.869+0000 7fa26fe00700 1 -- 172.21.15.90:0/2370752338 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2701a1a90 con 0x7fa27010a600 2024-04-24T09:09:53.573 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:52.869+0000 7fa267400700 1 -- 172.21.15.90:0/2370752338 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa260019be0 con 0x7fa27010a600 2024-04-24T09:09:53.573 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:52.869+0000 7fa267400700 1 -- 172.21.15.90:0/2370752338 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa260024730 con 0x7fa27010a600 2024-04-24T09:09:53.573 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:52.870+0000 7fa26fe00700 1 -- 172.21.15.90:0/2370752338 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa270064150 con 0x7fa27010a600 2024-04-24T09:09:53.573 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:52.873+0000 7fa267400700 1 -- 172.21.15.90:0/2370752338 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fa260003a50 con 0x7fa27010a600 2024-04-24T09:09:53.573 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:52.949+0000 7fa26fe00700 1 -- 172.21.15.90:0/2370752338 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fa2701a1c60 con 0x7fa27010a600 2024-04-24T09:09:53.573 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:52.950+0000 7fa267400700 1 -- 172.21.15.90:0/2370752338 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fa2600249d0 con 0x7fa27010a600 2024-04-24T09:09:53.573 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:52.951+0000 7fa26fe00700 1 -- 172.21.15.90:0/2370752338 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa27010a600 msgr2=0x7fa27019e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:53.573 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:52.951+0000 7fa26fe00700 1 --2- 172.21.15.90:0/2370752338 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa27010a600 0x7fa27019e5d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa260009fb0 tx=0x7fa260011bf0 comp rx=0 tx=0).stop 2024-04-24T09:09:53.573 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:52.951+0000 7fa26fe00700 1 -- 172.21.15.90:0/2370752338 shutdown_connections 2024-04-24T09:09:53.573 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:52.951+0000 7fa26fe00700 1 --2- 172.21.15.90:0/2370752338 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa27010a600 0x7fa27019e5d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:53.573 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:52.951+0000 7fa26fe00700 1 -- 172.21.15.90:0/2370752338 >> 172.21.15.90:0/2370752338 conn(0x7fa27007dbb0 msgr2=0x7fa270107cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:53.574 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:52.952+0000 7fa26fe00700 1 -- 172.21.15.90:0/2370752338 shutdown_connections 2024-04-24T09:09:53.574 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:52.952+0000 7fa26fe00700 1 -- 172.21.15.90:0/2370752338 wait complete. 2024-04-24T09:09:53.574 INFO:teuthology.orchestra.run.smithi090.stdout:Restarting the monitor... 2024-04-24T09:09:53.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:53 smithi090 systemd[1]: Stopping Ceph mon.a for 30ee9be2-021a-11ef-bc93-c7b262605968... 2024-04-24T09:09:54.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:54 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mon-a[27452]: 2024-04-24T09:09:54.049+0000 7f942ba00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:09:54.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:54 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mon-a[27452]: 2024-04-24T09:09:54.049+0000 7f942ba00700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-24T09:09:54.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:54 smithi090 podman[27699]: 2024-04-24 09:09:54.163955945 +0000 UTC m=+0.231319819 container died 1aea72b6c62be5084954525e3a4e80615edd7870324a41e25e2321830087ec1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-24T09:09:54.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:54 smithi090 podman[27699]: 2024-04-24 09:09:54.580541593 +0000 UTC m=+0.647905457 container cleanup 1aea72b6c62be5084954525e3a4e80615edd7870324a41e25e2321830087ec1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mon-a, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424) 2024-04-24T09:09:54.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:54 smithi090 bash[27699]: ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mon-a 2024-04-24T09:09:55.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:54 smithi090 podman[27713]: 2024-04-24 09:09:54.874399412 +0000 UTC m=+0.690701534 container remove 1aea72b6c62be5084954525e3a4e80615edd7870324a41e25e2321830087ec1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-24T09:09:55.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:55 smithi090 systemd[1]: ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-24T09:09:55.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:55 smithi090 systemd[1]: Stopped Ceph mon.a for 30ee9be2-021a-11ef-bc93-c7b262605968. 2024-04-24T09:09:55.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:55 smithi090 systemd[1]: ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mon.a.service: Consumed 1.942s CPU time. 2024-04-24T09:09:55.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:55 smithi090 systemd[1]: Starting Ceph mon.a for 30ee9be2-021a-11ef-bc93-c7b262605968... 2024-04-24T09:09:55.896 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:55 smithi090 podman[27806]: 2024-04-24 09:09:55.750566433 +0000 UTC m=+0.119633689 container create b8ed5a8fba8a52eb8bc0a64687ab90e04a2d6b7eac98cd7bf5b1f54038c4736b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T09:09:55.896 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:55 smithi090 podman[27806]: 2024-04-24 09:09:55.668156629 +0000 UTC m=+0.037223885 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T09:09:56.122 INFO:teuthology.orchestra.run.smithi090.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-24T09:09:56.274 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:55 smithi090 podman[27806]: 2024-04-24 09:09:55.979287706 +0000 UTC m=+0.348354999 container init b8ed5a8fba8a52eb8bc0a64687ab90e04a2d6b7eac98cd7bf5b1f54038c4736b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T09:09:56.274 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:55 smithi090 podman[27806]: 2024-04-24 09:09:55.989183957 +0000 UTC m=+0.358251250 container start b8ed5a8fba8a52eb8bc0a64687ab90e04a2d6b7eac98cd7bf5b1f54038c4736b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-24T09:09:56.274 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: set uid:gid to 167:167 (ceph:ceph) 2024-04-24T09:09:56.275 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev), process ceph-mon, pid 2 2024-04-24T09:09:56.275 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: pidfile_write: ignore empty --pid-file 2024-04-24T09:09:56.275 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: load: jerasure load: lrc 2024-04-24T09:09:56.275 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: RocksDB version: 7.9.2 2024-04-24T09:09:56.275 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Git sha 0 2024-04-24T09:09:56.275 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Compile date 2024-04-23 21:06:33 2024-04-24T09:09:56.275 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: DB SUMMARY 2024-04-24T09:09:56.275 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: DB Session ID: 1RFW3RSEXZ01N4QUENU0 2024-04-24T09:09:56.275 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: CURRENT file: CURRENT 2024-04-24T09:09:56.275 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: IDENTITY file: IDENTITY 2024-04-24T09:09:56.275 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-24T09:09:56.275 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-24T09:09:56.275 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105792 ; 2024-04-24T09:09:56.275 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.error_if_exists: 0 2024-04-24T09:09:56.275 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.create_if_missing: 0 2024-04-24T09:09:56.275 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.paranoid_checks: 1 2024-04-24T09:09:56.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-24T09:09:56.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-24T09:09:56.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-24T09:09:56.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.env: 0x5634e5e9ccc0 2024-04-24T09:09:56.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.fs: PosixFileSystem 2024-04-24T09:09:56.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.info_log: 0x563512279c60 2024-04-24T09:09:56.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.max_file_opening_threads: 16 2024-04-24T09:09:56.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.statistics: (nil) 2024-04-24T09:09:56.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.use_fsync: 0 2024-04-24T09:09:56.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.max_log_file_size: 0 2024-04-24T09:09:56.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-24T09:09:56.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-24T09:09:56.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.keep_log_file_num: 1000 2024-04-24T09:09:56.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.recycle_log_file_num: 0 2024-04-24T09:09:56.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.allow_fallocate: 1 2024-04-24T09:09:56.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.allow_mmap_reads: 0 2024-04-24T09:09:56.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.allow_mmap_writes: 0 2024-04-24T09:09:56.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.use_direct_reads: 0 2024-04-24T09:09:56.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-24T09:09:56.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.create_missing_column_families: 0 2024-04-24T09:09:56.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.db_log_dir: 2024-04-24T09:09:56.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.wal_dir: 2024-04-24T09:09:56.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-24T09:09:56.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-24T09:09:56.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-24T09:09:56.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-24T09:09:56.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-24T09:09:56.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-24T09:09:56.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.advise_random_on_open: 1 2024-04-24T09:09:56.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.db_write_buffer_size: 0 2024-04-24T09:09:56.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.write_buffer_manager: 0x5635122481e0 2024-04-24T09:09:56.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-24T09:09:56.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-24T09:09:56.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-24T09:09:56.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.rate_limiter: (nil) 2024-04-24T09:09:56.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-24T09:09:56.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.wal_recovery_mode: 2 2024-04-24T09:09:56.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.enable_thread_tracking: 0 2024-04-24T09:09:56.279 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.enable_pipelined_write: 0 2024-04-24T09:09:56.279 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.unordered_write: 0 2024-04-24T09:09:56.279 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-24T09:09:56.279 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-24T09:09:56.279 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-24T09:09:56.279 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-24T09:09:56.279 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.row_cache: None 2024-04-24T09:09:56.279 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.wal_filter: None 2024-04-24T09:09:56.280 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-24T09:09:56.280 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.allow_ingest_behind: 0 2024-04-24T09:09:56.280 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.two_write_queues: 0 2024-04-24T09:09:56.280 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.manual_wal_flush: 0 2024-04-24T09:09:56.280 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.wal_compression: 0 2024-04-24T09:09:56.280 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.atomic_flush: 0 2024-04-24T09:09:56.280 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-24T09:09:56.280 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-24T09:09:56.280 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-24T09:09:56.280 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.log_readahead_size: 0 2024-04-24T09:09:56.280 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-24T09:09:56.280 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.best_efforts_recovery: 0 2024-04-24T09:09:56.281 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-24T09:09:56.281 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-24T09:09:56.281 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.allow_data_in_errors: 0 2024-04-24T09:09:56.281 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.db_host_id: __hostname__ 2024-04-24T09:09:56.281 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-24T09:09:56.281 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.max_background_jobs: 2 2024-04-24T09:09:56.281 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.max_background_compactions: -1 2024-04-24T09:09:56.281 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.max_subcompactions: 1 2024-04-24T09:09:56.281 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-24T09:09:56.281 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-24T09:09:56.281 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-24T09:09:56.281 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.max_total_wal_size: 0 2024-04-24T09:09:56.281 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-24T09:09:56.281 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-24T09:09:56.282 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-24T09:09:56.282 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-24T09:09:56.282 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.max_open_files: -1 2024-04-24T09:09:56.282 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.bytes_per_sync: 0 2024-04-24T09:09:56.282 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-24T09:09:56.282 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-24T09:09:56.282 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.compaction_readahead_size: 0 2024-04-24T09:09:56.282 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.max_background_flushes: -1 2024-04-24T09:09:56.282 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Compression algorithms supported: 2024-04-24T09:09:56.282 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-24T09:09:56.282 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: kZSTD supported: 0 2024-04-24T09:09:56.282 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: kXpressCompression supported: 0 2024-04-24T09:09:56.282 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: kLZ4HCCompression supported: 1 2024-04-24T09:09:56.282 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: kZlibCompression supported: 1 2024-04-24T09:09:56.283 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: kSnappyCompression supported: 1 2024-04-24T09:09:56.283 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: kLZ4Compression supported: 1 2024-04-24T09:09:56.283 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: kBZip2Compression supported: 0 2024-04-24T09:09:56.283 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-24T09:09:56.283 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-24T09:09:56.283 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-24T09:09:56.283 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-24T09:09:56.283 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-24T09:09:56.283 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.merge_operator: 2024-04-24T09:09:56.283 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.compaction_filter: None 2024-04-24T09:09:56.283 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.compaction_filter_factory: None 2024-04-24T09:09:56.283 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.sst_partitioner_factory: None 2024-04-24T09:09:56.283 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-24T09:09:56.284 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-24T09:09:56.284 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x563512279e20) 2024-04-24T09:09:56.284 INFO:journalctl@ceph.mon.a.smithi090.stdout: cache_index_and_filter_blocks: 1 2024-04-24T09:09:56.284 INFO:journalctl@ceph.mon.a.smithi090.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-24T09:09:56.284 INFO:journalctl@ceph.mon.a.smithi090.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-24T09:09:56.284 INFO:journalctl@ceph.mon.a.smithi090.stdout: pin_top_level_index_and_filter: 1 2024-04-24T09:09:56.284 INFO:journalctl@ceph.mon.a.smithi090.stdout: index_type: 0 2024-04-24T09:09:56.284 INFO:journalctl@ceph.mon.a.smithi090.stdout: data_block_index_type: 0 2024-04-24T09:09:56.284 INFO:journalctl@ceph.mon.a.smithi090.stdout: index_shortening: 1 2024-04-24T09:09:56.284 INFO:journalctl@ceph.mon.a.smithi090.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-24T09:09:56.284 INFO:journalctl@ceph.mon.a.smithi090.stdout: checksum: 4 2024-04-24T09:09:56.284 INFO:journalctl@ceph.mon.a.smithi090.stdout: no_block_cache: 0 2024-04-24T09:09:56.284 INFO:journalctl@ceph.mon.a.smithi090.stdout: block_cache: 0x56351227b090 2024-04-24T09:09:56.284 INFO:journalctl@ceph.mon.a.smithi090.stdout: block_cache_name: BinnedLRUCache 2024-04-24T09:09:56.284 INFO:journalctl@ceph.mon.a.smithi090.stdout: block_cache_options: 2024-04-24T09:09:56.284 INFO:journalctl@ceph.mon.a.smithi090.stdout: capacity : 536870912 2024-04-24T09:09:56.285 INFO:journalctl@ceph.mon.a.smithi090.stdout: num_shard_bits : 4 2024-04-24T09:09:56.285 INFO:journalctl@ceph.mon.a.smithi090.stdout: strict_capacity_limit : 0 2024-04-24T09:09:56.285 INFO:journalctl@ceph.mon.a.smithi090.stdout: high_pri_pool_ratio: 0.000 2024-04-24T09:09:56.285 INFO:journalctl@ceph.mon.a.smithi090.stdout: block_cache_compressed: (nil) 2024-04-24T09:09:56.285 INFO:journalctl@ceph.mon.a.smithi090.stdout: persistent_cache: (nil) 2024-04-24T09:09:56.285 INFO:journalctl@ceph.mon.a.smithi090.stdout: block_size: 4096 2024-04-24T09:09:56.285 INFO:journalctl@ceph.mon.a.smithi090.stdout: block_size_deviation: 10 2024-04-24T09:09:56.285 INFO:journalctl@ceph.mon.a.smithi090.stdout: block_restart_interval: 16 2024-04-24T09:09:56.285 INFO:journalctl@ceph.mon.a.smithi090.stdout: index_block_restart_interval: 1 2024-04-24T09:09:56.285 INFO:journalctl@ceph.mon.a.smithi090.stdout: metadata_block_size: 4096 2024-04-24T09:09:56.285 INFO:journalctl@ceph.mon.a.smithi090.stdout: partition_filters: 0 2024-04-24T09:09:56.285 INFO:journalctl@ceph.mon.a.smithi090.stdout: use_delta_encoding: 1 2024-04-24T09:09:56.285 INFO:journalctl@ceph.mon.a.smithi090.stdout: filter_policy: bloomfilter 2024-04-24T09:09:56.285 INFO:journalctl@ceph.mon.a.smithi090.stdout: whole_key_filtering: 1 2024-04-24T09:09:56.286 INFO:journalctl@ceph.mon.a.smithi090.stdout: verify_compression: 0 2024-04-24T09:09:56.286 INFO:journalctl@ceph.mon.a.smithi090.stdout: read_amp_bytes_per_bit: 0 2024-04-24T09:09:56.286 INFO:journalctl@ceph.mon.a.smithi090.stdout: format_version: 5 2024-04-24T09:09:56.286 INFO:journalctl@ceph.mon.a.smithi090.stdout: enable_index_compression: 1 2024-04-24T09:09:56.286 INFO:journalctl@ceph.mon.a.smithi090.stdout: block_align: 0 2024-04-24T09:09:56.286 INFO:journalctl@ceph.mon.a.smithi090.stdout: max_auto_readahead_size: 262144 2024-04-24T09:09:56.286 INFO:journalctl@ceph.mon.a.smithi090.stdout: prepopulate_block_cache: 0 2024-04-24T09:09:56.286 INFO:journalctl@ceph.mon.a.smithi090.stdout: initial_auto_readahead_size: 8192 2024-04-24T09:09:56.286 INFO:journalctl@ceph.mon.a.smithi090.stdout: num_file_reads_for_auto_readahead: 2 2024-04-24T09:09:56.286 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.write_buffer_size: 33554432 2024-04-24T09:09:56.286 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.max_write_buffer_number: 2 2024-04-24T09:09:56.286 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.compression: NoCompression 2024-04-24T09:09:56.286 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.bottommost_compression: Disabled 2024-04-24T09:09:56.287 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.prefix_extractor: nullptr 2024-04-24T09:09:56.287 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-24T09:09:56.287 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.num_levels: 7 2024-04-24T09:09:56.287 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-24T09:09:56.287 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-24T09:09:56.287 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-24T09:09:56.287 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-24T09:09:56.287 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-24T09:09:56.287 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-24T09:09:56.287 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-24T09:09:56.287 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-24T09:09:56.287 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-24T09:09:56.287 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-24T09:09:56.287 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-24T09:09:56.288 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-24T09:09:56.288 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-24T09:09:56.288 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.compression_opts.level: 32767 2024-04-24T09:09:56.288 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.compression_opts.strategy: 0 2024-04-24T09:09:56.288 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-24T09:09:56.288 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-24T09:09:56.288 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-24T09:09:56.288 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-24T09:09:56.288 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.compression_opts.enabled: false 2024-04-24T09:09:56.288 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-24T09:09:56.288 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-24T09:09:56.288 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-24T09:09:56.288 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-24T09:09:56.288 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.target_file_size_base: 67108864 2024-04-24T09:09:56.289 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-24T09:09:56.289 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-24T09:09:56.289 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-24T09:09:56.289 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-24T09:09:56.289 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-24T09:09:56.289 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-24T09:09:56.290 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-24T09:09:56.290 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-24T09:09:56.290 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-24T09:09:56.290 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-24T09:09:56.290 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-24T09:09:56.290 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-24T09:09:56.290 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-24T09:09:56.290 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-24T09:09:56.290 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.arena_block_size: 1048576 2024-04-24T09:09:56.290 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-24T09:09:56.290 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-24T09:09:56.290 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.disable_auto_compactions: 0 2024-04-24T09:09:56.290 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-24T09:09:56.291 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-24T09:09:56.291 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-24T09:09:56.291 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-24T09:09:56.291 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-24T09:09:56.291 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-24T09:09:56.291 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-24T09:09:56.291 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-24T09:09:56.291 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-24T09:09:56.291 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-24T09:09:56.291 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-24T09:09:56.291 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.inplace_update_support: 0 2024-04-24T09:09:56.291 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-24T09:09:56.291 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-24T09:09:56.291 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-24T09:09:56.291 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-24T09:09:56.292 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.bloom_locality: 0 2024-04-24T09:09:56.292 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.max_successive_merges: 0 2024-04-24T09:09:56.292 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-24T09:09:56.292 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.paranoid_file_checks: 0 2024-04-24T09:09:56.292 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.force_consistency_checks: 1 2024-04-24T09:09:56.292 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.report_bg_io_stats: 0 2024-04-24T09:09:56.292 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.ttl: 2592000 2024-04-24T09:09:56.292 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-24T09:09:56.292 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-24T09:09:56.292 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-24T09:09:56.292 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.enable_blob_files: false 2024-04-24T09:09:56.292 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.min_blob_size: 0 2024-04-24T09:09:56.292 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.blob_file_size: 268435456 2024-04-24T09:09:56.292 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-24T09:09:56.293 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-24T09:09:56.293 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-24T09:09:56.293 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-24T09:09:56.293 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-24T09:09:56.293 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.blob_file_starting_level: 0 2024-04-24T09:09:56.293 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-24T09:09:56.293 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-24T09:09:56.293 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-24T09:09:56.293 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 71053d52-a544-45d2-a26d-de41dd5ba159 2024-04-24T09:09:56.293 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713949796046127, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-24T09:09:56.293 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-24T09:09:56.293 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713949796047945, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100875, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 328, "table_properties": {"data_size": 98764, "index_size": 303, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14328, "raw_average_key_size": 49, "raw_value_size": 90979, "raw_average_value_size": 317, "num_data_blocks": 12, "num_entries": 287, "num_filter_entries": 287, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713949796, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "71053d52-a544-45d2-a26d-de41dd5ba159", "db_session_id": "1RFW3RSEXZ01N4QUENU0", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-24T09:09:56.293 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713949796048152, "job": 1, "event": "recovery_finished"} 2024-04-24T09:09:56.293 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-24T09:09:56.293 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-24T09:09:56.294 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x563512368000 2024-04-24T09:09:56.294 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: DB pointer 0x563512352000 2024-04-24T09:09:56.294 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-24T09:09:56.294 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-24T09:09:56.294 INFO:journalctl@ceph.mon.a.smithi090.stdout: ** DB Stats ** 2024-04-24T09:09:56.294 INFO:journalctl@ceph.mon.a.smithi090.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T09:09:56.294 INFO:journalctl@ceph.mon.a.smithi090.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-24T09:09:56.294 INFO:journalctl@ceph.mon.a.smithi090.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T09:09:56.294 INFO:journalctl@ceph.mon.a.smithi090.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T09:09:56.294 INFO:journalctl@ceph.mon.a.smithi090.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-24T09:09:56.294 INFO:journalctl@ceph.mon.a.smithi090.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T09:09:56.294 INFO:journalctl@ceph.mon.a.smithi090.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T09:09:56.294 INFO:journalctl@ceph.mon.a.smithi090.stdout: 2024-04-24T09:09:56.295 INFO:journalctl@ceph.mon.a.smithi090.stdout: ** Compaction Stats [default] ** 2024-04-24T09:09:56.295 INFO:journalctl@ceph.mon.a.smithi090.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T09:09:56.295 INFO:journalctl@ceph.mon.a.smithi090.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-24T09:09:56.295 INFO:journalctl@ceph.mon.a.smithi090.stdout: L0 2/0 100.37 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 80.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T09:09:56.295 INFO:journalctl@ceph.mon.a.smithi090.stdout: Sum 2/0 100.37 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 80.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T09:09:56.295 INFO:journalctl@ceph.mon.a.smithi090.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 80.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T09:09:56.295 INFO:journalctl@ceph.mon.a.smithi090.stdout: 2024-04-24T09:09:56.295 INFO:journalctl@ceph.mon.a.smithi090.stdout: ** Compaction Stats [default] ** 2024-04-24T09:09:56.295 INFO:journalctl@ceph.mon.a.smithi090.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T09:09:56.295 INFO:journalctl@ceph.mon.a.smithi090.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-24T09:09:56.295 INFO:journalctl@ceph.mon.a.smithi090.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 80.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T09:09:56.295 INFO:journalctl@ceph.mon.a.smithi090.stdout: 2024-04-24T09:09:56.295 INFO:journalctl@ceph.mon.a.smithi090.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-24T09:09:56.295 INFO:journalctl@ceph.mon.a.smithi090.stdout: 2024-04-24T09:09:56.296 INFO:journalctl@ceph.mon.a.smithi090.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T09:09:56.296 INFO:journalctl@ceph.mon.a.smithi090.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-24T09:09:56.296 INFO:journalctl@ceph.mon.a.smithi090.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-24T09:09:56.296 INFO:journalctl@ceph.mon.a.smithi090.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-24T09:09:56.296 INFO:journalctl@ceph.mon.a.smithi090.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-24T09:09:56.296 INFO:journalctl@ceph.mon.a.smithi090.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-24T09:09:56.296 INFO:journalctl@ceph.mon.a.smithi090.stdout: Cumulative compaction: 0.00 GB write, 14.15 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T09:09:56.296 INFO:journalctl@ceph.mon.a.smithi090.stdout: Interval compaction: 0.00 GB write, 14.15 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T09:09:56.296 INFO:journalctl@ceph.mon.a.smithi090.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-24T09:09:56.296 INFO:journalctl@ceph.mon.a.smithi090.stdout: Block cache BinnedLRUCache@0x56351227b090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-04-24T09:09:56.296 INFO:journalctl@ceph.mon.a.smithi090.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-24T09:09:56.296 INFO:journalctl@ceph.mon.a.smithi090.stdout: 2024-04-24T09:09:56.296 INFO:journalctl@ceph.mon.a.smithi090.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-24T09:09:56.296 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: starting mon.a rank 0 at public addrs [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] at bind addrs [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 30ee9be2-021a-11ef-bc93-c7b262605968 2024-04-24T09:09:56.297 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: mon.a@-1(???) e1 preinit fsid 30ee9be2-021a-11ef-bc93-c7b262605968 2024-04-24T09:09:56.297 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: mon.a@-1(???).mds e1 new map 2024-04-24T09:09:56.297 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: mon.a@-1(???).mds e1 print_map 2024-04-24T09:09:56.297 INFO:journalctl@ceph.mon.a.smithi090.stdout: e1 2024-04-24T09:09:56.297 INFO:journalctl@ceph.mon.a.smithi090.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T09:09:56.297 INFO:journalctl@ceph.mon.a.smithi090.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-24T09:09:56.297 INFO:journalctl@ceph.mon.a.smithi090.stdout: legacy client fscid: -1 2024-04-24T09:09:56.297 INFO:journalctl@ceph.mon.a.smithi090.stdout: 2024-04-24T09:09:56.297 INFO:journalctl@ceph.mon.a.smithi090.stdout: No filesystems configured 2024-04-24T09:09:56.297 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-24T09:09:56.297 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T09:09:56.297 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T09:09:56.297 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T09:09:56.297 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-24T09:09:56.297 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 bash[27806]: b8ed5a8fba8a52eb8bc0a64687ab90e04a2d6b7eac98cd7bf5b1f54038c4736b 2024-04-24T09:09:56.297 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-24T09:09:56.298 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: monmap epoch 1 2024-04-24T09:09:56.298 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: fsid 30ee9be2-021a-11ef-bc93-c7b262605968 2024-04-24T09:09:56.298 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: last_changed 2024-04-24T09:09:44.652569+0000 2024-04-24T09:09:56.298 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: created 2024-04-24T09:09:44.652569+0000 2024-04-24T09:09:56.298 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: min_mon_release 19 (squid) 2024-04-24T09:09:56.298 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: election_strategy: 1 2024-04-24T09:09:56.298 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-04-24T09:09:56.298 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: fsmap 2024-04-24T09:09:56.298 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: osdmap e1: 0 total, 0 up, 0 in 2024-04-24T09:09:56.298 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 ceph-mon[27825]: mgrmap e1: no daemons active 2024-04-24T09:09:56.298 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:56 smithi090 systemd[1]: Started Ceph mon.a for 30ee9be2-021a-11ef-bc93-c7b262605968. 2024-04-24T09:09:57.604 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:56.816+0000 7faa84e00700 1 Processor -- start 2024-04-24T09:09:57.604 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:56.816+0000 7faa84e00700 1 -- start start 2024-04-24T09:09:57.604 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:56.817+0000 7faa84e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa8010a600 0x7faa8010aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:57.604 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:56.817+0000 7faa84e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7faa8010aff0 con 0x7faa8010a600 2024-04-24T09:09:57.604 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:56.818+0000 7faa7f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa8010a600 0x7faa8010aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:57.604 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:56.818+0000 7faa7f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa8010a600 0x7faa8010aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38754/0 (socket says 172.21.15.90:38754) 2024-04-24T09:09:57.604 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:56.818+0000 7faa7f400700 1 -- 172.21.15.90:0/16963595 learned_addr learned my addr 172.21.15.90:0/16963595 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:09:57.604 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:56.819+0000 7faa7f400700 1 -- 172.21.15.90:0/16963595 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa8010b7b0 con 0x7faa8010a600 2024-04-24T09:09:57.604 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:56.819+0000 7faa7f400700 1 --2- 172.21.15.90:0/16963595 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa8010a600 0x7faa8010aa20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7faa700098a0 tx=0x7faa70009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1487ad26dcd358fc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:57.604 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:56.820+0000 7faa7e000700 1 -- 172.21.15.90:0/16963595 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa70004160 con 0x7faa8010a600 2024-04-24T09:09:57.605 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:56.820+0000 7faa7e000700 1 -- 172.21.15.90:0/16963595 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7faa70004300 con 0x7faa8010a600 2024-04-24T09:09:57.605 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:56.820+0000 7faa7e000700 1 -- 172.21.15.90:0/16963595 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa70011b50 con 0x7faa8010a600 2024-04-24T09:09:57.605 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:56.820+0000 7faa84e00700 1 -- 172.21.15.90:0/16963595 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa8010a600 msgr2=0x7faa8010aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:57.605 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:56.820+0000 7faa84e00700 1 --2- 172.21.15.90:0/16963595 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa8010a600 0x7faa8010aa20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7faa700098a0 tx=0x7faa70009bb0 comp rx=0 tx=0).stop 2024-04-24T09:09:57.605 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:56.820+0000 7faa84e00700 1 -- 172.21.15.90:0/16963595 shutdown_connections 2024-04-24T09:09:57.605 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:56.820+0000 7faa84e00700 1 --2- 172.21.15.90:0/16963595 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa8010a600 0x7faa8010aa20 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:57.605 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:56.820+0000 7faa84e00700 1 -- 172.21.15.90:0/16963595 >> 172.21.15.90:0/16963595 conn(0x7faa8007dbb0 msgr2=0x7faa801084c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:57.605 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:56.821+0000 7faa84e00700 1 -- 172.21.15.90:0/16963595 shutdown_connections 2024-04-24T09:09:57.605 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:56.821+0000 7faa84e00700 1 -- 172.21.15.90:0/16963595 wait complete. 2024-04-24T09:09:57.605 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:56.822+0000 7faa84e00700 1 Processor -- start 2024-04-24T09:09:57.605 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:56.822+0000 7faa84e00700 1 -- start start 2024-04-24T09:09:57.605 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:56.822+0000 7faa84e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa8010a600 0x7faa8019e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:57.605 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:56.822+0000 7faa84e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7faa8010eb60 con 0x7faa8010a600 2024-04-24T09:09:57.605 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:56.822+0000 7faa7f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa8010a600 0x7faa8019e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:57.605 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:56.822+0000 7faa7f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa8010a600 0x7faa8019e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38762/0 (socket says 172.21.15.90:38762) 2024-04-24T09:09:57.606 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:56.822+0000 7faa7f400700 1 -- 172.21.15.90:0/2106451046 learned_addr learned my addr 172.21.15.90:0/2106451046 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:09:57.606 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:56.823+0000 7faa7f400700 1 -- 172.21.15.90:0/2106451046 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa8019eae0 con 0x7faa8010a600 2024-04-24T09:09:57.606 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:56.823+0000 7faa7f400700 1 --2- 172.21.15.90:0/2106451046 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa8010a600 0x7faa8019e5a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7faa70011850 tx=0x7faa70011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:57.606 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:56.823+0000 7faa77e00700 1 -- 172.21.15.90:0/2106451046 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa70003af0 con 0x7faa8010a600 2024-04-24T09:09:57.606 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:56.823+0000 7faa77e00700 1 -- 172.21.15.90:0/2106451046 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7faa70020070 con 0x7faa8010a600 2024-04-24T09:09:57.606 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:56.824+0000 7faa84e00700 1 -- 172.21.15.90:0/2106451046 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa8019ed70 con 0x7faa8010a600 2024-04-24T09:09:57.606 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:56.824+0000 7faa77e00700 1 -- 172.21.15.90:0/2106451046 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa700195a0 con 0x7faa8010a600 2024-04-24T09:09:57.606 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:56.824+0000 7faa84e00700 1 -- 172.21.15.90:0/2106451046 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa801a1a60 con 0x7faa8010a600 2024-04-24T09:09:57.606 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:56.824+0000 7faa77e00700 1 -- 172.21.15.90:0/2106451046 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7faa70019c30 con 0x7faa8010a600 2024-04-24T09:09:57.606 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:56.824+0000 7faa77e00700 1 -- 172.21.15.90:0/2106451046 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7faa70011e00 con 0x7faa8010a600 2024-04-24T09:09:57.606 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:56.825+0000 7faa84e00700 1 -- 172.21.15.90:0/2106451046 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa80064150 con 0x7faa8010a600 2024-04-24T09:09:57.606 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:56.828+0000 7faa77e00700 1 -- 172.21.15.90:0/2106451046 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7faa70003c90 con 0x7faa8010a600 2024-04-24T09:09:57.606 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:56.904+0000 7faa84e00700 1 -- 172.21.15.90:0/2106451046 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7faa801a1c30 con 0x7faa8010a600 2024-04-24T09:09:57.607 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:56.908+0000 7faa77e00700 1 -- 172.21.15.90:0/2106451046 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7faa7001a430 con 0x7faa8010a600 2024-04-24T09:09:57.607 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:56.909+0000 7faa84e00700 1 -- 172.21.15.90:0/2106451046 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa8010a600 msgr2=0x7faa8019e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:57.607 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:56.909+0000 7faa84e00700 1 --2- 172.21.15.90:0/2106451046 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa8010a600 0x7faa8019e5a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7faa70011850 tx=0x7faa70011c10 comp rx=0 tx=0).stop 2024-04-24T09:09:57.607 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:56.909+0000 7faa84e00700 1 -- 172.21.15.90:0/2106451046 shutdown_connections 2024-04-24T09:09:57.607 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:56.909+0000 7faa84e00700 1 --2- 172.21.15.90:0/2106451046 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa8010a600 0x7faa8019e5a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:57.607 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:56.909+0000 7faa84e00700 1 -- 172.21.15.90:0/2106451046 >> 172.21.15.90:0/2106451046 conn(0x7faa8007dbb0 msgr2=0x7faa80107cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:57.607 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:56.909+0000 7faa84e00700 1 -- 172.21.15.90:0/2106451046 shutdown_connections 2024-04-24T09:09:57.607 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:56.909+0000 7faa84e00700 1 -- 172.21.15.90:0/2106451046 wait complete. 2024-04-24T09:09:57.607 INFO:teuthology.orchestra.run.smithi090.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-24T09:09:57.628 INFO:teuthology.orchestra.run.smithi090.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-24T09:09:57.628 INFO:teuthology.orchestra.run.smithi090.stdout:Creating mgr... 2024-04-24T09:09:57.629 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-24T09:09:57.630 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-24T09:09:58.024 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mgr.x 2024-04-24T09:09:58.025 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Failed to reset failed state of unit ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-24T09:09:58.260 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-30ee9be2-021a-11ef-bc93-c7b262605968.target.wants/ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-30ee9be2-021a-11ef-bc93-c7b262605968@.service. 2024-04-24T09:09:58.287 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:57 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/2106451046' entity='client.admin' 2024-04-24T09:09:58.602 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:09:58 smithi090 systemd[1]: Starting Ceph mgr.x for 30ee9be2-021a-11ef-bc93-c7b262605968... 2024-04-24T09:09:58.951 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:09:58 smithi090 podman[28037]: 2024-04-24 09:09:58.542641225 +0000 UTC m=+0.032000965 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T09:09:58.951 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:09:58 smithi090 podman[28037]: 2024-04-24 09:09:58.640357438 +0000 UTC m=+0.129717209 container create 71027b126301ce439ed26339b3e252f56e5b3de33c2fec7549d1e6f82db4859e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS) 2024-04-24T09:09:58.952 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:09:58 smithi090 podman[28037]: 2024-04-24 09:09:58.87048715 +0000 UTC m=+0.359846934 container init 71027b126301ce439ed26339b3e252f56e5b3de33c2fec7549d1e6f82db4859e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T09:09:58.952 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:09:58 smithi090 podman[28037]: 2024-04-24 09:09:58.879916758 +0000 UTC m=+0.369276515 container start 71027b126301ce439ed26339b3e252f56e5b3de33c2fec7549d1e6f82db4859e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-24T09:09:58.989 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T09:09:58.989 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout disabled 2024-04-24T09:09:59.008 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T09:09:59.009 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout inactive 2024-04-24T09:09:59.009 INFO:teuthology.orchestra.run.smithi090.stdout:firewalld.service is not enabled 2024-04-24T09:09:59.009 INFO:teuthology.orchestra.run.smithi090.stdout:Not possible to enable service . firewalld.service is not available 2024-04-24T09:09:59.037 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T09:09:59.037 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout disabled 2024-04-24T09:09:59.059 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T09:09:59.059 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout inactive 2024-04-24T09:09:59.060 INFO:teuthology.orchestra.run.smithi090.stdout:firewalld.service is not enabled 2024-04-24T09:09:59.060 INFO:teuthology.orchestra.run.smithi090.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-24T09:09:59.060 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for mgr to start... 2024-04-24T09:09:59.060 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for mgr... 2024-04-24T09:09:59.215 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:09:58 smithi090 bash[28037]: 71027b126301ce439ed26339b3e252f56e5b3de33c2fec7549d1e6f82db4859e 2024-04-24T09:09:59.215 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:09:58 smithi090 systemd[1]: Started Ceph mgr.x for 30ee9be2-021a-11ef-bc93-c7b262605968. 2024-04-24T09:09:59.215 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:09:59 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:09:59.129+0000 7f75b5096200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T09:09:59.467 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:09:59 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:09:59.393+0000 7f75b5096200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T09:09:59.803 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:09:59 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:09:59.538+0000 7f75b5096200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T09:10:00.080 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:09:59 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/1927344303' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-24T09:10:00.080 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:09:59 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:09:59.961+0000 7f75b5096200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T09:10:00.080 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:00 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:00.079+0000 7f75b5096200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T09:10:00.453 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-24T09:10:00.465 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-04-24T09:10:00.465 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsid": "30ee9be2-021a-11ef-bc93-c7b262605968", 2024-04-24T09:10:00.465 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "health": { 2024-04-24T09:10:00.465 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-24T09:10:00.466 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-24T09:10:00.466 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-24T09:10:00.466 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T09:10:00.466 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-24T09:10:00.466 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-24T09:10:00.466 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 0 2024-04-24T09:10:00.466 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-04-24T09:10:00.466 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-24T09:10:00.466 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "a" 2024-04-24T09:10:00.466 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-04-24T09:10:00.466 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-04-24T09:10:00.466 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-24T09:10:00.466 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T09:10:00.467 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-24T09:10:00.467 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-24T09:10:00.467 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T09:10:00.467 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-24T09:10:00.467 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T09:10:00.467 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-24T09:10:00.467 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-24T09:10:00.467 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-24T09:10:00.467 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-24T09:10:00.467 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-24T09:10:00.467 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-24T09:10:00.467 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T09:10:00.467 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-24T09:10:00.467 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-24T09:10:00.467 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-24T09:10:00.467 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-24T09:10:00.467 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-24T09:10:00.468 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-24T09:10:00.468 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-24T09:10:00.468 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-24T09:10:00.468 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-24T09:10:00.468 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T09:10:00.468 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-24T09:10:00.468 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T09:10:00.468 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-24T09:10:00.468 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-24T09:10:00.468 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T09:10:00.468 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-24T09:10:00.468 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-24T09:10:00.468 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-24T09:10:00.468 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-24T09:10:00.468 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-24T09:10:00.469 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-24T09:10:00.469 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "restful" 2024-04-24T09:10:00.469 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-04-24T09:10:00.469 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T09:10:00.469 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T09:10:00.469 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-24T09:10:00.469 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T09:10:00.469 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modified": "2024-04-24T09:09:49.496572+0000", 2024-04-24T09:10:00.469 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T09:10:00.469 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T09:10:00.469 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-24T09:10:00.469 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-04-24T09:10:00.469 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:59.713+0000 7fce44e00700 1 Processor -- start 2024-04-24T09:10:00.469 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:59.713+0000 7fce44e00700 1 -- start start 2024-04-24T09:10:00.469 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:59.714+0000 7fce44e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce4007de60 0x7fce4007c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:00.470 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:59.714+0000 7fce44e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fce4007c8e0 con 0x7fce4007de60 2024-04-24T09:10:00.470 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:59.714+0000 7fce3f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce4007de60 0x7fce4007c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:00.470 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:59.714+0000 7fce3f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce4007de60 0x7fce4007c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38788/0 (socket says 172.21.15.90:38788) 2024-04-24T09:10:00.470 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:59.714+0000 7fce3f400700 1 -- 172.21.15.90:0/1674183277 learned_addr learned my addr 172.21.15.90:0/1674183277 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:10:00.470 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:59.715+0000 7fce3f400700 1 -- 172.21.15.90:0/1674183277 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce4007ca60 con 0x7fce4007de60 2024-04-24T09:10:00.470 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:59.715+0000 7fce3f400700 1 --2- 172.21.15.90:0/1674183277 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce4007de60 0x7fce4007c310 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fce280098a0 tx=0x7fce28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ff1ad10076cf1b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:00.470 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:59.715+0000 7fce3e000700 1 -- 172.21.15.90:0/1674183277 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce28004160 con 0x7fce4007de60 2024-04-24T09:10:00.470 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:59.716+0000 7fce3e000700 1 -- 172.21.15.90:0/1674183277 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fce28004300 con 0x7fce4007de60 2024-04-24T09:10:00.470 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:59.716+0000 7fce3e000700 1 -- 172.21.15.90:0/1674183277 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce28011b50 con 0x7fce4007de60 2024-04-24T09:10:00.470 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:59.716+0000 7fce44e00700 1 -- 172.21.15.90:0/1674183277 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce4007de60 msgr2=0x7fce4007c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:00.470 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:59.716+0000 7fce44e00700 1 --2- 172.21.15.90:0/1674183277 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce4007de60 0x7fce4007c310 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fce280098a0 tx=0x7fce28009bb0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.470 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:59.716+0000 7fce44e00700 1 -- 172.21.15.90:0/1674183277 shutdown_connections 2024-04-24T09:10:00.470 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:59.716+0000 7fce44e00700 1 --2- 172.21.15.90:0/1674183277 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce4007de60 0x7fce4007c310 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.470 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:59.716+0000 7fce44e00700 1 -- 172.21.15.90:0/1674183277 >> 172.21.15.90:0/1674183277 conn(0x7fce40103a50 msgr2=0x7fce40105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:00.471 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:59.717+0000 7fce44e00700 1 -- 172.21.15.90:0/1674183277 shutdown_connections 2024-04-24T09:10:00.471 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:59.717+0000 7fce44e00700 1 -- 172.21.15.90:0/1674183277 wait complete. 2024-04-24T09:10:00.471 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:59.718+0000 7fce44e00700 1 Processor -- start 2024-04-24T09:10:00.471 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:59.718+0000 7fce44e00700 1 -- start start 2024-04-24T09:10:00.471 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:59.718+0000 7fce44e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce4007de60 0x7fce4019e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:00.471 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:59.718+0000 7fce44e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fce4010e9a0 con 0x7fce4007de60 2024-04-24T09:10:00.471 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:59.719+0000 7fce3f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce4007de60 0x7fce4019e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:00.471 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:59.719+0000 7fce3f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce4007de60 0x7fce4019e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38796/0 (socket says 172.21.15.90:38796) 2024-04-24T09:10:00.471 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:59.719+0000 7fce3f400700 1 -- 172.21.15.90:0/1927344303 learned_addr learned my addr 172.21.15.90:0/1927344303 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:10:00.471 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:59.719+0000 7fce3f400700 1 -- 172.21.15.90:0/1927344303 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce4019eaf0 con 0x7fce4007de60 2024-04-24T09:10:00.471 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:59.719+0000 7fce3f400700 1 --2- 172.21.15.90:0/1927344303 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce4007de60 0x7fce4019e5b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fce28011b60 tx=0x7fce28011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:00.471 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:59.720+0000 7fce37e00700 1 -- 172.21.15.90:0/1927344303 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce28003970 con 0x7fce4007de60 2024-04-24T09:10:00.471 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:59.720+0000 7fce37e00700 1 -- 172.21.15.90:0/1927344303 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fce28003b10 con 0x7fce4007de60 2024-04-24T09:10:00.471 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:59.720+0000 7fce44e00700 1 -- 172.21.15.90:0/1927344303 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce4019ed80 con 0x7fce4007de60 2024-04-24T09:10:00.471 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:59.720+0000 7fce37e00700 1 -- 172.21.15.90:0/1927344303 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce28024610 con 0x7fce4007de60 2024-04-24T09:10:00.472 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:59.720+0000 7fce44e00700 1 -- 172.21.15.90:0/1927344303 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce401a1a70 con 0x7fce4007de60 2024-04-24T09:10:00.472 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:59.720+0000 7fce37e00700 1 -- 172.21.15.90:0/1927344303 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fce28024ca0 con 0x7fce4007de60 2024-04-24T09:10:00.472 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:59.720+0000 7fce37e00700 1 -- 172.21.15.90:0/1927344303 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fce28011e00 con 0x7fce4007de60 2024-04-24T09:10:00.472 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:59.721+0000 7fce44e00700 1 -- 172.21.15.90:0/1927344303 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce40064150 con 0x7fce4007de60 2024-04-24T09:10:00.472 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:59.724+0000 7fce37e00700 1 -- 172.21.15.90:0/1927344303 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fce28043e50 con 0x7fce4007de60 2024-04-24T09:10:00.472 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:59.803+0000 7fce44e00700 1 -- 172.21.15.90:0/1927344303 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fce401a1c40 con 0x7fce4007de60 2024-04-24T09:10:00.472 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:59.804+0000 7fce37e00700 1 -- 172.21.15.90:0/1927344303 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fce28032030 con 0x7fce4007de60 2024-04-24T09:10:00.472 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:59.805+0000 7fce44e00700 1 -- 172.21.15.90:0/1927344303 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce4007de60 msgr2=0x7fce4019e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:00.472 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:59.805+0000 7fce44e00700 1 --2- 172.21.15.90:0/1927344303 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce4007de60 0x7fce4019e5b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fce28011b60 tx=0x7fce28011c40 comp rx=0 tx=0).stop 2024-04-24T09:10:00.472 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:59.805+0000 7fce44e00700 1 -- 172.21.15.90:0/1927344303 shutdown_connections 2024-04-24T09:10:00.472 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:59.805+0000 7fce44e00700 1 --2- 172.21.15.90:0/1927344303 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce4007de60 0x7fce4019e5b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:00.472 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:59.805+0000 7fce44e00700 1 -- 172.21.15.90:0/1927344303 >> 172.21.15.90:0/1927344303 conn(0x7fce40103a50 msgr2=0x7fce401045b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:00.472 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:59.805+0000 7fce44e00700 1 -- 172.21.15.90:0/1927344303 shutdown_connections 2024-04-24T09:10:00.472 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:09:59.805+0000 7fce44e00700 1 -- 172.21.15.90:0/1927344303 wait complete. 2024-04-24T09:10:00.472 INFO:teuthology.orchestra.run.smithi090.stdout:mgr not available, waiting (1/15)... 2024-04-24T09:10:02.092 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:01 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:01.657+0000 7f75b5096200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T09:10:02.092 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:01 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:01.881+0000 7f75b5096200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T09:10:02.352 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:02 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:02.090+0000 7f75b5096200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T09:10:02.738 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:02 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:02.464+0000 7f75b5096200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T09:10:02.739 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:02 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:02.588+0000 7f75b5096200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T09:10:03.035 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:02 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:02.847+0000 7f75b5096200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T09:10:03.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:03 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/3122391322' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-24T09:10:03.352 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:03 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:03.105+0000 7f75b5096200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T09:10:03.752 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-24T09:10:03.752 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-04-24T09:10:03.752 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsid": "30ee9be2-021a-11ef-bc93-c7b262605968", 2024-04-24T09:10:03.752 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "health": { 2024-04-24T09:10:03.752 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-24T09:10:03.752 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-24T09:10:03.753 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-24T09:10:03.753 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T09:10:03.753 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-24T09:10:03.753 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-24T09:10:03.753 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 0 2024-04-24T09:10:03.753 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-04-24T09:10:03.753 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-24T09:10:03.753 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "a" 2024-04-24T09:10:03.753 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-04-24T09:10:03.753 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-04-24T09:10:03.753 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-24T09:10:03.753 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T09:10:03.753 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-24T09:10:03.753 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-24T09:10:03.753 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T09:10:03.754 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-24T09:10:03.754 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T09:10:03.754 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-24T09:10:03.754 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-24T09:10:03.754 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-24T09:10:03.762 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-24T09:10:03.762 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-24T09:10:03.762 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-24T09:10:03.762 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T09:10:03.762 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-24T09:10:03.762 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-24T09:10:03.762 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-24T09:10:03.762 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-24T09:10:03.763 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-24T09:10:03.763 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-24T09:10:03.763 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-24T09:10:03.763 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-24T09:10:03.763 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-24T09:10:03.763 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T09:10:03.763 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-24T09:10:03.763 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T09:10:03.763 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-24T09:10:03.763 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-24T09:10:03.763 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T09:10:03.763 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-24T09:10:03.763 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-24T09:10:03.763 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-24T09:10:03.764 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-24T09:10:03.764 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-24T09:10:03.764 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-24T09:10:03.764 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "restful" 2024-04-24T09:10:03.764 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-04-24T09:10:03.764 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T09:10:03.764 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T09:10:03.764 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-24T09:10:03.764 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T09:10:03.764 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modified": "2024-04-24T09:09:49.496572+0000", 2024-04-24T09:10:03.764 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T09:10:03.764 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T09:10:03.765 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-24T09:10:03.765 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-04-24T09:10:03.765 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:03.034+0000 7ff5fda00700 1 Processor -- start 2024-04-24T09:10:03.765 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:03.035+0000 7ff5fda00700 1 -- start start 2024-04-24T09:10:03.765 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:03.035+0000 7ff5fda00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5f8108410 0x7ff5f8108830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:03.765 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:03.035+0000 7ff5fda00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f8108e00 con 0x7ff5f8108410 2024-04-24T09:10:03.765 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:03.035+0000 7ff5f7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5f8108410 0x7ff5f8108830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:03.765 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:03.035+0000 7ff5f7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5f8108410 0x7ff5f8108830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38804/0 (socket says 172.21.15.90:38804) 2024-04-24T09:10:03.765 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:03.035+0000 7ff5f7e00700 1 -- 172.21.15.90:0/4060509677 learned_addr learned my addr 172.21.15.90:0/4060509677 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:10:03.765 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:03.036+0000 7ff5f7e00700 1 -- 172.21.15.90:0/4060509677 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f81095c0 con 0x7ff5f8108410 2024-04-24T09:10:03.765 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:03.036+0000 7ff5f7e00700 1 --2- 172.21.15.90:0/4060509677 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5f8108410 0x7ff5f8108830 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff5e80098a0 tx=0x7ff5e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b9ab23523e941ef server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:03.765 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:03.036+0000 7ff5f6a00700 1 -- 172.21.15.90:0/4060509677 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5e8004160 con 0x7ff5f8108410 2024-04-24T09:10:03.765 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:03.036+0000 7ff5f6a00700 1 -- 172.21.15.90:0/4060509677 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff5e8004300 con 0x7ff5f8108410 2024-04-24T09:10:03.766 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:03.036+0000 7ff5f6a00700 1 -- 172.21.15.90:0/4060509677 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5e8011b50 con 0x7ff5f8108410 2024-04-24T09:10:03.766 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:03.037+0000 7ff5fda00700 1 -- 172.21.15.90:0/4060509677 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5f8108410 msgr2=0x7ff5f8108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:03.766 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:03.037+0000 7ff5fda00700 1 --2- 172.21.15.90:0/4060509677 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5f8108410 0x7ff5f8108830 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff5e80098a0 tx=0x7ff5e8009bb0 comp rx=0 tx=0).stop 2024-04-24T09:10:03.766 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:03.037+0000 7ff5fda00700 1 -- 172.21.15.90:0/4060509677 shutdown_connections 2024-04-24T09:10:03.766 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:03.037+0000 7ff5fda00700 1 --2- 172.21.15.90:0/4060509677 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5f8108410 0x7ff5f8108830 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:03.766 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:03.037+0000 7ff5fda00700 1 -- 172.21.15.90:0/4060509677 >> 172.21.15.90:0/4060509677 conn(0x7ff5f8103a50 msgr2=0x7ff5f8105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:03.766 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:03.037+0000 7ff5fda00700 1 -- 172.21.15.90:0/4060509677 shutdown_connections 2024-04-24T09:10:03.766 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:03.037+0000 7ff5fda00700 1 -- 172.21.15.90:0/4060509677 wait complete. 2024-04-24T09:10:03.766 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:03.038+0000 7ff5fda00700 1 Processor -- start 2024-04-24T09:10:03.766 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:03.039+0000 7ff5fda00700 1 -- start start 2024-04-24T09:10:03.766 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:03.039+0000 7ff5fda00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5f8108410 0x7ff5f819c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:03.766 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:03.040+0000 7ff5fda00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f810c970 con 0x7ff5f8108410 2024-04-24T09:10:03.766 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:03.040+0000 7ff5f7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5f8108410 0x7ff5f819c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:03.767 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:03.040+0000 7ff5f7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5f8108410 0x7ff5f819c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38818/0 (socket says 172.21.15.90:38818) 2024-04-24T09:10:03.767 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:03.040+0000 7ff5f7e00700 1 -- 172.21.15.90:0/3122391322 learned_addr learned my addr 172.21.15.90:0/3122391322 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:10:03.767 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:03.041+0000 7ff5f7e00700 1 -- 172.21.15.90:0/3122391322 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f819c920 con 0x7ff5f8108410 2024-04-24T09:10:03.767 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:03.041+0000 7ff5f7e00700 1 --2- 172.21.15.90:0/3122391322 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5f8108410 0x7ff5f819c3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8011850 tx=0x7ff5e8011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:03.767 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:03.041+0000 7ff5f4c00700 1 -- 172.21.15.90:0/3122391322 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5e8003af0 con 0x7ff5f8108410 2024-04-24T09:10:03.767 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:03.041+0000 7ff5f4c00700 1 -- 172.21.15.90:0/3122391322 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff5e8020070 con 0x7ff5f8108410 2024-04-24T09:10:03.767 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:03.041+0000 7ff5f4c00700 1 -- 172.21.15.90:0/3122391322 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5e80195a0 con 0x7ff5f8108410 2024-04-24T09:10:03.767 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:03.041+0000 7ff5fda00700 1 -- 172.21.15.90:0/3122391322 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5f819cbb0 con 0x7ff5f8108410 2024-04-24T09:10:03.767 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:03.042+0000 7ff5fda00700 1 -- 172.21.15.90:0/3122391322 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5f819f8a0 con 0x7ff5f8108410 2024-04-24T09:10:03.767 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:03.042+0000 7ff5f4c00700 1 -- 172.21.15.90:0/3122391322 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff5e8019be0 con 0x7ff5f8108410 2024-04-24T09:10:03.767 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:03.042+0000 7ff5f4c00700 1 -- 172.21.15.90:0/3122391322 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff5e8011e00 con 0x7ff5f8108410 2024-04-24T09:10:03.767 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:03.043+0000 7ff5fda00700 1 -- 172.21.15.90:0/3122391322 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5f8064150 con 0x7ff5f8108410 2024-04-24T09:10:03.768 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:03.045+0000 7ff5f4c00700 1 -- 172.21.15.90:0/3122391322 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7ff5e8003c90 con 0x7ff5f8108410 2024-04-24T09:10:03.768 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:03.125+0000 7ff5fda00700 1 -- 172.21.15.90:0/3122391322 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ff5f819fe60 con 0x7ff5f8108410 2024-04-24T09:10:03.768 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:03.126+0000 7ff5f4c00700 1 -- 172.21.15.90:0/3122391322 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7ff5e801a430 con 0x7ff5f8108410 2024-04-24T09:10:03.768 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:03.127+0000 7ff5fda00700 1 -- 172.21.15.90:0/3122391322 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5f8108410 msgr2=0x7ff5f819c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:03.768 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:03.127+0000 7ff5fda00700 1 --2- 172.21.15.90:0/3122391322 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5f8108410 0x7ff5f819c3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8011850 tx=0x7ff5e8011c10 comp rx=0 tx=0).stop 2024-04-24T09:10:03.768 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:03.127+0000 7ff5fda00700 1 -- 172.21.15.90:0/3122391322 shutdown_connections 2024-04-24T09:10:03.768 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:03.127+0000 7ff5fda00700 1 --2- 172.21.15.90:0/3122391322 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5f8108410 0x7ff5f819c3e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:03.768 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:03.127+0000 7ff5fda00700 1 -- 172.21.15.90:0/3122391322 >> 172.21.15.90:0/3122391322 conn(0x7ff5f8103a50 msgr2=0x7ff5f8104600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:03.768 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:03.127+0000 7ff5fda00700 1 -- 172.21.15.90:0/3122391322 shutdown_connections 2024-04-24T09:10:03.768 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:03.127+0000 7ff5fda00700 1 -- 172.21.15.90:0/3122391322 wait complete. 2024-04-24T09:10:03.768 INFO:teuthology.orchestra.run.smithi090.stdout:mgr not available, waiting (2/15)... 2024-04-24T09:10:04.102 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:03 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:03.917+0000 7f75b5096200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T09:10:04.850 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:04 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:04.464+0000 7f75b5096200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T09:10:04.850 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:04 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:04.589+0000 7f75b5096200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T09:10:04.851 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:04 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:04.849+0000 7f75b5096200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T09:10:05.852 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:05 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:05.481+0000 7f75b5096200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T09:10:05.852 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:05 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:05.601+0000 7f75b5096200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T09:10:06.232 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:05 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:05.922+0000 7f75b5096200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T09:10:06.232 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:06 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:06.107+0000 7f75b5096200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T09:10:06.232 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:06 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:06.232+0000 7f75b5096200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T09:10:06.493 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:06 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:06.389+0000 7f75b5096200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T09:10:06.767 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:06 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/841518648' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-24T09:10:07.101 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:06 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:06.767+0000 7f75b5096200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T09:10:07.264 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-24T09:10:07.265 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-04-24T09:10:07.265 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsid": "30ee9be2-021a-11ef-bc93-c7b262605968", 2024-04-24T09:10:07.265 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "health": { 2024-04-24T09:10:07.265 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-24T09:10:07.265 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-24T09:10:07.265 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-24T09:10:07.266 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T09:10:07.266 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-24T09:10:07.266 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-24T09:10:07.266 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 0 2024-04-24T09:10:07.266 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-04-24T09:10:07.266 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-24T09:10:07.266 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "a" 2024-04-24T09:10:07.272 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-04-24T09:10:07.272 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-04-24T09:10:07.272 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-24T09:10:07.272 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T09:10:07.272 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-24T09:10:07.272 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-24T09:10:07.272 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T09:10:07.272 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-24T09:10:07.272 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T09:10:07.272 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-24T09:10:07.272 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-24T09:10:07.272 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-24T09:10:07.272 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-24T09:10:07.273 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-24T09:10:07.273 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-24T09:10:07.273 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T09:10:07.273 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-24T09:10:07.273 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-24T09:10:07.273 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-24T09:10:07.273 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-24T09:10:07.273 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-24T09:10:07.273 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-24T09:10:07.273 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-24T09:10:07.273 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-24T09:10:07.273 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-24T09:10:07.273 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T09:10:07.273 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-24T09:10:07.274 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T09:10:07.274 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-24T09:10:07.274 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-24T09:10:07.274 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T09:10:07.274 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-24T09:10:07.274 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-24T09:10:07.274 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-24T09:10:07.274 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-24T09:10:07.274 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-24T09:10:07.274 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-24T09:10:07.274 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "restful" 2024-04-24T09:10:07.274 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-04-24T09:10:07.274 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T09:10:07.274 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T09:10:07.275 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-24T09:10:07.275 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T09:10:07.275 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modified": "2024-04-24T09:09:49.496572+0000", 2024-04-24T09:10:07.275 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T09:10:07.275 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T09:10:07.275 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-24T09:10:07.275 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-04-24T09:10:07.275 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:06.401+0000 7f5257e00700 1 Processor -- start 2024-04-24T09:10:07.275 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:06.402+0000 7f5257e00700 1 -- start start 2024-04-24T09:10:07.280 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:06.402+0000 7f5257e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f525807de80 0x7f525807c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:07.280 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:06.402+0000 7f5257e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f525807c900 con 0x7f525807de80 2024-04-24T09:10:07.280 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:06.402+0000 7f5256a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f525807de80 0x7f525807c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:07.280 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:06.402+0000 7f5256a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f525807de80 0x7f525807c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57814/0 (socket says 172.21.15.90:57814) 2024-04-24T09:10:07.280 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:06.403+0000 7f5256a00700 1 -- 172.21.15.90:0/311859803 learned_addr learned my addr 172.21.15.90:0/311859803 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:10:07.280 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:06.403+0000 7f5256a00700 1 -- 172.21.15.90:0/311859803 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f525807ca80 con 0x7f525807de80 2024-04-24T09:10:07.280 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:06.403+0000 7f5256a00700 1 --2- 172.21.15.90:0/311859803 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f525807de80 0x7f525807c330 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f52480098a0 tx=0x7f5248009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb4dfeec0e024821 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:07.280 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:06.403+0000 7f5255600700 1 -- 172.21.15.90:0/311859803 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5248004160 con 0x7f525807de80 2024-04-24T09:10:07.280 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:06.404+0000 7f5255600700 1 -- 172.21.15.90:0/311859803 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5248004300 con 0x7f525807de80 2024-04-24T09:10:07.281 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:06.404+0000 7f5255600700 1 -- 172.21.15.90:0/311859803 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5248011b50 con 0x7f525807de80 2024-04-24T09:10:07.281 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:06.404+0000 7f5257e00700 1 -- 172.21.15.90:0/311859803 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f525807de80 msgr2=0x7f525807c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:07.281 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:06.404+0000 7f5257e00700 1 --2- 172.21.15.90:0/311859803 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f525807de80 0x7f525807c330 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f52480098a0 tx=0x7f5248009bb0 comp rx=0 tx=0).stop 2024-04-24T09:10:07.281 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:06.404+0000 7f5257e00700 1 -- 172.21.15.90:0/311859803 shutdown_connections 2024-04-24T09:10:07.281 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:06.404+0000 7f5257e00700 1 --2- 172.21.15.90:0/311859803 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f525807de80 0x7f525807c330 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:07.281 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:06.404+0000 7f5257e00700 1 -- 172.21.15.90:0/311859803 >> 172.21.15.90:0/311859803 conn(0x7f5258103a50 msgr2=0x7f5258105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:07.281 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:06.405+0000 7f5257e00700 1 -- 172.21.15.90:0/311859803 shutdown_connections 2024-04-24T09:10:07.281 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:06.405+0000 7f5257e00700 1 -- 172.21.15.90:0/311859803 wait complete. 2024-04-24T09:10:07.281 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:06.405+0000 7f5257e00700 1 Processor -- start 2024-04-24T09:10:07.281 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:06.406+0000 7f5257e00700 1 -- start start 2024-04-24T09:10:07.281 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:06.406+0000 7f5257e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f525807de80 0x7f525819e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:07.281 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:06.406+0000 7f5257e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f525810e9e0 con 0x7f525807de80 2024-04-24T09:10:07.281 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:06.406+0000 7f5256a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f525807de80 0x7f525819e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:07.282 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:06.406+0000 7f5256a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f525807de80 0x7f525819e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57816/0 (socket says 172.21.15.90:57816) 2024-04-24T09:10:07.282 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:06.406+0000 7f5256a00700 1 -- 172.21.15.90:0/841518648 learned_addr learned my addr 172.21.15.90:0/841518648 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:10:07.282 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:06.407+0000 7f5256a00700 1 -- 172.21.15.90:0/841518648 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f525819eaf0 con 0x7f525807de80 2024-04-24T09:10:07.282 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:06.407+0000 7f5256a00700 1 --2- 172.21.15.90:0/841518648 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f525807de80 0x7f525819e5b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5248011b60 tx=0x7f5248011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:07.282 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:06.407+0000 7f524f400700 1 -- 172.21.15.90:0/841518648 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5248003970 con 0x7f525807de80 2024-04-24T09:10:07.282 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:06.407+0000 7f524f400700 1 -- 172.21.15.90:0/841518648 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5248003b10 con 0x7f525807de80 2024-04-24T09:10:07.282 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:06.407+0000 7f5257e00700 1 -- 172.21.15.90:0/841518648 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f525819ed80 con 0x7f525807de80 2024-04-24T09:10:07.282 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:06.407+0000 7f524f400700 1 -- 172.21.15.90:0/841518648 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5248024610 con 0x7f525807de80 2024-04-24T09:10:07.282 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:06.408+0000 7f5257e00700 1 -- 172.21.15.90:0/841518648 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52581a1a70 con 0x7f525807de80 2024-04-24T09:10:07.282 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:06.408+0000 7f524f400700 1 -- 172.21.15.90:0/841518648 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5248024ca0 con 0x7f525807de80 2024-04-24T09:10:07.282 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:06.408+0000 7f524f400700 1 -- 172.21.15.90:0/841518648 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5248011e00 con 0x7f525807de80 2024-04-24T09:10:07.282 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:06.409+0000 7f524cc00700 1 -- 172.21.15.90:0/841518648 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5230005330 con 0x7f525807de80 2024-04-24T09:10:07.282 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:06.411+0000 7f524f400700 1 -- 172.21.15.90:0/841518648 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f5248043e50 con 0x7f525807de80 2024-04-24T09:10:07.282 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:06.492+0000 7f524cc00700 1 -- 172.21.15.90:0/841518648 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f52300058f0 con 0x7f525807de80 2024-04-24T09:10:07.283 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:06.493+0000 7f524f400700 1 -- 172.21.15.90:0/841518648 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7f5248032030 con 0x7f525807de80 2024-04-24T09:10:07.283 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:06.494+0000 7f524cc00700 1 -- 172.21.15.90:0/841518648 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f525807de80 msgr2=0x7f525819e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:07.283 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:06.494+0000 7f524cc00700 1 --2- 172.21.15.90:0/841518648 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f525807de80 0x7f525819e5b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5248011b60 tx=0x7f5248011c40 comp rx=0 tx=0).stop 2024-04-24T09:10:07.283 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:06.495+0000 7f524cc00700 1 -- 172.21.15.90:0/841518648 shutdown_connections 2024-04-24T09:10:07.283 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:06.495+0000 7f524cc00700 1 --2- 172.21.15.90:0/841518648 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f525807de80 0x7f525819e5b0 secure :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5248011b60 tx=0x7f5248011c40 comp rx=0 tx=0).stop 2024-04-24T09:10:07.283 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:06.495+0000 7f524cc00700 1 -- 172.21.15.90:0/841518648 >> 172.21.15.90:0/841518648 conn(0x7f5258103a50 msgr2=0x7f52581045b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:07.283 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:06.495+0000 7f524cc00700 1 -- 172.21.15.90:0/841518648 shutdown_connections 2024-04-24T09:10:07.283 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:06.495+0000 7f524cc00700 1 -- 172.21.15.90:0/841518648 wait complete. 2024-04-24T09:10:07.283 INFO:teuthology.orchestra.run.smithi090.stdout:mgr not available, waiting (3/15)... 2024-04-24T09:10:07.808 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:07 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:07.528+0000 7f75b5096200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T09:10:08.102 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:07 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:07.806+0000 7f75b5096200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T09:10:08.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:07 smithi090 ceph-mon[27825]: Activating manager daemon x 2024-04-24T09:10:08.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:07 smithi090 ceph-mon[27825]: mgrmap e2: x(active, starting, since 0.0108069s) 2024-04-24T09:10:08.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:07 smithi090 ceph-mon[27825]: from='mgr.14100 172.21.15.90:0/2265659422' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T09:10:08.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:07 smithi090 ceph-mon[27825]: from='mgr.14100 172.21.15.90:0/2265659422' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T09:10:08.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:07 smithi090 ceph-mon[27825]: from='mgr.14100 172.21.15.90:0/2265659422' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T09:10:08.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:07 smithi090 ceph-mon[27825]: from='mgr.14100 172.21.15.90:0/2265659422' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T09:10:08.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:07 smithi090 ceph-mon[27825]: from='mgr.14100 172.21.15.90:0/2265659422' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-24T09:10:08.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:07 smithi090 ceph-mon[27825]: Manager daemon x is now available 2024-04-24T09:10:08.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:07 smithi090 ceph-mon[27825]: from='mgr.14100 172.21.15.90:0/2265659422' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T09:10:08.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:07 smithi090 ceph-mon[27825]: from='mgr.14100 172.21.15.90:0/2265659422' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T09:10:08.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:07 smithi090 ceph-mon[27825]: from='mgr.14100 172.21.15.90:0/2265659422' entity='mgr.x' 2024-04-24T09:10:08.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:07 smithi090 ceph-mon[27825]: from='mgr.14100 172.21.15.90:0/2265659422' entity='mgr.x' 2024-04-24T09:10:08.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:07 smithi090 ceph-mon[27825]: from='mgr.14100 172.21.15.90:0/2265659422' entity='mgr.x' 2024-04-24T09:10:10.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:09 smithi090 ceph-mon[27825]: mgrmap e3: x(active, since 1.01703s) 2024-04-24T09:10:10.972 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:10 smithi090 ceph-mon[27825]: mgrmap e4: x(active, since 2s) 2024-04-24T09:10:10.972 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:10 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/3843491658' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-24T09:10:10.979 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-24T09:10:10.982 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-04-24T09:10:10.982 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsid": "30ee9be2-021a-11ef-bc93-c7b262605968", 2024-04-24T09:10:10.982 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "health": { 2024-04-24T09:10:10.982 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-24T09:10:10.982 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-24T09:10:10.983 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-24T09:10:10.983 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T09:10:10.983 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-24T09:10:10.983 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-24T09:10:10.983 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 0 2024-04-24T09:10:10.983 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-04-24T09:10:10.983 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-24T09:10:10.983 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "a" 2024-04-24T09:10:10.983 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-04-24T09:10:10.983 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-04-24T09:10:10.983 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-24T09:10:10.983 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T09:10:10.983 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-24T09:10:10.983 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-24T09:10:10.983 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T09:10:10.983 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-24T09:10:10.984 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T09:10:10.984 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-24T09:10:10.984 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-24T09:10:10.984 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-24T09:10:10.993 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-24T09:10:10.993 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-24T09:10:10.993 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-24T09:10:10.993 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T09:10:10.993 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-24T09:10:10.993 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-24T09:10:10.993 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-24T09:10:10.993 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-24T09:10:10.993 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-24T09:10:10.993 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-24T09:10:10.993 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-24T09:10:10.994 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-24T09:10:10.994 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-24T09:10:10.994 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T09:10:10.994 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-24T09:10:10.994 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T09:10:10.994 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-24T09:10:10.994 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-24T09:10:10.994 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T09:10:10.994 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-24T09:10:10.994 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-24T09:10:10.994 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-24T09:10:10.994 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-24T09:10:10.994 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-24T09:10:10.994 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-24T09:10:10.994 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "restful" 2024-04-24T09:10:10.995 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-04-24T09:10:10.995 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T09:10:10.995 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T09:10:10.995 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-24T09:10:10.995 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T09:10:10.995 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modified": "2024-04-24T09:09:49.496572+0000", 2024-04-24T09:10:10.995 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T09:10:10.995 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-24T09:10:10.995 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-24T09:10:10.995 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-04-24T09:10:10.995 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:09.931+0000 7f19a5000700 1 Processor -- start 2024-04-24T09:10:10.995 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:09.932+0000 7f19a5000700 1 -- start start 2024-04-24T09:10:10.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:09.932+0000 7f19a5000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f19a010a840 0x7f19a010ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:10.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:09.932+0000 7f19a5000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f19a010b230 con 0x7f19a010a840 2024-04-24T09:10:10.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:09.932+0000 7f199de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f19a010a840 0x7f19a010ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:10.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:09.932+0000 7f199de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f19a010a840 0x7f19a010ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57888/0 (socket says 172.21.15.90:57888) 2024-04-24T09:10:10.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:09.932+0000 7f199de00700 1 -- 172.21.15.90:0/4043020975 learned_addr learned my addr 172.21.15.90:0/4043020975 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:10:10.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:09.933+0000 7f199de00700 1 -- 172.21.15.90:0/4043020975 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19a010b9f0 con 0x7f19a010a840 2024-04-24T09:10:10.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:09.933+0000 7f199de00700 1 --2- 172.21.15.90:0/4043020975 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f19a010a840 0x7f19a010ac60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f19900098a0 tx=0x7f1990009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b4c38f5e52eb6c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:10.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:09.933+0000 7f199d400700 1 -- 172.21.15.90:0/4043020975 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1990004160 con 0x7f19a010a840 2024-04-24T09:10:10.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:09.933+0000 7f199d400700 1 -- 172.21.15.90:0/4043020975 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1990004300 con 0x7f19a010a840 2024-04-24T09:10:10.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:09.934+0000 7f199d400700 1 -- 172.21.15.90:0/4043020975 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1990011b50 con 0x7f19a010a840 2024-04-24T09:10:10.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:09.934+0000 7f19a5000700 1 -- 172.21.15.90:0/4043020975 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f19a010a840 msgr2=0x7f19a010ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:10.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:09.934+0000 7f19a5000700 1 --2- 172.21.15.90:0/4043020975 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f19a010a840 0x7f19a010ac60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f19900098a0 tx=0x7f1990009bb0 comp rx=0 tx=0).stop 2024-04-24T09:10:10.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:09.934+0000 7f19a5000700 1 -- 172.21.15.90:0/4043020975 shutdown_connections 2024-04-24T09:10:10.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:09.934+0000 7f19a5000700 1 --2- 172.21.15.90:0/4043020975 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f19a010a840 0x7f19a010ac60 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:10.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:09.934+0000 7f19a5000700 1 -- 172.21.15.90:0/4043020975 >> 172.21.15.90:0/4043020975 conn(0x7f19a007de00 msgr2=0x7f19a01084c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:10.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:09.934+0000 7f19a5000700 1 -- 172.21.15.90:0/4043020975 shutdown_connections 2024-04-24T09:10:10.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:09.934+0000 7f19a5000700 1 -- 172.21.15.90:0/4043020975 wait complete. 2024-04-24T09:10:10.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:09.935+0000 7f19a5000700 1 Processor -- start 2024-04-24T09:10:10.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:09.936+0000 7f19a5000700 1 -- start start 2024-04-24T09:10:10.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:09.936+0000 7f19a5000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f19a010a840 0x7f19a019e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:10.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:09.936+0000 7f19a5000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f19a010eda0 con 0x7f19a010a840 2024-04-24T09:10:10.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:09.936+0000 7f199de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f19a010a840 0x7f19a019e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:10.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:09.936+0000 7f199de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f19a010a840 0x7f19a019e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57902/0 (socket says 172.21.15.90:57902) 2024-04-24T09:10:10.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:09.936+0000 7f199de00700 1 -- 172.21.15.90:0/3843491658 learned_addr learned my addr 172.21.15.90:0/3843491658 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:10:10.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:09.937+0000 7f199de00700 1 -- 172.21.15.90:0/3843491658 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19a019ec50 con 0x7f19a010a840 2024-04-24T09:10:10.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:09.937+0000 7f199de00700 1 --2- 172.21.15.90:0/3843491658 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f19a010a840 0x7f19a019e710 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1990000c00 tx=0x7f1990011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:10.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:09.937+0000 7f1996a00700 1 -- 172.21.15.90:0/3843491658 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1990003b20 con 0x7f19a010a840 2024-04-24T09:10:10.998 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:09.937+0000 7f1996a00700 1 -- 172.21.15.90:0/3843491658 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1990003cc0 con 0x7f19a010a840 2024-04-24T09:10:10.998 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:09.937+0000 7f1996a00700 1 -- 172.21.15.90:0/3843491658 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1990024910 con 0x7f19a010a840 2024-04-24T09:10:10.998 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:09.937+0000 7f19a5000700 1 -- 172.21.15.90:0/3843491658 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19a019eee0 con 0x7f19a010a840 2024-04-24T09:10:10.998 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:09.937+0000 7f19a5000700 1 -- 172.21.15.90:0/3843491658 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19a01a1bd0 con 0x7f19a010a840 2024-04-24T09:10:10.998 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:09.938+0000 7f1996a00700 1 -- 172.21.15.90:0/3843491658 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f1990022020 con 0x7f19a010a840 2024-04-24T09:10:10.998 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:09.938+0000 7f19a5000700 1 -- 172.21.15.90:0/3843491658 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19a0064150 con 0x7f19a010a840 2024-04-24T09:10:10.998 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:09.939+0000 7f1996a00700 1 --2- 172.21.15.90:0/3843491658 >> [v2:172.21.15.90:6800/3089140842,v1:172.21.15.90:6801/3089140842] conn(0x7f1984039560 0x7f198403ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:10.998 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:09.939+0000 7f1996a00700 1 -- 172.21.15.90:0/3843491658 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1990011e00 con 0x7f19a010a840 2024-04-24T09:10:10.998 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:09.939+0000 7f1997e00700 1 --2- 172.21.15.90:0/3843491658 >> [v2:172.21.15.90:6800/3089140842,v1:172.21.15.90:6801/3089140842] conn(0x7f1984039560 0x7f198403ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:10.998 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:09.940+0000 7f1997e00700 1 --2- 172.21.15.90:0/3843491658 >> [v2:172.21.15.90:6800/3089140842,v1:172.21.15.90:6801/3089140842] conn(0x7f1984039560 0x7f198403ba20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1988006fd0 tx=0x7f1988006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:10.998 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:09.945+0000 7f1996a00700 1 -- 172.21.15.90:0/3843491658 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f199001ad80 con 0x7f19a010a840 2024-04-24T09:10:10.998 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:10.258+0000 7f19a5000700 1 -- 172.21.15.90:0/3843491658 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f19a01a1da0 con 0x7f19a010a840 2024-04-24T09:10:10.998 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:10.259+0000 7f1996a00700 1 -- 172.21.15.90:0/3843491658 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f19a01a1da0 con 0x7f19a010a840 2024-04-24T09:10:10.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:10.263+0000 7f19a5000700 1 -- 172.21.15.90:0/3843491658 >> [v2:172.21.15.90:6800/3089140842,v1:172.21.15.90:6801/3089140842] conn(0x7f1984039560 msgr2=0x7f198403ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:10.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:10.263+0000 7f19a5000700 1 --2- 172.21.15.90:0/3843491658 >> [v2:172.21.15.90:6800/3089140842,v1:172.21.15.90:6801/3089140842] conn(0x7f1984039560 0x7f198403ba20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1988006fd0 tx=0x7f1988006e40 comp rx=0 tx=0).stop 2024-04-24T09:10:10.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:10.263+0000 7f19a5000700 1 -- 172.21.15.90:0/3843491658 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f19a010a840 msgr2=0x7f19a019e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:10.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:10.263+0000 7f19a5000700 1 --2- 172.21.15.90:0/3843491658 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f19a010a840 0x7f19a019e710 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1990000c00 tx=0x7f1990011c10 comp rx=0 tx=0).stop 2024-04-24T09:10:10.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:10.263+0000 7f19a5000700 1 -- 172.21.15.90:0/3843491658 shutdown_connections 2024-04-24T09:10:10.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:10.263+0000 7f19a5000700 1 --2- 172.21.15.90:0/3843491658 >> [v2:172.21.15.90:6800/3089140842,v1:172.21.15.90:6801/3089140842] conn(0x7f1984039560 0x7f198403ba20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:10.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:10.263+0000 7f19a5000700 1 --2- 172.21.15.90:0/3843491658 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f19a010a840 0x7f19a019e710 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:10.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:10.263+0000 7f19a5000700 1 -- 172.21.15.90:0/3843491658 >> 172.21.15.90:0/3843491658 conn(0x7f19a007de00 msgr2=0x7f19a0107c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:10.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:10.263+0000 7f19a5000700 1 -- 172.21.15.90:0/3843491658 shutdown_connections 2024-04-24T09:10:10.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:10.263+0000 7f19a5000700 1 -- 172.21.15.90:0/3843491658 wait complete. 2024-04-24T09:10:10.999 INFO:teuthology.orchestra.run.smithi090.stdout:mgr is available 2024-04-24T09:10:12.479 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:12 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/2808183540' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-24T09:10:12.810 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-24T09:10:12.810 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout [global] 2024-04-24T09:10:12.810 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout fsid = 30ee9be2-021a-11ef-bc93-c7b262605968 2024-04-24T09:10:12.810 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-24T09:10:12.810 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-24T09:10:12.810 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-24T09:10:12.810 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-24T09:10:12.810 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-24T09:10:12.810 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-24T09:10:12.810 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-24T09:10:12.811 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-24T09:10:12.811 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout [osd] 2024-04-24T09:10:12.811 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-24T09:10:12.811 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-24T09:10:12.811 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-24T09:10:12.816 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-24T09:10:12.816 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-24T09:10:12.816 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:11.870+0000 7f05f9800700 1 Processor -- start 2024-04-24T09:10:12.816 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:11.871+0000 7f05f9800700 1 -- start start 2024-04-24T09:10:12.816 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:11.871+0000 7f05f9800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05f407de20 0x7f05f407c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:12.816 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:11.871+0000 7f05f9800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f05f407c810 con 0x7f05f407de20 2024-04-24T09:10:12.817 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:11.871+0000 7f05f3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05f407de20 0x7f05f407c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:12.817 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:11.871+0000 7f05f3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05f407de20 0x7f05f407c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57914/0 (socket says 172.21.15.90:57914) 2024-04-24T09:10:12.817 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:11.871+0000 7f05f3e00700 1 -- 172.21.15.90:0/1717149796 learned_addr learned my addr 172.21.15.90:0/1717149796 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:10:12.817 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:11.872+0000 7f05f3e00700 1 -- 172.21.15.90:0/1717149796 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05f407c990 con 0x7f05f407de20 2024-04-24T09:10:12.817 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:11.872+0000 7f05f3e00700 1 --2- 172.21.15.90:0/1717149796 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05f407de20 0x7f05f407c2d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f05e40098a0 tx=0x7f05e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47819db0ff2a781c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:12.817 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:11.872+0000 7f05f2a00700 1 -- 172.21.15.90:0/1717149796 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05e4004160 con 0x7f05f407de20 2024-04-24T09:10:12.817 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:11.872+0000 7f05f2a00700 1 -- 172.21.15.90:0/1717149796 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f05e4004300 con 0x7f05f407de20 2024-04-24T09:10:12.817 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:11.872+0000 7f05f2a00700 1 -- 172.21.15.90:0/1717149796 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05e4011b50 con 0x7f05f407de20 2024-04-24T09:10:12.817 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:11.873+0000 7f05f9800700 1 -- 172.21.15.90:0/1717149796 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05f407de20 msgr2=0x7f05f407c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:12.817 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:11.873+0000 7f05f9800700 1 --2- 172.21.15.90:0/1717149796 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05f407de20 0x7f05f407c2d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f05e40098a0 tx=0x7f05e4009bb0 comp rx=0 tx=0).stop 2024-04-24T09:10:12.817 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:11.873+0000 7f05f9800700 1 -- 172.21.15.90:0/1717149796 shutdown_connections 2024-04-24T09:10:12.817 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:11.873+0000 7f05f9800700 1 --2- 172.21.15.90:0/1717149796 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05f407de20 0x7f05f407c2d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:12.817 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:11.873+0000 7f05f9800700 1 -- 172.21.15.90:0/1717149796 >> 172.21.15.90:0/1717149796 conn(0x7f05f4103a60 msgr2=0x7f05f4105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:12.817 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:11.873+0000 7f05f9800700 1 -- 172.21.15.90:0/1717149796 shutdown_connections 2024-04-24T09:10:12.818 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:11.873+0000 7f05f9800700 1 -- 172.21.15.90:0/1717149796 wait complete. 2024-04-24T09:10:12.818 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:11.874+0000 7f05f9800700 1 Processor -- start 2024-04-24T09:10:12.818 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:11.874+0000 7f05f9800700 1 -- start start 2024-04-24T09:10:12.818 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:11.875+0000 7f05f9800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05f407de20 0x7f05f419e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:12.818 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:11.875+0000 7f05f9800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f05f410e9f0 con 0x7f05f407de20 2024-04-24T09:10:12.818 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:11.875+0000 7f05f3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05f407de20 0x7f05f419e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:12.818 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:11.875+0000 7f05f3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05f407de20 0x7f05f419e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57922/0 (socket says 172.21.15.90:57922) 2024-04-24T09:10:12.818 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:11.875+0000 7f05f3e00700 1 -- 172.21.15.90:0/2808183540 learned_addr learned my addr 172.21.15.90:0/2808183540 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:10:12.818 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:11.875+0000 7f05f3e00700 1 -- 172.21.15.90:0/2808183540 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05f419eb30 con 0x7f05f407de20 2024-04-24T09:10:12.818 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:11.876+0000 7f05f3e00700 1 --2- 172.21.15.90:0/2808183540 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05f407de20 0x7f05f419e5f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f05e4011b60 tx=0x7f05e4011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:12.818 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:11.876+0000 7f05f0c00700 1 -- 172.21.15.90:0/2808183540 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05e4003970 con 0x7f05f407de20 2024-04-24T09:10:12.818 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:11.876+0000 7f05f0c00700 1 -- 172.21.15.90:0/2808183540 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f05e4003b10 con 0x7f05f407de20 2024-04-24T09:10:12.818 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:11.876+0000 7f05f0c00700 1 -- 172.21.15.90:0/2808183540 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05e4024610 con 0x7f05f407de20 2024-04-24T09:10:12.819 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:11.876+0000 7f05f9800700 1 -- 172.21.15.90:0/2808183540 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05f419edc0 con 0x7f05f407de20 2024-04-24T09:10:12.819 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:11.876+0000 7f05f9800700 1 -- 172.21.15.90:0/2808183540 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05f41a1ab0 con 0x7f05f407de20 2024-04-24T09:10:12.819 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:11.877+0000 7f05f0c00700 1 -- 172.21.15.90:0/2808183540 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f05e401a9f0 con 0x7f05f407de20 2024-04-24T09:10:12.819 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:11.877+0000 7f05f9800700 1 -- 172.21.15.90:0/2808183540 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05f4064150 con 0x7f05f407de20 2024-04-24T09:10:12.819 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:11.878+0000 7f05f0c00700 1 --2- 172.21.15.90:0/2808183540 >> [v2:172.21.15.90:6800/3089140842,v1:172.21.15.90:6801/3089140842] conn(0x7f05d80395b0 0x7f05d803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:12.819 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:11.878+0000 7f05f0c00700 1 -- 172.21.15.90:0/2808183540 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f05e4011e00 con 0x7f05f407de20 2024-04-24T09:10:12.819 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:11.878+0000 7f05f3400700 1 --2- 172.21.15.90:0/2808183540 >> [v2:172.21.15.90:6800/3089140842,v1:172.21.15.90:6801/3089140842] conn(0x7f05d80395b0 0x7f05d803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:12.819 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:11.879+0000 7f05f3400700 1 --2- 172.21.15.90:0/2808183540 >> [v2:172.21.15.90:6800/3089140842,v1:172.21.15.90:6801/3089140842] conn(0x7f05d80395b0 0x7f05d803ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f05e0006fd0 tx=0x7f05e0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:12.819 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:11.883+0000 7f05f0c00700 1 -- 172.21.15.90:0/2808183540 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f05e400b3e0 con 0x7f05f407de20 2024-04-24T09:10:12.819 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:12.108+0000 7f05f9800700 1 -- 172.21.15.90:0/2808183540 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f05f41a1c80 con 0x7f05f407de20 2024-04-24T09:10:12.819 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:12.109+0000 7f05f0c00700 1 -- 172.21.15.90:0/2808183540 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7f05f41a1c80 con 0x7f05f407de20 2024-04-24T09:10:12.819 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:12.112+0000 7f05f9800700 1 -- 172.21.15.90:0/2808183540 >> [v2:172.21.15.90:6800/3089140842,v1:172.21.15.90:6801/3089140842] conn(0x7f05d80395b0 msgr2=0x7f05d803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:12.819 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:12.112+0000 7f05f9800700 1 --2- 172.21.15.90:0/2808183540 >> [v2:172.21.15.90:6800/3089140842,v1:172.21.15.90:6801/3089140842] conn(0x7f05d80395b0 0x7f05d803ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f05e0006fd0 tx=0x7f05e0006e40 comp rx=0 tx=0).stop 2024-04-24T09:10:12.819 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:12.112+0000 7f05f9800700 1 -- 172.21.15.90:0/2808183540 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05f407de20 msgr2=0x7f05f419e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:12.820 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:12.112+0000 7f05f9800700 1 --2- 172.21.15.90:0/2808183540 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05f407de20 0x7f05f419e5f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f05e4011b60 tx=0x7f05e4011c40 comp rx=0 tx=0).stop 2024-04-24T09:10:12.820 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:12.112+0000 7f05f9800700 1 -- 172.21.15.90:0/2808183540 shutdown_connections 2024-04-24T09:10:12.820 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:12.112+0000 7f05f9800700 1 --2- 172.21.15.90:0/2808183540 >> [v2:172.21.15.90:6800/3089140842,v1:172.21.15.90:6801/3089140842] conn(0x7f05d80395b0 0x7f05d803ba70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:12.820 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:12.112+0000 7f05f9800700 1 --2- 172.21.15.90:0/2808183540 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05f407de20 0x7f05f419e5f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:12.820 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:12.112+0000 7f05f9800700 1 -- 172.21.15.90:0/2808183540 >> 172.21.15.90:0/2808183540 conn(0x7f05f4103a60 msgr2=0x7f05f4104650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:12.820 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:12.112+0000 7f05f9800700 1 -- 172.21.15.90:0/2808183540 shutdown_connections 2024-04-24T09:10:12.820 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:12.113+0000 7f05f9800700 1 -- 172.21.15.90:0/2808183540 wait complete. 2024-04-24T09:10:12.820 INFO:teuthology.orchestra.run.smithi090.stdout:Enabling cephadm module... 2024-04-24T09:10:14.034 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:13 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/364858316' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-24T09:10:14.035 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:13 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: ignoring --setuser ceph since I am not root 2024-04-24T09:10:14.035 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:13 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: ignoring --setgroup ceph since I am not root 2024-04-24T09:10:14.035 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:14 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:14.034+0000 7f220c4b5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T09:10:14.352 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:14 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:14.270+0000 7f220c4b5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T09:10:14.583 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.408+0000 7fa8e3800700 1 Processor -- start 2024-04-24T09:10:14.583 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.408+0000 7fa8e3800700 1 -- start start 2024-04-24T09:10:14.583 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.409+0000 7fa8e3800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8dc108600 0x7fa8dc108a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:14.583 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.409+0000 7fa8e3800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa8dc108ff0 con 0x7fa8dc108600 2024-04-24T09:10:14.583 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.409+0000 7fa8e0e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8dc108600 0x7fa8dc108a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:14.584 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.409+0000 7fa8e0e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8dc108600 0x7fa8dc108a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57934/0 (socket says 172.21.15.90:57934) 2024-04-24T09:10:14.584 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.409+0000 7fa8e0e00700 1 -- 172.21.15.90:0/2972935605 learned_addr learned my addr 172.21.15.90:0/2972935605 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:10:14.584 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.410+0000 7fa8e0e00700 1 -- 172.21.15.90:0/2972935605 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8dc1097b0 con 0x7fa8dc108600 2024-04-24T09:10:14.584 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.410+0000 7fa8e0e00700 1 --2- 172.21.15.90:0/2972935605 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8dc108600 0x7fa8dc108a20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa8d00098a0 tx=0x7fa8d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60700ed937ff00e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:14.584 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.410+0000 7fa8db400700 1 -- 172.21.15.90:0/2972935605 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8d0004160 con 0x7fa8dc108600 2024-04-24T09:10:14.584 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.410+0000 7fa8db400700 1 -- 172.21.15.90:0/2972935605 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa8d0004300 con 0x7fa8dc108600 2024-04-24T09:10:14.584 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.410+0000 7fa8db400700 1 -- 172.21.15.90:0/2972935605 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8d0011b50 con 0x7fa8dc108600 2024-04-24T09:10:14.584 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.411+0000 7fa8e3800700 1 -- 172.21.15.90:0/2972935605 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8dc108600 msgr2=0x7fa8dc108a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:14.584 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.411+0000 7fa8e3800700 1 --2- 172.21.15.90:0/2972935605 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8dc108600 0x7fa8dc108a20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa8d00098a0 tx=0x7fa8d0009bb0 comp rx=0 tx=0).stop 2024-04-24T09:10:14.585 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.411+0000 7fa8e3800700 1 -- 172.21.15.90:0/2972935605 shutdown_connections 2024-04-24T09:10:14.585 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.411+0000 7fa8e3800700 1 --2- 172.21.15.90:0/2972935605 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8dc108600 0x7fa8dc108a20 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:14.585 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.411+0000 7fa8e3800700 1 -- 172.21.15.90:0/2972935605 >> 172.21.15.90:0/2972935605 conn(0x7fa8dc103c80 msgr2=0x7fa8dc1060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:14.585 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.411+0000 7fa8e3800700 1 -- 172.21.15.90:0/2972935605 shutdown_connections 2024-04-24T09:10:14.585 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.411+0000 7fa8e3800700 1 -- 172.21.15.90:0/2972935605 wait complete. 2024-04-24T09:10:14.585 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.412+0000 7fa8e3800700 1 Processor -- start 2024-04-24T09:10:14.585 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.412+0000 7fa8e3800700 1 -- start start 2024-04-24T09:10:14.585 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.413+0000 7fa8e3800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8dc108600 0x7fa8dc19c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:14.585 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.413+0000 7fa8e3800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa8dc10cb60 con 0x7fa8dc108600 2024-04-24T09:10:14.585 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.413+0000 7fa8e0e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8dc108600 0x7fa8dc19c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:14.585 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.413+0000 7fa8e0e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8dc108600 0x7fa8dc19c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57938/0 (socket says 172.21.15.90:57938) 2024-04-24T09:10:14.585 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.413+0000 7fa8e0e00700 1 -- 172.21.15.90:0/364858316 learned_addr learned my addr 172.21.15.90:0/364858316 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:10:14.586 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.413+0000 7fa8e0e00700 1 -- 172.21.15.90:0/364858316 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8dc19ca60 con 0x7fa8dc108600 2024-04-24T09:10:14.586 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.414+0000 7fa8e0e00700 1 --2- 172.21.15.90:0/364858316 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8dc108600 0x7fa8dc19c520 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0000c00 tx=0x7fa8d0011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:14.586 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.414+0000 7fa8d9600700 1 -- 172.21.15.90:0/364858316 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8d0003b20 con 0x7fa8dc108600 2024-04-24T09:10:14.586 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.414+0000 7fa8d9600700 1 -- 172.21.15.90:0/364858316 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa8d0003cc0 con 0x7fa8dc108600 2024-04-24T09:10:14.586 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.414+0000 7fa8d9600700 1 -- 172.21.15.90:0/364858316 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8d0024910 con 0x7fa8dc108600 2024-04-24T09:10:14.586 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.414+0000 7fa8e3800700 1 -- 172.21.15.90:0/364858316 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8dc19ccf0 con 0x7fa8dc108600 2024-04-24T09:10:14.586 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.414+0000 7fa8e3800700 1 -- 172.21.15.90:0/364858316 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8dc19f9e0 con 0x7fa8dc108600 2024-04-24T09:10:14.586 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.415+0000 7fa8d9600700 1 -- 172.21.15.90:0/364858316 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fa8d0022020 con 0x7fa8dc108600 2024-04-24T09:10:14.586 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.415+0000 7fa8e3800700 1 -- 172.21.15.90:0/364858316 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8dc064150 con 0x7fa8dc108600 2024-04-24T09:10:14.586 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.416+0000 7fa8d9600700 1 --2- 172.21.15.90:0/364858316 >> [v2:172.21.15.90:6800/3089140842,v1:172.21.15.90:6801/3089140842] conn(0x7fa8c40395b0 0x7fa8c403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:14.586 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.416+0000 7fa8d9600700 1 -- 172.21.15.90:0/364858316 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa8d0011e00 con 0x7fa8dc108600 2024-04-24T09:10:14.586 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.416+0000 7fa8dbe00700 1 --2- 172.21.15.90:0/364858316 >> [v2:172.21.15.90:6800/3089140842,v1:172.21.15.90:6801/3089140842] conn(0x7fa8c40395b0 0x7fa8c403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:14.586 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.417+0000 7fa8dbe00700 1 --2- 172.21.15.90:0/364858316 >> [v2:172.21.15.90:6800/3089140842,v1:172.21.15.90:6801/3089140842] conn(0x7fa8c40395b0 0x7fa8c403ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa8c8006fd0 tx=0x7fa8c8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:14.587 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.421+0000 7fa8d9600700 1 -- 172.21.15.90:0/364858316 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa8d004d0d0 con 0x7fa8dc108600 2024-04-24T09:10:14.587 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.706+0000 7fa8e3800700 1 -- 172.21.15.90:0/364858316 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fa8dc19fbb0 con 0x7fa8dc108600 2024-04-24T09:10:14.587 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.760+0000 7fa8d9600700 1 -- 172.21.15.90:0/364858316 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fa8dc19fbb0 con 0x7fa8dc108600 2024-04-24T09:10:14.587 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.763+0000 7fa8d9600700 1 -- 172.21.15.90:0/364858316 <== mon.0 v2:172.21.15.90:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fa8d001a430 con 0x7fa8dc108600 2024-04-24T09:10:14.587 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.764+0000 7fa8e3800700 1 -- 172.21.15.90:0/364858316 >> [v2:172.21.15.90:6800/3089140842,v1:172.21.15.90:6801/3089140842] conn(0x7fa8c40395b0 msgr2=0x7fa8c403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:14.587 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.764+0000 7fa8e3800700 1 --2- 172.21.15.90:0/364858316 >> [v2:172.21.15.90:6800/3089140842,v1:172.21.15.90:6801/3089140842] conn(0x7fa8c40395b0 0x7fa8c403ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa8c8006fd0 tx=0x7fa8c8006e40 comp rx=0 tx=0).stop 2024-04-24T09:10:14.587 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.765+0000 7fa8e3800700 1 -- 172.21.15.90:0/364858316 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8dc108600 msgr2=0x7fa8dc19c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:14.587 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.765+0000 7fa8e3800700 1 --2- 172.21.15.90:0/364858316 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8dc108600 0x7fa8dc19c520 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0000c00 tx=0x7fa8d0011c10 comp rx=0 tx=0).stop 2024-04-24T09:10:14.587 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.765+0000 7fa8e3800700 1 -- 172.21.15.90:0/364858316 shutdown_connections 2024-04-24T09:10:14.587 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.765+0000 7fa8e3800700 1 --2- 172.21.15.90:0/364858316 >> [v2:172.21.15.90:6800/3089140842,v1:172.21.15.90:6801/3089140842] conn(0x7fa8c40395b0 0x7fa8c403ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:14.587 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.765+0000 7fa8e3800700 1 --2- 172.21.15.90:0/364858316 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8dc108600 0x7fa8dc19c520 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:14.587 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.765+0000 7fa8e3800700 1 -- 172.21.15.90:0/364858316 >> 172.21.15.90:0/364858316 conn(0x7fa8dc103c80 msgr2=0x7fa8dc104740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:14.587 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.765+0000 7fa8e3800700 1 -- 172.21.15.90:0/364858316 shutdown_connections 2024-04-24T09:10:14.587 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:13.765+0000 7fa8e3800700 1 -- 172.21.15.90:0/364858316 wait complete. 2024-04-24T09:10:14.731 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:14 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:14.399+0000 7f220c4b5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T09:10:15.085 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:14 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/364858316' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-24T09:10:15.086 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:14 smithi090 ceph-mon[27825]: mgrmap e5: x(active, since 5s) 2024-04-24T09:10:15.086 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:14 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:14.803+0000 7f220c4b5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T09:10:15.086 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:14 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:14.919+0000 7f220c4b5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T09:10:16.021 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:15 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/262115873' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-24T09:10:16.257 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-04-24T09:10:16.258 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-24T09:10:16.258 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-24T09:10:16.258 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-24T09:10:16.258 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-24T09:10:16.258 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-04-24T09:10:16.258 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.239+0000 7fd140c00700 1 Processor -- start 2024-04-24T09:10:16.258 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.239+0000 7fd140c00700 1 -- start start 2024-04-24T09:10:16.259 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.240+0000 7fd140c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd13c107310 0x7fd13c107730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:16.262 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.240+0000 7fd140c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd13c107d00 con 0x7fd13c107310 2024-04-24T09:10:16.262 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.240+0000 7fd13b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd13c107310 0x7fd13c107730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:16.262 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.240+0000 7fd13b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd13c107310 0x7fd13c107730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57966/0 (socket says 172.21.15.90:57966) 2024-04-24T09:10:16.262 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.240+0000 7fd13b400700 1 -- 172.21.15.90:0/2237881779 learned_addr learned my addr 172.21.15.90:0/2237881779 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:10:16.262 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.241+0000 7fd13b400700 1 -- 172.21.15.90:0/2237881779 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd13c108510 con 0x7fd13c107310 2024-04-24T09:10:16.262 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.241+0000 7fd13b400700 1 --2- 172.21.15.90:0/2237881779 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd13c107310 0x7fd13c107730 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd12c0098a0 tx=0x7fd12c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f14d342c4207a404 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:16.262 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.241+0000 7fd13aa00700 1 -- 172.21.15.90:0/2237881779 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd12c004160 con 0x7fd13c107310 2024-04-24T09:10:16.262 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.241+0000 7fd13aa00700 1 -- 172.21.15.90:0/2237881779 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd12c004300 con 0x7fd13c107310 2024-04-24T09:10:16.262 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.241+0000 7fd13aa00700 1 -- 172.21.15.90:0/2237881779 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd12c011b50 con 0x7fd13c107310 2024-04-24T09:10:16.263 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.242+0000 7fd140c00700 1 -- 172.21.15.90:0/2237881779 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd13c107310 msgr2=0x7fd13c107730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:16.263 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.242+0000 7fd140c00700 1 --2- 172.21.15.90:0/2237881779 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd13c107310 0x7fd13c107730 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd12c0098a0 tx=0x7fd12c009bb0 comp rx=0 tx=0).stop 2024-04-24T09:10:16.263 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.242+0000 7fd140c00700 1 -- 172.21.15.90:0/2237881779 shutdown_connections 2024-04-24T09:10:16.263 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.242+0000 7fd140c00700 1 --2- 172.21.15.90:0/2237881779 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd13c107310 0x7fd13c107730 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:16.263 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.242+0000 7fd140c00700 1 -- 172.21.15.90:0/2237881779 >> 172.21.15.90:0/2237881779 conn(0x7fd13c1028d0 msgr2=0x7fd13c104cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:16.263 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.243+0000 7fd140c00700 1 -- 172.21.15.90:0/2237881779 shutdown_connections 2024-04-24T09:10:16.263 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.243+0000 7fd140c00700 1 -- 172.21.15.90:0/2237881779 wait complete. 2024-04-24T09:10:16.263 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.244+0000 7fd140c00700 1 Processor -- start 2024-04-24T09:10:16.263 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.244+0000 7fd140c00700 1 -- start start 2024-04-24T09:10:16.263 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.245+0000 7fd140c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd13c107310 0x7fd13c192470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:16.263 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.245+0000 7fd140c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd13c10b8c0 con 0x7fd13c107310 2024-04-24T09:10:16.263 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.245+0000 7fd13b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd13c107310 0x7fd13c192470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:16.263 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.245+0000 7fd13b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd13c107310 0x7fd13c192470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57982/0 (socket says 172.21.15.90:57982) 2024-04-24T09:10:16.263 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.245+0000 7fd13b400700 1 -- 172.21.15.90:0/262115873 learned_addr learned my addr 172.21.15.90:0/262115873 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:10:16.263 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.245+0000 7fd13b400700 1 -- 172.21.15.90:0/262115873 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd13c1929b0 con 0x7fd13c107310 2024-04-24T09:10:16.264 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.246+0000 7fd13b400700 1 --2- 172.21.15.90:0/262115873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd13c107310 0x7fd13c192470 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd12c009fb0 tx=0x7fd12c011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:16.264 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.246+0000 7fd138c00700 1 -- 172.21.15.90:0/262115873 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd12c0038b0 con 0x7fd13c107310 2024-04-24T09:10:16.264 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.246+0000 7fd138c00700 1 -- 172.21.15.90:0/262115873 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd12c011e50 con 0x7fd13c107310 2024-04-24T09:10:16.264 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.246+0000 7fd138c00700 1 -- 172.21.15.90:0/262115873 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd12c019550 con 0x7fd13c107310 2024-04-24T09:10:16.264 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.246+0000 7fd140c00700 1 -- 172.21.15.90:0/262115873 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd13c190bf0 con 0x7fd13c107310 2024-04-24T09:10:16.266 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.246+0000 7fd140c00700 1 -- 172.21.15.90:0/262115873 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd13c1910d0 con 0x7fd13c107310 2024-04-24T09:10:16.267 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.247+0000 7fd140c00700 1 -- 172.21.15.90:0/262115873 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd13c064150 con 0x7fd13c107310 2024-04-24T09:10:16.267 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.247+0000 7fd138c00700 1 -- 172.21.15.90:0/262115873 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fd12c003a50 con 0x7fd13c107310 2024-04-24T09:10:16.267 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.248+0000 7fd138c00700 1 --2- 172.21.15.90:0/262115873 >> [v2:172.21.15.90:6800/3089140842,v1:172.21.15.90:6801/3089140842] conn(0x7fd1200395b0 0x7fd12003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:16.267 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.248+0000 7fd138c00700 1 -- 172.21.15.90:0/262115873 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd12c04d9a0 con 0x7fd13c107310 2024-04-24T09:10:16.267 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.248+0000 7fd133e00700 1 -- 172.21.15.90:0/262115873 >> [v2:172.21.15.90:6800/3089140842,v1:172.21.15.90:6801/3089140842] conn(0x7fd1200395b0 msgr2=0x7fd12003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/3089140842 2024-04-24T09:10:16.267 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.248+0000 7fd133e00700 1 --2- 172.21.15.90:0/262115873 >> [v2:172.21.15.90:6800/3089140842,v1:172.21.15.90:6801/3089140842] conn(0x7fd1200395b0 0x7fd12003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T09:10:16.267 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.253+0000 7fd138c00700 1 -- 172.21.15.90:0/262115873 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd12c04d3c0 con 0x7fd13c107310 2024-04-24T09:10:16.267 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.449+0000 7fd133e00700 1 -- 172.21.15.90:0/262115873 >> [v2:172.21.15.90:6800/3089140842,v1:172.21.15.90:6801/3089140842] conn(0x7fd1200395b0 msgr2=0x7fd12003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/3089140842 2024-04-24T09:10:16.267 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.449+0000 7fd133e00700 1 --2- 172.21.15.90:0/262115873 >> [v2:172.21.15.90:6800/3089140842,v1:172.21.15.90:6801/3089140842] conn(0x7fd1200395b0 0x7fd12003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T09:10:16.267 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.577+0000 7fd140c00700 1 -- 172.21.15.90:0/262115873 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fd13c196280 con 0x7fd13c107310 2024-04-24T09:10:16.267 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.577+0000 7fd138c00700 1 -- 172.21.15.90:0/262115873 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fd12c00fda0 con 0x7fd13c107310 2024-04-24T09:10:16.267 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.580+0000 7fd140c00700 1 -- 172.21.15.90:0/262115873 >> [v2:172.21.15.90:6800/3089140842,v1:172.21.15.90:6801/3089140842] conn(0x7fd1200395b0 msgr2=0x7fd12003ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T09:10:16.267 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.580+0000 7fd140c00700 1 --2- 172.21.15.90:0/262115873 >> [v2:172.21.15.90:6800/3089140842,v1:172.21.15.90:6801/3089140842] conn(0x7fd1200395b0 0x7fd12003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:16.267 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.580+0000 7fd140c00700 1 -- 172.21.15.90:0/262115873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd13c107310 msgr2=0x7fd13c192470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:16.267 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.580+0000 7fd140c00700 1 --2- 172.21.15.90:0/262115873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd13c107310 0x7fd13c192470 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd12c009fb0 tx=0x7fd12c011bf0 comp rx=0 tx=0).stop 2024-04-24T09:10:16.268 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.580+0000 7fd140c00700 1 -- 172.21.15.90:0/262115873 shutdown_connections 2024-04-24T09:10:16.268 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.580+0000 7fd140c00700 1 --2- 172.21.15.90:0/262115873 >> [v2:172.21.15.90:6800/3089140842,v1:172.21.15.90:6801/3089140842] conn(0x7fd1200395b0 0x7fd12003ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:16.268 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.580+0000 7fd140c00700 1 --2- 172.21.15.90:0/262115873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd13c107310 0x7fd13c192470 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:16.268 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.580+0000 7fd140c00700 1 -- 172.21.15.90:0/262115873 >> 172.21.15.90:0/262115873 conn(0x7fd13c1028d0 msgr2=0x7fd13c103800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:16.268 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.580+0000 7fd140c00700 1 -- 172.21.15.90:0/262115873 shutdown_connections 2024-04-24T09:10:16.268 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:15.580+0000 7fd140c00700 1 -- 172.21.15.90:0/262115873 wait complete. 2024-04-24T09:10:16.268 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for the mgr to restart... 2024-04-24T09:10:16.268 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for mgr epoch 5... 2024-04-24T09:10:16.676 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:16 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:16.510+0000 7f220c4b5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T09:10:17.102 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:16 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:16.742+0000 7f220c4b5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T09:10:17.102 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:16 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:16.964+0000 7f220c4b5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T09:10:17.467 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:17 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:17.346+0000 7f220c4b5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T09:10:17.467 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:17 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:17.467+0000 7f220c4b5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T09:10:17.974 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:17 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:17.719+0000 7f220c4b5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T09:10:17.974 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:17 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:17.973+0000 7f220c4b5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T09:10:19.101 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:18 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:18.759+0000 7f220c4b5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T09:10:19.602 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:19 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:19.303+0000 7f220c4b5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T09:10:19.602 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:19 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:19.428+0000 7f220c4b5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T09:10:19.942 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:19 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:19.689+0000 7f220c4b5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T09:10:20.602 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:20 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:20.325+0000 7f220c4b5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T09:10:20.602 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:20 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:20.447+0000 7f220c4b5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T09:10:21.063 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:20 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:20.763+0000 7f220c4b5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T09:10:21.063 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:20 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:20.939+0000 7f220c4b5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T09:10:21.063 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:21 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:21.062+0000 7f220c4b5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T09:10:21.352 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:21 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:21.212+0000 7f220c4b5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T09:10:21.851 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:21 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:21.572+0000 7f220c4b5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T09:10:22.596 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:22 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:22.316+0000 7f220c4b5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T09:10:22.852 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:22 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:22.596+0000 7f220c4b5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T09:10:22.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:22 smithi090 ceph-mon[27825]: Active manager daemon x restarted 2024-04-24T09:10:22.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:22 smithi090 ceph-mon[27825]: Activating manager daemon x 2024-04-24T09:10:22.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:22 smithi090 ceph-mon[27825]: osdmap e2: 0 total, 0 up, 0 in 2024-04-24T09:10:22.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:22 smithi090 ceph-mon[27825]: mgrmap e6: x(active, starting, since 0.00887062s) 2024-04-24T09:10:22.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:22 smithi090 ceph-mon[27825]: from='mgr.14120 172.21.15.90:0/738983790' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T09:10:22.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:22 smithi090 ceph-mon[27825]: from='mgr.14120 172.21.15.90:0/738983790' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-24T09:10:22.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:22 smithi090 ceph-mon[27825]: from='mgr.14120 172.21.15.90:0/738983790' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T09:10:22.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:22 smithi090 ceph-mon[27825]: from='mgr.14120 172.21.15.90:0/738983790' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T09:10:22.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:22 smithi090 ceph-mon[27825]: from='mgr.14120 172.21.15.90:0/738983790' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T09:10:22.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:22 smithi090 ceph-mon[27825]: Manager daemon x is now available 2024-04-24T09:10:24.036 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:23 smithi090 ceph-mon[27825]: Found migration_current of "None". Setting to last migration. 2024-04-24T09:10:24.036 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:23 smithi090 ceph-mon[27825]: from='mgr.14120 172.21.15.90:0/738983790' entity='mgr.x' 2024-04-24T09:10:24.036 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:23 smithi090 ceph-mon[27825]: from='mgr.14120 172.21.15.90:0/738983790' entity='mgr.x' 2024-04-24T09:10:24.036 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:23 smithi090 ceph-mon[27825]: from='mgr.14120 172.21.15.90:0/738983790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:10:24.036 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:23 smithi090 ceph-mon[27825]: from='mgr.14120 172.21.15.90:0/738983790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:10:24.036 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:23 smithi090 ceph-mon[27825]: from='mgr.14120 172.21.15.90:0/738983790' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T09:10:24.036 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:23 smithi090 ceph-mon[27825]: from='mgr.14120 172.21.15.90:0/738983790' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T09:10:24.036 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:23 smithi090 ceph-mon[27825]: from='mgr.14120 172.21.15.90:0/738983790' entity='mgr.x' 2024-04-24T09:10:24.036 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:23 smithi090 ceph-mon[27825]: from='mgr.14120 172.21.15.90:0/738983790' entity='mgr.x' 2024-04-24T09:10:24.036 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:23 smithi090 ceph-mon[27825]: mgrmap e7: x(active, since 1.012s) 2024-04-24T09:10:24.391 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-04-24T09:10:24.391 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-24T09:10:24.391 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-24T09:10:24.391 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-04-24T09:10:24.391 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:16.927+0000 7f4b64600700 1 Processor -- start 2024-04-24T09:10:24.395 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:16.927+0000 7f4b64600700 1 -- start start 2024-04-24T09:10:24.395 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:16.928+0000 7f4b64600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b5c07de00 0x7f4b5c07c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:24.395 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:16.928+0000 7f4b64600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5c07c7f0 con 0x7f4b5c07de00 2024-04-24T09:10:24.395 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:16.928+0000 7f4b61c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b5c07de00 0x7f4b5c07c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:24.395 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:16.928+0000 7f4b61c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b5c07de00 0x7f4b5c07c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43986/0 (socket says 172.21.15.90:43986) 2024-04-24T09:10:24.395 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:16.928+0000 7f4b61c00700 1 -- 172.21.15.90:0/929523935 learned_addr learned my addr 172.21.15.90:0/929523935 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:10:24.395 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:16.929+0000 7f4b61c00700 1 -- 172.21.15.90:0/929523935 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b5c07c970 con 0x7f4b5c07de00 2024-04-24T09:10:24.395 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:16.929+0000 7f4b61c00700 1 --2- 172.21.15.90:0/929523935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b5c07de00 0x7f4b5c07c2b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4b500098a0 tx=0x7f4b50009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40bd72b828789a31 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:24.395 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:16.929+0000 7f4b5be00700 1 -- 172.21.15.90:0/929523935 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b50004160 con 0x7f4b5c07de00 2024-04-24T09:10:24.395 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:16.929+0000 7f4b5be00700 1 -- 172.21.15.90:0/929523935 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4b50004300 con 0x7f4b5c07de00 2024-04-24T09:10:24.395 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:16.929+0000 7f4b5be00700 1 -- 172.21.15.90:0/929523935 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b50011b50 con 0x7f4b5c07de00 2024-04-24T09:10:24.395 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:16.930+0000 7f4b64600700 1 -- 172.21.15.90:0/929523935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b5c07de00 msgr2=0x7f4b5c07c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:24.396 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:16.930+0000 7f4b64600700 1 --2- 172.21.15.90:0/929523935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b5c07de00 0x7f4b5c07c2b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4b500098a0 tx=0x7f4b50009bb0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.396 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:16.930+0000 7f4b64600700 1 -- 172.21.15.90:0/929523935 shutdown_connections 2024-04-24T09:10:24.396 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:16.930+0000 7f4b64600700 1 --2- 172.21.15.90:0/929523935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b5c07de00 0x7f4b5c07c2b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.396 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:16.930+0000 7f4b64600700 1 -- 172.21.15.90:0/929523935 >> 172.21.15.90:0/929523935 conn(0x7f4b5c103c10 msgr2=0x7f4b5c106030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:24.396 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:16.930+0000 7f4b64600700 1 -- 172.21.15.90:0/929523935 shutdown_connections 2024-04-24T09:10:24.396 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:16.930+0000 7f4b64600700 1 -- 172.21.15.90:0/929523935 wait complete. 2024-04-24T09:10:24.396 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:16.932+0000 7f4b64600700 1 Processor -- start 2024-04-24T09:10:24.396 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:16.932+0000 7f4b64600700 1 -- start start 2024-04-24T09:10:24.396 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:16.932+0000 7f4b64600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b5c07de00 0x7f4b5c19e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:24.396 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:16.932+0000 7f4b64600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5c10ec00 con 0x7f4b5c07de00 2024-04-24T09:10:24.396 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:16.932+0000 7f4b61c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b5c07de00 0x7f4b5c19e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:24.396 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:16.933+0000 7f4b61c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b5c07de00 0x7f4b5c19e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44000/0 (socket says 172.21.15.90:44000) 2024-04-24T09:10:24.396 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:16.933+0000 7f4b61c00700 1 -- 172.21.15.90:0/3517717387 learned_addr learned my addr 172.21.15.90:0/3517717387 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:10:24.396 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:16.933+0000 7f4b61c00700 1 -- 172.21.15.90:0/3517717387 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b5c19ecc0 con 0x7f4b5c07de00 2024-04-24T09:10:24.397 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:16.933+0000 7f4b61c00700 1 --2- 172.21.15.90:0/3517717387 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b5c07de00 0x7f4b5c19e780 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4b50011b60 tx=0x7f4b50011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:24.397 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:16.933+0000 7f4b5a000700 1 -- 172.21.15.90:0/3517717387 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b50003970 con 0x7f4b5c07de00 2024-04-24T09:10:24.397 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:16.933+0000 7f4b5a000700 1 -- 172.21.15.90:0/3517717387 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4b50003b10 con 0x7f4b5c07de00 2024-04-24T09:10:24.397 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:16.933+0000 7f4b5a000700 1 -- 172.21.15.90:0/3517717387 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b50024610 con 0x7f4b5c07de00 2024-04-24T09:10:24.397 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:16.933+0000 7f4b64600700 1 -- 172.21.15.90:0/3517717387 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b5c19ef50 con 0x7f4b5c07de00 2024-04-24T09:10:24.401 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:16.934+0000 7f4b64600700 1 -- 172.21.15.90:0/3517717387 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b5c1a1c40 con 0x7f4b5c07de00 2024-04-24T09:10:24.401 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:16.935+0000 7f4b5a000700 1 -- 172.21.15.90:0/3517717387 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f4b5001a9f0 con 0x7f4b5c07de00 2024-04-24T09:10:24.401 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:16.936+0000 7f4b5a000700 1 --2- 172.21.15.90:0/3517717387 >> [v2:172.21.15.90:6800/3089140842,v1:172.21.15.90:6801/3089140842] conn(0x7f4b44039600 0x7f4b4403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:24.401 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:16.936+0000 7f4b5a000700 1 -- 172.21.15.90:0/3517717387 --> [v2:172.21.15.90:6800/3089140842,v1:172.21.15.90:6801/3089140842] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4b4403c1d0 con 0x7f4b44039600 2024-04-24T09:10:24.401 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:16.936+0000 7f4b5a000700 1 -- 172.21.15.90:0/3517717387 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4b50011e00 con 0x7f4b5c07de00 2024-04-24T09:10:24.401 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:16.936+0000 7f4b61200700 1 -- 172.21.15.90:0/3517717387 >> [v2:172.21.15.90:6800/3089140842,v1:172.21.15.90:6801/3089140842] conn(0x7f4b44039600 msgr2=0x7f4b4403bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/3089140842 2024-04-24T09:10:24.401 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:16.936+0000 7f4b61200700 1 --2- 172.21.15.90:0/3517717387 >> [v2:172.21.15.90:6800/3089140842,v1:172.21.15.90:6801/3089140842] conn(0x7f4b44039600 0x7f4b4403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T09:10:24.401 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:17.137+0000 7f4b61200700 1 -- 172.21.15.90:0/3517717387 >> [v2:172.21.15.90:6800/3089140842,v1:172.21.15.90:6801/3089140842] conn(0x7f4b44039600 msgr2=0x7f4b4403bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/3089140842 2024-04-24T09:10:24.401 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:17.137+0000 7f4b61200700 1 --2- 172.21.15.90:0/3517717387 >> [v2:172.21.15.90:6800/3089140842,v1:172.21.15.90:6801/3089140842] conn(0x7f4b44039600 0x7f4b4403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T09:10:24.401 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:17.538+0000 7f4b61200700 1 -- 172.21.15.90:0/3517717387 >> [v2:172.21.15.90:6800/3089140842,v1:172.21.15.90:6801/3089140842] conn(0x7f4b44039600 msgr2=0x7f4b4403bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/3089140842 2024-04-24T09:10:24.402 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:17.538+0000 7f4b61200700 1 --2- 172.21.15.90:0/3517717387 >> [v2:172.21.15.90:6800/3089140842,v1:172.21.15.90:6801/3089140842] conn(0x7f4b44039600 0x7f4b4403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-24T09:10:24.402 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:18.339+0000 7f4b61200700 1 -- 172.21.15.90:0/3517717387 >> [v2:172.21.15.90:6800/3089140842,v1:172.21.15.90:6801/3089140842] conn(0x7f4b44039600 msgr2=0x7f4b4403bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/3089140842 2024-04-24T09:10:24.402 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:18.339+0000 7f4b61200700 1 --2- 172.21.15.90:0/3517717387 >> [v2:172.21.15.90:6800/3089140842,v1:172.21.15.90:6801/3089140842] conn(0x7f4b44039600 0x7f4b4403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-24T09:10:24.402 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:19.941+0000 7f4b61200700 1 -- 172.21.15.90:0/3517717387 >> [v2:172.21.15.90:6800/3089140842,v1:172.21.15.90:6801/3089140842] conn(0x7f4b44039600 msgr2=0x7f4b4403bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/3089140842 2024-04-24T09:10:24.402 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:19.941+0000 7f4b61200700 1 --2- 172.21.15.90:0/3517717387 >> [v2:172.21.15.90:6800/3089140842,v1:172.21.15.90:6801/3089140842] conn(0x7f4b44039600 0x7f4b4403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-24T09:10:24.402 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:22.605+0000 7f4b5a000700 1 -- 172.21.15.90:0/3517717387 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7f4b5001a430 con 0x7f4b5c07de00 2024-04-24T09:10:24.402 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:22.605+0000 7f4b5a000700 1 -- 172.21.15.90:0/3517717387 >> [v2:172.21.15.90:6800/3089140842,v1:172.21.15.90:6801/3089140842] conn(0x7f4b44039600 msgr2=0x7f4b4403bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T09:10:24.402 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:22.605+0000 7f4b5a000700 1 --2- 172.21.15.90:0/3517717387 >> [v2:172.21.15.90:6800/3089140842,v1:172.21.15.90:6801/3089140842] conn(0x7f4b44039600 0x7f4b4403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.402 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:23.607+0000 7f4b5a000700 1 -- 172.21.15.90:0/3517717387 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f4b5004f370 con 0x7f4b5c07de00 2024-04-24T09:10:24.402 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:23.608+0000 7f4b5a000700 1 --2- 172.21.15.90:0/3517717387 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f4b4403cef0 0x7f4b4403f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:24.402 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:23.608+0000 7f4b5a000700 1 -- 172.21.15.90:0/3517717387 --> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4b4403c1d0 con 0x7f4b4403cef0 2024-04-24T09:10:24.402 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:23.609+0000 7f4b61200700 1 --2- 172.21.15.90:0/3517717387 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f4b4403cef0 0x7f4b4403f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:24.402 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:23.611+0000 7f4b61200700 1 --2- 172.21.15.90:0/3517717387 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f4b4403cef0 0x7f4b4403f2e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4b48003a10 tx=0x7f4b480092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:24.402 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:23.613+0000 7f4b5a000700 1 -- 172.21.15.90:0/3517717387 <== mgr.14120 v2:172.21.15.90:6800/1809038230 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f4b4403c1d0 con 0x7f4b4403cef0 2024-04-24T09:10:24.403 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:23.620+0000 7f4b64600700 1 -- 172.21.15.90:0/3517717387 --> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f4b5c064150 con 0x7f4b4403cef0 2024-04-24T09:10:24.403 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:23.620+0000 7f4b5a000700 1 -- 172.21.15.90:0/3517717387 <== mgr.14120 v2:172.21.15.90:6800/1809038230 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f4b5c064150 con 0x7f4b4403cef0 2024-04-24T09:10:24.403 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:23.621+0000 7f4b64600700 1 -- 172.21.15.90:0/3517717387 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f4b4403cef0 msgr2=0x7f4b4403f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:24.403 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:23.621+0000 7f4b64600700 1 --2- 172.21.15.90:0/3517717387 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f4b4403cef0 0x7f4b4403f2e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4b48003a10 tx=0x7f4b480092b0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.403 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:23.621+0000 7f4b64600700 1 -- 172.21.15.90:0/3517717387 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b5c07de00 msgr2=0x7f4b5c19e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:24.403 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:23.621+0000 7f4b64600700 1 --2- 172.21.15.90:0/3517717387 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b5c07de00 0x7f4b5c19e780 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4b50011b60 tx=0x7f4b50011c40 comp rx=0 tx=0).stop 2024-04-24T09:10:24.403 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:23.621+0000 7f4b64600700 1 -- 172.21.15.90:0/3517717387 shutdown_connections 2024-04-24T09:10:24.403 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:23.621+0000 7f4b64600700 1 --2- 172.21.15.90:0/3517717387 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b5c07de00 0x7f4b5c19e780 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.403 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:23.621+0000 7f4b64600700 1 --2- 172.21.15.90:0/3517717387 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f4b4403cef0 0x7f4b4403f2e0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.403 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:23.621+0000 7f4b64600700 1 --2- 172.21.15.90:0/3517717387 >> [v2:172.21.15.90:6800/3089140842,v1:172.21.15.90:6801/3089140842] conn(0x7f4b44039600 0x7f4b4403bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.403 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:23.621+0000 7f4b64600700 1 -- 172.21.15.90:0/3517717387 >> 172.21.15.90:0/3517717387 conn(0x7f4b5c103c10 msgr2=0x7f4b5c104740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:24.403 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:23.621+0000 7f4b64600700 1 -- 172.21.15.90:0/3517717387 shutdown_connections 2024-04-24T09:10:24.403 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:23.622+0000 7f4b64600700 1 -- 172.21.15.90:0/3517717387 wait complete. 2024-04-24T09:10:24.403 INFO:teuthology.orchestra.run.smithi090.stdout:mgr epoch 5 is available 2024-04-24T09:10:24.404 INFO:teuthology.orchestra.run.smithi090.stdout:Setting orchestrator backend to cephadm... 2024-04-24T09:10:24.986 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:24 smithi090 ceph-mon[27825]: [24/Apr/2024:09:10:23] ENGINE Bus STARTING 2024-04-24T09:10:24.986 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:24 smithi090 ceph-mon[27825]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-24T09:10:24.986 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:24 smithi090 ceph-mon[27825]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-24T09:10:24.986 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:24 smithi090 ceph-mon[27825]: [24/Apr/2024:09:10:23] ENGINE Serving on https://172.21.15.90:7150 2024-04-24T09:10:24.986 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:24 smithi090 ceph-mon[27825]: [24/Apr/2024:09:10:23] ENGINE Serving on http://172.21.15.90:8765 2024-04-24T09:10:24.986 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:24 smithi090 ceph-mon[27825]: [24/Apr/2024:09:10:23] ENGINE Bus STARTED 2024-04-24T09:10:24.986 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:24 smithi090 ceph-mon[27825]: from='mgr.14120 172.21.15.90:0/738983790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:10:26.003 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:25 smithi090 ceph-mon[27825]: mgrmap e8: x(active, since 2s) 2024-04-24T09:10:26.003 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:25 smithi090 ceph-mon[27825]: from='mgr.14120 172.21.15.90:0/738983790' entity='mgr.x' 2024-04-24T09:10:26.003 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:25 smithi090 ceph-mon[27825]: from='mgr.14120 172.21.15.90:0/738983790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:10:26.012 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.061+0000 7fd102a00700 1 Processor -- start 2024-04-24T09:10:26.013 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.061+0000 7fd102a00700 1 -- start start 2024-04-24T09:10:26.013 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.062+0000 7fd102a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0fc07de20 0x7fd0fc07c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:26.013 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.062+0000 7fd102a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd0fc07c810 con 0x7fd0fc07de20 2024-04-24T09:10:26.013 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.062+0000 7fd101600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0fc07de20 0x7fd0fc07c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:26.014 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.062+0000 7fd101600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0fc07de20 0x7fd0fc07c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44080/0 (socket says 172.21.15.90:44080) 2024-04-24T09:10:26.014 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.062+0000 7fd101600700 1 -- 172.21.15.90:0/1815515801 learned_addr learned my addr 172.21.15.90:0/1815515801 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:10:26.015 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.063+0000 7fd101600700 1 -- 172.21.15.90:0/1815515801 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0fc07c990 con 0x7fd0fc07de20 2024-04-24T09:10:26.015 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.063+0000 7fd101600700 1 --2- 172.21.15.90:0/1815515801 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0fc07de20 0x7fd0fc07c2d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd0f00098a0 tx=0x7fd0f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4647178b319a3ddd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:26.015 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.063+0000 7fd0fbe00700 1 -- 172.21.15.90:0/1815515801 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0f0004160 con 0x7fd0fc07de20 2024-04-24T09:10:26.015 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.063+0000 7fd0fbe00700 1 -- 172.21.15.90:0/1815515801 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd0f0004300 con 0x7fd0fc07de20 2024-04-24T09:10:26.015 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.063+0000 7fd0fbe00700 1 -- 172.21.15.90:0/1815515801 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0f0011b50 con 0x7fd0fc07de20 2024-04-24T09:10:26.015 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.064+0000 7fd102a00700 1 -- 172.21.15.90:0/1815515801 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0fc07de20 msgr2=0x7fd0fc07c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:26.016 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.064+0000 7fd102a00700 1 --2- 172.21.15.90:0/1815515801 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0fc07de20 0x7fd0fc07c2d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd0f00098a0 tx=0x7fd0f0009bb0 comp rx=0 tx=0).stop 2024-04-24T09:10:26.016 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.064+0000 7fd102a00700 1 -- 172.21.15.90:0/1815515801 shutdown_connections 2024-04-24T09:10:26.016 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.064+0000 7fd102a00700 1 --2- 172.21.15.90:0/1815515801 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0fc07de20 0x7fd0fc07c2d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:26.016 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.064+0000 7fd102a00700 1 -- 172.21.15.90:0/1815515801 >> 172.21.15.90:0/1815515801 conn(0x7fd0fc103a50 msgr2=0x7fd0fc105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:26.017 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.064+0000 7fd102a00700 1 -- 172.21.15.90:0/1815515801 shutdown_connections 2024-04-24T09:10:26.017 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.064+0000 7fd102a00700 1 -- 172.21.15.90:0/1815515801 wait complete. 2024-04-24T09:10:26.017 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.065+0000 7fd102a00700 1 Processor -- start 2024-04-24T09:10:26.018 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.065+0000 7fd102a00700 1 -- start start 2024-04-24T09:10:26.018 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.066+0000 7fd102a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0fc07de20 0x7fd0fc19e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:26.018 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.066+0000 7fd102a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd0fc10e9e0 con 0x7fd0fc07de20 2024-04-24T09:10:26.018 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.066+0000 7fd101600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0fc07de20 0x7fd0fc19e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:26.019 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.066+0000 7fd101600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0fc07de20 0x7fd0fc19e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44086/0 (socket says 172.21.15.90:44086) 2024-04-24T09:10:26.019 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.066+0000 7fd101600700 1 -- 172.21.15.90:0/1069367106 learned_addr learned my addr 172.21.15.90:0/1069367106 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:10:26.019 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.066+0000 7fd101600700 1 -- 172.21.15.90:0/1069367106 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0fc19ead0 con 0x7fd0fc07de20 2024-04-24T09:10:26.019 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.067+0000 7fd101600700 1 --2- 172.21.15.90:0/1069367106 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0fc07de20 0x7fd0fc19e590 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0011b60 tx=0x7fd0f0011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:26.019 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.067+0000 7fd0fa000700 1 -- 172.21.15.90:0/1069367106 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0f0003970 con 0x7fd0fc07de20 2024-04-24T09:10:26.020 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.067+0000 7fd0fa000700 1 -- 172.21.15.90:0/1069367106 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd0f0003b10 con 0x7fd0fc07de20 2024-04-24T09:10:26.020 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.067+0000 7fd0fa000700 1 -- 172.21.15.90:0/1069367106 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0f0024610 con 0x7fd0fc07de20 2024-04-24T09:10:26.020 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.067+0000 7fd102a00700 1 -- 172.21.15.90:0/1069367106 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0fc19ed60 con 0x7fd0fc07de20 2024-04-24T09:10:26.020 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.067+0000 7fd102a00700 1 -- 172.21.15.90:0/1069367106 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0fc1a1a50 con 0x7fd0fc07de20 2024-04-24T09:10:26.022 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.068+0000 7fd102a00700 1 -- 172.21.15.90:0/1069367106 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0fc064150 con 0x7fd0fc07de20 2024-04-24T09:10:26.022 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.068+0000 7fd0fa000700 1 -- 172.21.15.90:0/1069367106 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fd0f001a430 con 0x7fd0fc07de20 2024-04-24T09:10:26.022 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.069+0000 7fd0fa000700 1 --2- 172.21.15.90:0/1069367106 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7fd0e4039600 0x7fd0e403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:26.022 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.069+0000 7fd0fa000700 1 -- 172.21.15.90:0/1069367106 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd0f0011e00 con 0x7fd0fc07de20 2024-04-24T09:10:26.023 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.069+0000 7fd100c00700 1 --2- 172.21.15.90:0/1069367106 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7fd0e4039600 0x7fd0e403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:26.023 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.070+0000 7fd100c00700 1 --2- 172.21.15.90:0/1069367106 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7fd0e4039600 0x7fd0e403bac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8006fd0 tx=0x7fd0e8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:26.023 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.075+0000 7fd0fa000700 1 -- 172.21.15.90:0/1069367106 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd0f00212a0 con 0x7fd0fc07de20 2024-04-24T09:10:26.023 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.319+0000 7fd102a00700 1 -- 172.21.15.90:0/1069367106 --> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fd0fc104c90 con 0x7fd0e4039600 2024-04-24T09:10:26.023 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.331+0000 7fd0fa000700 1 -- 172.21.15.90:0/1069367106 <== mgr.14120 v2:172.21.15.90:6800/1809038230 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd0fc104c90 con 0x7fd0e4039600 2024-04-24T09:10:26.023 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.334+0000 7fd102a00700 1 -- 172.21.15.90:0/1069367106 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7fd0e4039600 msgr2=0x7fd0e403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:26.023 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.334+0000 7fd102a00700 1 --2- 172.21.15.90:0/1069367106 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7fd0e4039600 0x7fd0e403bac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8006fd0 tx=0x7fd0e8006e40 comp rx=0 tx=0).stop 2024-04-24T09:10:26.023 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.334+0000 7fd102a00700 1 -- 172.21.15.90:0/1069367106 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0fc07de20 msgr2=0x7fd0fc19e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:26.023 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.334+0000 7fd102a00700 1 --2- 172.21.15.90:0/1069367106 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0fc07de20 0x7fd0fc19e590 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0011b60 tx=0x7fd0f0011c40 comp rx=0 tx=0).stop 2024-04-24T09:10:26.023 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.334+0000 7fd102a00700 1 -- 172.21.15.90:0/1069367106 shutdown_connections 2024-04-24T09:10:26.023 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.334+0000 7fd102a00700 1 --2- 172.21.15.90:0/1069367106 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7fd0e4039600 0x7fd0e403bac0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:26.023 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.334+0000 7fd102a00700 1 --2- 172.21.15.90:0/1069367106 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0fc07de20 0x7fd0fc19e590 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:26.023 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.334+0000 7fd102a00700 1 -- 172.21.15.90:0/1069367106 >> 172.21.15.90:0/1069367106 conn(0x7fd0fc103a50 msgr2=0x7fd0fc104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:26.024 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.334+0000 7fd102a00700 1 -- 172.21.15.90:0/1069367106 shutdown_connections 2024-04-24T09:10:26.024 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:25.334+0000 7fd102a00700 1 -- 172.21.15.90:0/1069367106 wait complete. 2024-04-24T09:10:26.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:26 smithi090 ceph-mon[27825]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:10:27.649 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-24T09:10:27.649 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.618+0000 7f63ce600700 1 Processor -- start 2024-04-24T09:10:27.650 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.618+0000 7f63ce600700 1 -- start start 2024-04-24T09:10:27.650 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.619+0000 7f63ce600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63c810ffc0 0x7f63c81103e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:27.650 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.619+0000 7f63ce600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f63c81109b0 con 0x7f63c810ffc0 2024-04-24T09:10:27.650 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.619+0000 7f63cd200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63c810ffc0 0x7f63c81103e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:27.650 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.619+0000 7f63cd200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63c810ffc0 0x7f63c81103e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40742/0 (socket says 172.21.15.90:40742) 2024-04-24T09:10:27.650 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.619+0000 7f63cd200700 1 -- 172.21.15.90:0/245876635 learned_addr learned my addr 172.21.15.90:0/245876635 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:10:27.650 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.620+0000 7f63cd200700 1 -- 172.21.15.90:0/245876635 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63c810ecc0 con 0x7f63c810ffc0 2024-04-24T09:10:27.650 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.620+0000 7f63cd200700 1 --2- 172.21.15.90:0/245876635 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63c810ffc0 0x7f63c81103e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f63bc0098a0 tx=0x7f63bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f6488cca510e486 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:27.650 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.620+0000 7f63c7400700 1 -- 172.21.15.90:0/245876635 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63bc004160 con 0x7f63c810ffc0 2024-04-24T09:10:27.650 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.620+0000 7f63c7400700 1 -- 172.21.15.90:0/245876635 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f63bc004300 con 0x7f63c810ffc0 2024-04-24T09:10:27.650 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.620+0000 7f63c7400700 1 -- 172.21.15.90:0/245876635 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63bc011b50 con 0x7f63c810ffc0 2024-04-24T09:10:27.651 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.621+0000 7f63ce600700 1 -- 172.21.15.90:0/245876635 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63c810ffc0 msgr2=0x7f63c81103e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:27.651 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.621+0000 7f63ce600700 1 --2- 172.21.15.90:0/245876635 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63c810ffc0 0x7f63c81103e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f63bc0098a0 tx=0x7f63bc009bb0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.651 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.621+0000 7f63ce600700 1 -- 172.21.15.90:0/245876635 shutdown_connections 2024-04-24T09:10:27.651 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.621+0000 7f63ce600700 1 --2- 172.21.15.90:0/245876635 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63c810ffc0 0x7f63c81103e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.651 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.621+0000 7f63ce600700 1 -- 172.21.15.90:0/245876635 >> 172.21.15.90:0/245876635 conn(0x7f63c807dbe0 msgr2=0x7f63c8080000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:27.651 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.621+0000 7f63ce600700 1 -- 172.21.15.90:0/245876635 shutdown_connections 2024-04-24T09:10:27.651 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.621+0000 7f63ce600700 1 -- 172.21.15.90:0/245876635 wait complete. 2024-04-24T09:10:27.651 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.622+0000 7f63ce600700 1 Processor -- start 2024-04-24T09:10:27.651 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.623+0000 7f63ce600700 1 -- start start 2024-04-24T09:10:27.651 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.623+0000 7f63ce600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63c810ffc0 0x7f63c81a29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:27.651 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.623+0000 7f63ce600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f63c8112f00 con 0x7f63c810ffc0 2024-04-24T09:10:27.651 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.623+0000 7f63cd200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63c810ffc0 0x7f63c81a29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:27.651 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.623+0000 7f63cd200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63c810ffc0 0x7f63c81a29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40750/0 (socket says 172.21.15.90:40750) 2024-04-24T09:10:27.652 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.623+0000 7f63cd200700 1 -- 172.21.15.90:0/3121050445 learned_addr learned my addr 172.21.15.90:0/3121050445 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:10:27.652 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.624+0000 7f63cd200700 1 -- 172.21.15.90:0/3121050445 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63c81a2f10 con 0x7f63c810ffc0 2024-04-24T09:10:27.652 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.624+0000 7f63cd200700 1 --2- 172.21.15.90:0/3121050445 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63c810ffc0 0x7f63c81a29d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f63bc011850 tx=0x7f63bc011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:27.652 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.624+0000 7f63c5600700 1 -- 172.21.15.90:0/3121050445 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63bc003af0 con 0x7f63c810ffc0 2024-04-24T09:10:27.652 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.624+0000 7f63c5600700 1 -- 172.21.15.90:0/3121050445 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f63bc020070 con 0x7f63c810ffc0 2024-04-24T09:10:27.652 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.624+0000 7f63ce600700 1 -- 172.21.15.90:0/3121050445 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63c81a31a0 con 0x7f63c810ffc0 2024-04-24T09:10:27.652 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.624+0000 7f63c5600700 1 -- 172.21.15.90:0/3121050445 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63bc0195a0 con 0x7f63c810ffc0 2024-04-24T09:10:27.652 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.624+0000 7f63ce600700 1 -- 172.21.15.90:0/3121050445 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63c81a5e90 con 0x7f63c810ffc0 2024-04-24T09:10:27.652 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.625+0000 7f63ce600700 1 -- 172.21.15.90:0/3121050445 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63c8113080 con 0x7f63c810ffc0 2024-04-24T09:10:27.652 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.625+0000 7f63c5600700 1 -- 172.21.15.90:0/3121050445 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f63bc019850 con 0x7f63c810ffc0 2024-04-24T09:10:27.652 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.626+0000 7f63c5600700 1 --2- 172.21.15.90:0/3121050445 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f63b0039600 0x7f63b003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:27.652 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.626+0000 7f63c5600700 1 -- 172.21.15.90:0/3121050445 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f63bc011e00 con 0x7f63c810ffc0 2024-04-24T09:10:27.652 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.626+0000 7f63c7e00700 1 --2- 172.21.15.90:0/3121050445 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f63b0039600 0x7f63b003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:27.653 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.627+0000 7f63c7e00700 1 --2- 172.21.15.90:0/3121050445 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f63b0039600 0x7f63b003bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f63b4006fd0 tx=0x7f63b4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:27.653 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.632+0000 7f63c5600700 1 -- 172.21.15.90:0/3121050445 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f63bc00e260 con 0x7f63c810ffc0 2024-04-24T09:10:27.653 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.859+0000 7f63ce600700 1 -- 172.21.15.90:0/3121050445 --> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f63c807ff10 con 0x7f63b0039600 2024-04-24T09:10:27.653 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.860+0000 7f63c5600700 1 -- 172.21.15.90:0/3121050445 <== mgr.14120 v2:172.21.15.90:6800/1809038230 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f63c807ff10 con 0x7f63b0039600 2024-04-24T09:10:27.653 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.863+0000 7f63ce600700 1 -- 172.21.15.90:0/3121050445 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f63b0039600 msgr2=0x7f63b003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:27.653 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.863+0000 7f63ce600700 1 --2- 172.21.15.90:0/3121050445 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f63b0039600 0x7f63b003bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f63b4006fd0 tx=0x7f63b4006e40 comp rx=0 tx=0).stop 2024-04-24T09:10:27.653 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.863+0000 7f63ce600700 1 -- 172.21.15.90:0/3121050445 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63c810ffc0 msgr2=0x7f63c81a29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:27.653 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.863+0000 7f63ce600700 1 --2- 172.21.15.90:0/3121050445 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63c810ffc0 0x7f63c81a29d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f63bc011850 tx=0x7f63bc011c10 comp rx=0 tx=0).stop 2024-04-24T09:10:27.653 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.863+0000 7f63ce600700 1 -- 172.21.15.90:0/3121050445 shutdown_connections 2024-04-24T09:10:27.653 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.863+0000 7f63ce600700 1 --2- 172.21.15.90:0/3121050445 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f63b0039600 0x7f63b003bac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.653 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.863+0000 7f63ce600700 1 --2- 172.21.15.90:0/3121050445 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63c810ffc0 0x7f63c81a29d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.653 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.863+0000 7f63ce600700 1 -- 172.21.15.90:0/3121050445 >> 172.21.15.90:0/3121050445 conn(0x7f63c807dbe0 msgr2=0x7f63c807f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:27.653 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.863+0000 7f63ce600700 1 -- 172.21.15.90:0/3121050445 shutdown_connections 2024-04-24T09:10:27.653 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:26.863+0000 7f63ce600700 1 -- 172.21.15.90:0/3121050445 wait complete. 2024-04-24T09:10:27.653 INFO:teuthology.orchestra.run.smithi090.stdout:Generating ssh key... 2024-04-24T09:10:27.927 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:27 smithi090 ceph-mon[27825]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:10:29.104 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:28 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: Generating public/private rsa key pair. 2024-04-24T09:10:29.104 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:28 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: Your identification has been saved in /tmp/tmprudhe6wa/key. 2024-04-24T09:10:29.104 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:28 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: Your public key has been saved in /tmp/tmprudhe6wa/key.pub. 2024-04-24T09:10:29.104 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:28 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: The key fingerprint is: 2024-04-24T09:10:29.104 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:28 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: SHA256:0Lox04Dnm5Edl6GUJws68qar598uAcUT+89GjQ2/WOo ceph-30ee9be2-021a-11ef-bc93-c7b262605968 2024-04-24T09:10:29.104 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:28 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: The key's randomart image is: 2024-04-24T09:10:29.104 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:28 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: +---[RSA 3072]----+ 2024-04-24T09:10:29.104 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:28 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: | ... ... | 2024-04-24T09:10:29.104 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:28 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: | +o.oo..o | 2024-04-24T09:10:29.104 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:28 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: | .oo+.=+o | 2024-04-24T09:10:29.104 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:28 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: | o o+ B.O | 2024-04-24T09:10:29.105 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:28 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: | + .X S = | 2024-04-24T09:10:29.105 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:28 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: | + % + . | 2024-04-24T09:10:29.105 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:28 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: | o .+ * . | 2024-04-24T09:10:29.105 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:28 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: | o .. o | 2024-04-24T09:10:29.105 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:28 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: |.+oo.oo E | 2024-04-24T09:10:29.105 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:28 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: +----[SHA256]-----+ 2024-04-24T09:10:29.504 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.232+0000 7f92dd200700 1 Processor -- start 2024-04-24T09:10:29.504 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.233+0000 7f92dd200700 1 -- start start 2024-04-24T09:10:29.504 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.233+0000 7f92dd200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92d810a7d0 0x7f92d810abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:29.504 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.233+0000 7f92dd200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f92d810b1c0 con 0x7f92d810a7d0 2024-04-24T09:10:29.504 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.233+0000 7f92d5e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92d810a7d0 0x7f92d810abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:29.504 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.234+0000 7f92d5e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92d810a7d0 0x7f92d810abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40756/0 (socket says 172.21.15.90:40756) 2024-04-24T09:10:29.504 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.234+0000 7f92d5e00700 1 -- 172.21.15.90:0/4204860666 learned_addr learned my addr 172.21.15.90:0/4204860666 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:10:29.504 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.234+0000 7f92d5e00700 1 -- 172.21.15.90:0/4204860666 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92d810b980 con 0x7f92d810a7d0 2024-04-24T09:10:29.504 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.235+0000 7f92d5e00700 1 --2- 172.21.15.90:0/4204860666 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92d810a7d0 0x7f92d810abf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f92c8009e80 tx=0x7f92c800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79d11f0c39582450 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:29.504 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.235+0000 7f92d5400700 1 -- 172.21.15.90:0/4204860666 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92c8017070 con 0x7f92d810a7d0 2024-04-24T09:10:29.504 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.235+0000 7f92d5400700 1 -- 172.21.15.90:0/4204860666 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f92c800b8c0 con 0x7f92d810a7d0 2024-04-24T09:10:29.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.235+0000 7f92d5400700 1 -- 172.21.15.90:0/4204860666 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92c80045c0 con 0x7f92d810a7d0 2024-04-24T09:10:29.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.236+0000 7f92dd200700 1 -- 172.21.15.90:0/4204860666 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92d810a7d0 msgr2=0x7f92d810abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:29.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.236+0000 7f92dd200700 1 --2- 172.21.15.90:0/4204860666 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92d810a7d0 0x7f92d810abf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f92c8009e80 tx=0x7f92c800b330 comp rx=0 tx=0).stop 2024-04-24T09:10:29.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.236+0000 7f92dd200700 1 -- 172.21.15.90:0/4204860666 shutdown_connections 2024-04-24T09:10:29.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.236+0000 7f92dd200700 1 --2- 172.21.15.90:0/4204860666 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92d810a7d0 0x7f92d810abf0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:29.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.236+0000 7f92dd200700 1 -- 172.21.15.90:0/4204860666 >> 172.21.15.90:0/4204860666 conn(0x7f92d807ddc0 msgr2=0x7f92d81084c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:29.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.236+0000 7f92dd200700 1 -- 172.21.15.90:0/4204860666 shutdown_connections 2024-04-24T09:10:29.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.236+0000 7f92dd200700 1 -- 172.21.15.90:0/4204860666 wait complete. 2024-04-24T09:10:29.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.237+0000 7f92dd200700 1 Processor -- start 2024-04-24T09:10:29.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.237+0000 7f92dd200700 1 -- start start 2024-04-24T09:10:29.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.237+0000 7f92dd200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92d810a7d0 0x7f92d819e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:29.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.237+0000 7f92dd200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f92d810ed30 con 0x7f92d810a7d0 2024-04-24T09:10:29.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.238+0000 7f92d5e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92d810a7d0 0x7f92d819e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:29.506 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.238+0000 7f92d5e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92d810a7d0 0x7f92d819e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40760/0 (socket says 172.21.15.90:40760) 2024-04-24T09:10:29.506 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.238+0000 7f92d5e00700 1 -- 172.21.15.90:0/2113278896 learned_addr learned my addr 172.21.15.90:0/2113278896 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:10:29.506 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.238+0000 7f92d5e00700 1 -- 172.21.15.90:0/2113278896 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92d819ec30 con 0x7f92d810a7d0 2024-04-24T09:10:29.506 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.239+0000 7f92d5e00700 1 --2- 172.21.15.90:0/2113278896 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92d810a7d0 0x7f92d819e6f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f92c80072a0 tx=0x7f92c800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:29.506 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.239+0000 7f92cf400700 1 -- 172.21.15.90:0/2113278896 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92c8020020 con 0x7f92d810a7d0 2024-04-24T09:10:29.506 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.239+0000 7f92cf400700 1 -- 172.21.15.90:0/2113278896 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f92c8003b50 con 0x7f92d810a7d0 2024-04-24T09:10:29.506 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.239+0000 7f92dd200700 1 -- 172.21.15.90:0/2113278896 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92d819eec0 con 0x7f92d810a7d0 2024-04-24T09:10:29.506 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.239+0000 7f92cf400700 1 -- 172.21.15.90:0/2113278896 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92c8017040 con 0x7f92d810a7d0 2024-04-24T09:10:29.506 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.239+0000 7f92dd200700 1 -- 172.21.15.90:0/2113278896 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92d81a1bb0 con 0x7f92d810a7d0 2024-04-24T09:10:29.506 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.240+0000 7f92cf400700 1 -- 172.21.15.90:0/2113278896 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f92c8023020 con 0x7f92d810a7d0 2024-04-24T09:10:29.506 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.240+0000 7f92dd200700 1 -- 172.21.15.90:0/2113278896 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92d8064150 con 0x7f92d810a7d0 2024-04-24T09:10:29.506 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.241+0000 7f92cf400700 1 --2- 172.21.15.90:0/2113278896 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f92bc0395b0 0x7f92bc03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:29.506 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.241+0000 7f92cf400700 1 -- 172.21.15.90:0/2113278896 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f92c800f050 con 0x7f92d810a7d0 2024-04-24T09:10:29.506 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.241+0000 7f92cd400700 1 --2- 172.21.15.90:0/2113278896 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f92bc0395b0 0x7f92bc03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:29.507 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.242+0000 7f92cd400700 1 --2- 172.21.15.90:0/2113278896 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f92bc0395b0 0x7f92bc03ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f92c0006fd0 tx=0x7f92c0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:29.507 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.246+0000 7f92cf400700 1 -- 172.21.15.90:0/2113278896 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f92c802a020 con 0x7f92d810a7d0 2024-04-24T09:10:29.507 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.472+0000 7f92dd200700 1 -- 172.21.15.90:0/2113278896 --> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f92d81082b0 con 0x7f92bc0395b0 2024-04-24T09:10:29.507 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.685+0000 7f92cf400700 1 -- 172.21.15.90:0/2113278896 <== mgr.14120 v2:172.21.15.90:6800/1809038230 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f92d81082b0 con 0x7f92bc0395b0 2024-04-24T09:10:29.507 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.688+0000 7f92dd200700 1 -- 172.21.15.90:0/2113278896 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f92bc0395b0 msgr2=0x7f92bc03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:29.507 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.688+0000 7f92dd200700 1 --2- 172.21.15.90:0/2113278896 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f92bc0395b0 0x7f92bc03ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f92c0006fd0 tx=0x7f92c0006e40 comp rx=0 tx=0).stop 2024-04-24T09:10:29.507 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.688+0000 7f92dd200700 1 -- 172.21.15.90:0/2113278896 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92d810a7d0 msgr2=0x7f92d819e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:29.507 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.688+0000 7f92dd200700 1 --2- 172.21.15.90:0/2113278896 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92d810a7d0 0x7f92d819e6f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f92c80072a0 tx=0x7f92c800bd50 comp rx=0 tx=0).stop 2024-04-24T09:10:29.507 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.688+0000 7f92dd200700 1 -- 172.21.15.90:0/2113278896 shutdown_connections 2024-04-24T09:10:29.507 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.688+0000 7f92dd200700 1 --2- 172.21.15.90:0/2113278896 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f92bc0395b0 0x7f92bc03ba70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:29.507 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.688+0000 7f92dd200700 1 --2- 172.21.15.90:0/2113278896 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92d810a7d0 0x7f92d819e6f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:29.507 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.688+0000 7f92dd200700 1 -- 172.21.15.90:0/2113278896 >> 172.21.15.90:0/2113278896 conn(0x7f92d807ddc0 msgr2=0x7f92d8107ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:29.507 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.688+0000 7f92dd200700 1 -- 172.21.15.90:0/2113278896 shutdown_connections 2024-04-24T09:10:29.508 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:28.688+0000 7f92dd200700 1 -- 172.21.15.90:0/2113278896 wait complete. 2024-04-24T09:10:29.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:29 smithi090 ceph-mon[27825]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:10:29.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:29 smithi090 ceph-mon[27825]: Generating ssh key... 2024-04-24T09:10:29.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:29 smithi090 ceph-mon[27825]: from='mgr.14120 172.21.15.90:0/738983790' entity='mgr.x' 2024-04-24T09:10:29.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:29 smithi090 ceph-mon[27825]: from='mgr.14120 172.21.15.90:0/738983790' entity='mgr.x' 2024-04-24T09:10:31.101 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:30 smithi090 ceph-mon[27825]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:10:31.119 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDja/7j8fYYTD44wRLm7SUM4mZ0M5HulNpxXPWEYnjrmIB8sx4wwfBTPb5TbjIat4ZCIhjUCI7v07EyDtl3xRuvTuSSzX6BgMXX7eM/ZMFkDud7LXypYEfUtjAqe1zlub/BH1betO9E6ZfExXVgNp4H9m+k6H9JY+bZx1005epsH+/eIw0PeEtioC86oWkbE3ieJVkP/GbhUEvaP95oc1D+y86QVNiQQbk26KnRzzwU8K0YcXceABY1qugtNMIpAbJay0Am7RZyTelvkLl5BX6+wlPnyRXkUDOV67nzuwUXCi3lUQeEgHt7jTpfx+Wzn0z4HTThNQxHdUxhG50AN5UK4TRCmTAYCkGRJ+MLa6Ldh/hS5+TVvtZP73oSlmGoU+VVimk4ug9cNT8FiJUdeGhlcE4RRlK9YQgWrhsdtJohcJHpZrS+p+YWOrTHIMA8xtWHS38QOFybuss4ksFqo7nuIlekbro3YqUnyTUCKNCsVkmMMZ4TffJRnDi1RNiPGzU= ceph-30ee9be2-021a-11ef-bc93-c7b262605968 2024-04-24T09:10:31.119 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.168+0000 7f493fe00700 1 Processor -- start 2024-04-24T09:10:31.119 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.168+0000 7f493fe00700 1 -- start start 2024-04-24T09:10:31.119 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.169+0000 7f493fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4940106c70 0x7f49401090a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:31.119 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.169+0000 7f493fe00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f494007bea0 con 0x7f4940106c70 2024-04-24T09:10:31.120 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.169+0000 7f493ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4940106c70 0x7f49401090a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:31.120 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.169+0000 7f493ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4940106c70 0x7f49401090a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40764/0 (socket says 172.21.15.90:40764) 2024-04-24T09:10:31.120 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.169+0000 7f493ea00700 1 -- 172.21.15.90:0/2944082361 learned_addr learned my addr 172.21.15.90:0/2944082361 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:10:31.120 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.170+0000 7f493ea00700 1 -- 172.21.15.90:0/2944082361 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49401095e0 con 0x7f4940106c70 2024-04-24T09:10:31.120 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.170+0000 7f493ea00700 1 --2- 172.21.15.90:0/2944082361 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4940106c70 0x7f49401090a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f49300098a0 tx=0x7f4930009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3224374306ed1c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:31.120 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.171+0000 7f493d600700 1 -- 172.21.15.90:0/2944082361 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4930004160 con 0x7f4940106c70 2024-04-24T09:10:31.120 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.171+0000 7f493d600700 1 -- 172.21.15.90:0/2944082361 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4930004300 con 0x7f4940106c70 2024-04-24T09:10:31.120 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.171+0000 7f493d600700 1 -- 172.21.15.90:0/2944082361 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4930011b50 con 0x7f4940106c70 2024-04-24T09:10:31.120 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.171+0000 7f493fe00700 1 -- 172.21.15.90:0/2944082361 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4940106c70 msgr2=0x7f49401090a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:31.120 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.171+0000 7f493fe00700 1 --2- 172.21.15.90:0/2944082361 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4940106c70 0x7f49401090a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f49300098a0 tx=0x7f4930009bb0 comp rx=0 tx=0).stop 2024-04-24T09:10:31.120 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.171+0000 7f493fe00700 1 -- 172.21.15.90:0/2944082361 shutdown_connections 2024-04-24T09:10:31.120 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.171+0000 7f493fe00700 1 --2- 172.21.15.90:0/2944082361 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4940106c70 0x7f49401090a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:31.120 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.172+0000 7f493fe00700 1 -- 172.21.15.90:0/2944082361 >> 172.21.15.90:0/2944082361 conn(0x7f49401028d0 msgr2=0x7f4940104cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:31.120 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.172+0000 7f493fe00700 1 -- 172.21.15.90:0/2944082361 shutdown_connections 2024-04-24T09:10:31.120 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.172+0000 7f493fe00700 1 -- 172.21.15.90:0/2944082361 wait complete. 2024-04-24T09:10:31.121 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.173+0000 7f493fe00700 1 Processor -- start 2024-04-24T09:10:31.121 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.173+0000 7f493fe00700 1 -- start start 2024-04-24T09:10:31.121 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.173+0000 7f493fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4940106c70 0x7f49401a2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:31.121 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.173+0000 7f493fe00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f494010a670 con 0x7f4940106c70 2024-04-24T09:10:31.121 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.173+0000 7f493ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4940106c70 0x7f49401a2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:31.121 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.174+0000 7f493ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4940106c70 0x7f49401a2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40772/0 (socket says 172.21.15.90:40772) 2024-04-24T09:10:31.121 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.174+0000 7f493ea00700 1 -- 172.21.15.90:0/3474712937 learned_addr learned my addr 172.21.15.90:0/3474712937 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:10:31.121 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.174+0000 7f493ea00700 1 -- 172.21.15.90:0/3474712937 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49401a2f50 con 0x7f4940106c70 2024-04-24T09:10:31.121 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.174+0000 7f493ea00700 1 --2- 172.21.15.90:0/3474712937 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4940106c70 0x7f49401a2a10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4930011b60 tx=0x7f4930011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:31.121 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.174+0000 7f4937400700 1 -- 172.21.15.90:0/3474712937 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4930003970 con 0x7f4940106c70 2024-04-24T09:10:31.121 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.174+0000 7f4937400700 1 -- 172.21.15.90:0/3474712937 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4930003b10 con 0x7f4940106c70 2024-04-24T09:10:31.121 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.174+0000 7f493fe00700 1 -- 172.21.15.90:0/3474712937 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49401a31e0 con 0x7f4940106c70 2024-04-24T09:10:31.121 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.174+0000 7f4937400700 1 -- 172.21.15.90:0/3474712937 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4930024610 con 0x7f4940106c70 2024-04-24T09:10:31.121 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.175+0000 7f493fe00700 1 -- 172.21.15.90:0/3474712937 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49401a5ed0 con 0x7f4940106c70 2024-04-24T09:10:31.121 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.176+0000 7f4937400700 1 -- 172.21.15.90:0/3474712937 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f493001a9f0 con 0x7f4940106c70 2024-04-24T09:10:31.121 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.176+0000 7f493fe00700 1 -- 172.21.15.90:0/3474712937 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4940064150 con 0x7f4940106c70 2024-04-24T09:10:31.122 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.176+0000 7f4937400700 1 --2- 172.21.15.90:0/3474712937 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f4924039660 0x7f492403bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:31.122 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.176+0000 7f4937400700 1 -- 172.21.15.90:0/3474712937 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4930011e00 con 0x7f4940106c70 2024-04-24T09:10:31.122 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.177+0000 7f493e000700 1 --2- 172.21.15.90:0/3474712937 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f4924039660 0x7f492403bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:31.122 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.177+0000 7f493e000700 1 --2- 172.21.15.90:0/3474712937 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f4924039660 0x7f492403bb20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f492c006fd0 tx=0x7f492c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:31.122 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.182+0000 7f4937400700 1 -- 172.21.15.90:0/3474712937 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f493000b5a0 con 0x7f4940106c70 2024-04-24T09:10:31.122 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.406+0000 7f493fe00700 1 -- 172.21.15.90:0/3474712937 --> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f4940062f30 con 0x7f4924039660 2024-04-24T09:10:31.122 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.408+0000 7f4937400700 1 -- 172.21.15.90:0/3474712937 <== mgr.14120 v2:172.21.15.90:6800/1809038230 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f4940062f30 con 0x7f4924039660 2024-04-24T09:10:31.122 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.411+0000 7f493fe00700 1 -- 172.21.15.90:0/3474712937 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f4924039660 msgr2=0x7f492403bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:31.122 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.411+0000 7f493fe00700 1 --2- 172.21.15.90:0/3474712937 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f4924039660 0x7f492403bb20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f492c006fd0 tx=0x7f492c006e40 comp rx=0 tx=0).stop 2024-04-24T09:10:31.122 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.411+0000 7f493fe00700 1 -- 172.21.15.90:0/3474712937 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4940106c70 msgr2=0x7f49401a2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:31.122 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.411+0000 7f493fe00700 1 --2- 172.21.15.90:0/3474712937 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4940106c70 0x7f49401a2a10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4930011b60 tx=0x7f4930011c40 comp rx=0 tx=0).stop 2024-04-24T09:10:31.122 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.411+0000 7f493fe00700 1 -- 172.21.15.90:0/3474712937 shutdown_connections 2024-04-24T09:10:31.123 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.411+0000 7f493fe00700 1 --2- 172.21.15.90:0/3474712937 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f4924039660 0x7f492403bb20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:31.123 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.411+0000 7f493fe00700 1 --2- 172.21.15.90:0/3474712937 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4940106c70 0x7f49401a2a10 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:31.123 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.411+0000 7f493fe00700 1 -- 172.21.15.90:0/3474712937 >> 172.21.15.90:0/3474712937 conn(0x7f49401028d0 msgr2=0x7f4940104cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:31.123 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.412+0000 7f493fe00700 1 -- 172.21.15.90:0/3474712937 shutdown_connections 2024-04-24T09:10:31.123 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:30.412+0000 7f493fe00700 1 -- 172.21.15.90:0/3474712937 wait complete. 2024-04-24T09:10:31.123 INFO:teuthology.orchestra.run.smithi090.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-24T09:10:31.123 INFO:teuthology.orchestra.run.smithi090.stdout:Adding key to root@localhost authorized_keys... 2024-04-24T09:10:31.127 INFO:teuthology.orchestra.run.smithi090.stdout:Adding host smithi090... 2024-04-24T09:10:33.220 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:32 smithi090 ceph-mon[27825]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi090", "addr": "172.21.15.90", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:10:34.056 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:33 smithi090 ceph-mon[27825]: Deploying cephadm binary to smithi090 2024-04-24T09:10:38.637 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout Added host 'smithi090' with addr '172.21.15.90' 2024-04-24T09:10:38.638 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:31.740+0000 7fc7aca00700 1 Processor -- start 2024-04-24T09:10:38.638 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:31.740+0000 7fc7aca00700 1 -- start start 2024-04-24T09:10:38.638 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:31.741+0000 7fc7aca00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7a807de80 0x7fc7a807c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:38.638 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:31.741+0000 7fc7aca00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a807c900 con 0x7fc7a807de80 2024-04-24T09:10:38.639 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:31.741+0000 7fc7a7400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7a807de80 0x7fc7a807c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:38.639 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:31.741+0000 7fc7a7400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7a807de80 0x7fc7a807c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40778/0 (socket says 172.21.15.90:40778) 2024-04-24T09:10:38.639 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:31.741+0000 7fc7a7400700 1 -- 172.21.15.90:0/748970663 learned_addr learned my addr 172.21.15.90:0/748970663 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:10:38.639 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:31.742+0000 7fc7a7400700 1 -- 172.21.15.90:0/748970663 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7a807ca80 con 0x7fc7a807de80 2024-04-24T09:10:38.639 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:31.742+0000 7fc7a7400700 1 --2- 172.21.15.90:0/748970663 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7a807de80 0x7fc7a807c330 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc7980098a0 tx=0x7fc798009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf2748c25b9f46f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:38.639 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:31.742+0000 7fc7a6000700 1 -- 172.21.15.90:0/748970663 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc798004160 con 0x7fc7a807de80 2024-04-24T09:10:38.640 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:31.742+0000 7fc7a6000700 1 -- 172.21.15.90:0/748970663 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc798004300 con 0x7fc7a807de80 2024-04-24T09:10:38.640 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:31.742+0000 7fc7a6000700 1 -- 172.21.15.90:0/748970663 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc798011b50 con 0x7fc7a807de80 2024-04-24T09:10:38.642 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:31.743+0000 7fc7aca00700 1 -- 172.21.15.90:0/748970663 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7a807de80 msgr2=0x7fc7a807c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:38.642 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:31.743+0000 7fc7aca00700 1 --2- 172.21.15.90:0/748970663 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7a807de80 0x7fc7a807c330 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc7980098a0 tx=0x7fc798009bb0 comp rx=0 tx=0).stop 2024-04-24T09:10:38.642 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:31.743+0000 7fc7aca00700 1 -- 172.21.15.90:0/748970663 shutdown_connections 2024-04-24T09:10:38.642 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:31.743+0000 7fc7aca00700 1 --2- 172.21.15.90:0/748970663 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7a807de80 0x7fc7a807c330 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:38.642 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:31.743+0000 7fc7aca00700 1 -- 172.21.15.90:0/748970663 >> 172.21.15.90:0/748970663 conn(0x7fc7a8103a60 msgr2=0x7fc7a8105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:38.642 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:31.743+0000 7fc7aca00700 1 -- 172.21.15.90:0/748970663 shutdown_connections 2024-04-24T09:10:38.642 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:31.743+0000 7fc7aca00700 1 -- 172.21.15.90:0/748970663 wait complete. 2024-04-24T09:10:38.642 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:31.744+0000 7fc7aca00700 1 Processor -- start 2024-04-24T09:10:38.643 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:31.745+0000 7fc7aca00700 1 -- start start 2024-04-24T09:10:38.643 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:31.745+0000 7fc7aca00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7a807de80 0x7fc7a81a2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:38.643 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:31.745+0000 7fc7aca00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a810a5a0 con 0x7fc7a807de80 2024-04-24T09:10:38.643 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:31.745+0000 7fc7a7400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7a807de80 0x7fc7a81a2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:38.643 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:31.745+0000 7fc7a7400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7a807de80 0x7fc7a81a2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40782/0 (socket says 172.21.15.90:40782) 2024-04-24T09:10:38.643 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:31.745+0000 7fc7a7400700 1 -- 172.21.15.90:0/3381619811 learned_addr learned my addr 172.21.15.90:0/3381619811 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:10:38.643 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:31.746+0000 7fc7a7400700 1 -- 172.21.15.90:0/3381619811 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7a81a2e80 con 0x7fc7a807de80 2024-04-24T09:10:38.643 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:31.746+0000 7fc7a7400700 1 --2- 172.21.15.90:0/3381619811 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7a807de80 0x7fc7a81a2940 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc798011b60 tx=0x7fc798011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:38.643 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:31.746+0000 7fc79fe00700 1 -- 172.21.15.90:0/3381619811 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc798003b20 con 0x7fc7a807de80 2024-04-24T09:10:38.643 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:31.746+0000 7fc7aca00700 1 -- 172.21.15.90:0/3381619811 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7a81a3110 con 0x7fc7a807de80 2024-04-24T09:10:38.646 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:31.746+0000 7fc79fe00700 1 -- 172.21.15.90:0/3381619811 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc798003cc0 con 0x7fc7a807de80 2024-04-24T09:10:38.646 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:31.746+0000 7fc79fe00700 1 -- 172.21.15.90:0/3381619811 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc798024840 con 0x7fc7a807de80 2024-04-24T09:10:38.646 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:31.747+0000 7fc7aca00700 1 -- 172.21.15.90:0/3381619811 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7a81a5e00 con 0x7fc7a807de80 2024-04-24T09:10:38.646 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:31.748+0000 7fc79fe00700 1 -- 172.21.15.90:0/3381619811 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fc798022020 con 0x7fc7a807de80 2024-04-24T09:10:38.646 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:31.748+0000 7fc7aca00700 1 -- 172.21.15.90:0/3381619811 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7a8064150 con 0x7fc7a807de80 2024-04-24T09:10:38.646 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:31.749+0000 7fc79fe00700 1 --2- 172.21.15.90:0/3381619811 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7fc78c0395b0 0x7fc78c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:38.646 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:31.749+0000 7fc79fe00700 1 -- 172.21.15.90:0/3381619811 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc798011e00 con 0x7fc7a807de80 2024-04-24T09:10:38.646 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:31.749+0000 7fc7a6a00700 1 --2- 172.21.15.90:0/3381619811 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7fc78c0395b0 0x7fc78c03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:38.646 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:31.750+0000 7fc7a6a00700 1 --2- 172.21.15.90:0/3381619811 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7fc78c0395b0 0x7fc78c03ba70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc794006fd0 tx=0x7fc794006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:38.646 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:31.757+0000 7fc79fe00700 1 -- 172.21.15.90:0/3381619811 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc79804d240 con 0x7fc7a807de80 2024-04-24T09:10:38.647 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:31.987+0000 7fc7aca00700 1 -- 172.21.15.90:0/3381619811 --> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi090", "addr": "172.21.15.90", "target": ["mon-mgr", ""]}) -- 0x7fc7a8062f30 con 0x7fc78c0395b0 2024-04-24T09:10:38.647 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:37.934+0000 7fc79fe00700 1 -- 172.21.15.90:0/3381619811 <== mgr.14120 v2:172.21.15.90:6800/1809038230 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fc7a8062f30 con 0x7fc78c0395b0 2024-04-24T09:10:38.647 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:37.939+0000 7fc7aca00700 1 -- 172.21.15.90:0/3381619811 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7fc78c0395b0 msgr2=0x7fc78c03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:38.647 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:37.939+0000 7fc7aca00700 1 --2- 172.21.15.90:0/3381619811 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7fc78c0395b0 0x7fc78c03ba70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc794006fd0 tx=0x7fc794006e40 comp rx=0 tx=0).stop 2024-04-24T09:10:38.647 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:37.939+0000 7fc7aca00700 1 -- 172.21.15.90:0/3381619811 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7a807de80 msgr2=0x7fc7a81a2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:38.647 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:37.939+0000 7fc7aca00700 1 --2- 172.21.15.90:0/3381619811 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7a807de80 0x7fc7a81a2940 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc798011b60 tx=0x7fc798011c40 comp rx=0 tx=0).stop 2024-04-24T09:10:38.647 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:37.939+0000 7fc7aca00700 1 -- 172.21.15.90:0/3381619811 shutdown_connections 2024-04-24T09:10:38.647 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:37.939+0000 7fc7aca00700 1 --2- 172.21.15.90:0/3381619811 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7fc78c0395b0 0x7fc78c03ba70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:38.647 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:37.939+0000 7fc7aca00700 1 --2- 172.21.15.90:0/3381619811 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7a807de80 0x7fc7a81a2940 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:38.647 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:37.939+0000 7fc7aca00700 1 -- 172.21.15.90:0/3381619811 >> 172.21.15.90:0/3381619811 conn(0x7fc7a8103a60 msgr2=0x7fc7a8105e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:38.647 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:37.940+0000 7fc7aca00700 1 -- 172.21.15.90:0/3381619811 shutdown_connections 2024-04-24T09:10:38.647 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:37.940+0000 7fc7aca00700 1 -- 172.21.15.90:0/3381619811 wait complete. 2024-04-24T09:10:38.647 INFO:teuthology.orchestra.run.smithi090.stdout:Deploying unmanaged mon service... 2024-04-24T09:10:39.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:38 smithi090 ceph-mon[27825]: from='mgr.14120 172.21.15.90:0/738983790' entity='mgr.x' 2024-04-24T09:10:39.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:38 smithi090 ceph-mon[27825]: Added host smithi090 2024-04-24T09:10:39.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:38 smithi090 ceph-mon[27825]: from='mgr.14120 172.21.15.90:0/738983790' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:10:40.719 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-24T09:10:40.719 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.602+0000 7f2528c00700 1 Processor -- start 2024-04-24T09:10:40.719 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.602+0000 7f2528c00700 1 -- start start 2024-04-24T09:10:40.719 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.602+0000 7f2528c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f252410f0a0 0x7f252410f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:40.719 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.602+0000 7f2528c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f252410fa90 con 0x7f252410f0a0 2024-04-24T09:10:40.719 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.603+0000 7f2521e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f252410f0a0 0x7f252410f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:40.719 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.603+0000 7f2521e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f252410f0a0 0x7f252410f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54260/0 (socket says 172.21.15.90:54260) 2024-04-24T09:10:40.719 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.603+0000 7f2521e00700 1 -- 172.21.15.90:0/1732843383 learned_addr learned my addr 172.21.15.90:0/1732843383 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:10:40.719 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.604+0000 7f2521e00700 1 -- 172.21.15.90:0/1732843383 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f252410fd70 con 0x7f252410f0a0 2024-04-24T09:10:40.719 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.604+0000 7f2521e00700 1 --2- 172.21.15.90:0/1732843383 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f252410f0a0 0x7f252410f4c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f25140098a0 tx=0x7f2514009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ada7db1917a8701 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:40.720 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.604+0000 7f2520a00700 1 -- 172.21.15.90:0/1732843383 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2514004160 con 0x7f252410f0a0 2024-04-24T09:10:40.720 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.604+0000 7f2520a00700 1 -- 172.21.15.90:0/1732843383 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2514004300 con 0x7f252410f0a0 2024-04-24T09:10:40.720 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.604+0000 7f2520a00700 1 -- 172.21.15.90:0/1732843383 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2514011b50 con 0x7f252410f0a0 2024-04-24T09:10:40.720 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.605+0000 7f2528c00700 1 -- 172.21.15.90:0/1732843383 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f252410f0a0 msgr2=0x7f252410f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:40.720 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.605+0000 7f2528c00700 1 --2- 172.21.15.90:0/1732843383 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f252410f0a0 0x7f252410f4c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f25140098a0 tx=0x7f2514009bb0 comp rx=0 tx=0).stop 2024-04-24T09:10:40.720 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.605+0000 7f2528c00700 1 -- 172.21.15.90:0/1732843383 shutdown_connections 2024-04-24T09:10:40.720 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.605+0000 7f2528c00700 1 --2- 172.21.15.90:0/1732843383 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f252410f0a0 0x7f252410f4c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:40.720 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.605+0000 7f2528c00700 1 -- 172.21.15.90:0/1732843383 >> 172.21.15.90:0/1732843383 conn(0x7f252407ddc0 msgr2=0x7f25240801e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:40.720 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.605+0000 7f2528c00700 1 -- 172.21.15.90:0/1732843383 shutdown_connections 2024-04-24T09:10:40.720 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.605+0000 7f2528c00700 1 -- 172.21.15.90:0/1732843383 wait complete. 2024-04-24T09:10:40.720 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.606+0000 7f2528c00700 1 Processor -- start 2024-04-24T09:10:40.720 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.606+0000 7f2528c00700 1 -- start start 2024-04-24T09:10:40.721 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.607+0000 7f2528c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f252410f0a0 0x7f25241a2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:40.721 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.607+0000 7f2528c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2524113120 con 0x7f252410f0a0 2024-04-24T09:10:40.721 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.607+0000 7f2521e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f252410f0a0 0x7f25241a2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:40.721 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.607+0000 7f2521e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f252410f0a0 0x7f25241a2b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54272/0 (socket says 172.21.15.90:54272) 2024-04-24T09:10:40.721 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.607+0000 7f2521e00700 1 -- 172.21.15.90:0/2607121867 learned_addr learned my addr 172.21.15.90:0/2607121867 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:10:40.721 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.607+0000 7f2521e00700 1 -- 172.21.15.90:0/2607121867 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25241a3080 con 0x7f252410f0a0 2024-04-24T09:10:40.721 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.607+0000 7f2521e00700 1 --2- 172.21.15.90:0/2607121867 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f252410f0a0 0x7f25241a2b40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2514000c00 tx=0x7f2514011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:40.721 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.608+0000 7f251aa00700 1 -- 172.21.15.90:0/2607121867 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2514003b20 con 0x7f252410f0a0 2024-04-24T09:10:40.721 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.608+0000 7f251aa00700 1 -- 172.21.15.90:0/2607121867 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2514003cc0 con 0x7f252410f0a0 2024-04-24T09:10:40.721 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.608+0000 7f251aa00700 1 -- 172.21.15.90:0/2607121867 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2514024910 con 0x7f252410f0a0 2024-04-24T09:10:40.721 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.608+0000 7f2528c00700 1 -- 172.21.15.90:0/2607121867 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25241a3310 con 0x7f252410f0a0 2024-04-24T09:10:40.721 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.608+0000 7f2528c00700 1 -- 172.21.15.90:0/2607121867 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25241a6000 con 0x7f252410f0a0 2024-04-24T09:10:40.721 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.609+0000 7f251aa00700 1 -- 172.21.15.90:0/2607121867 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f2514022020 con 0x7f252410f0a0 2024-04-24T09:10:40.722 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.609+0000 7f2528c00700 1 -- 172.21.15.90:0/2607121867 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2524064150 con 0x7f252410f0a0 2024-04-24T09:10:40.722 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.609+0000 7f251aa00700 1 --2- 172.21.15.90:0/2607121867 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f2508039600 0x7f250803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:40.722 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.610+0000 7f251aa00700 1 -- 172.21.15.90:0/2607121867 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2514011e00 con 0x7f252410f0a0 2024-04-24T09:10:40.722 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.610+0000 7f2521400700 1 --2- 172.21.15.90:0/2607121867 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f2508039600 0x7f250803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:40.722 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.610+0000 7f2521400700 1 --2- 172.21.15.90:0/2607121867 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f2508039600 0x7f250803bac0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2510006fd0 tx=0x7f2510006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:40.722 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.615+0000 7f251aa00700 1 -- 172.21.15.90:0/2607121867 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f251404d130 con 0x7f252410f0a0 2024-04-24T09:10:40.722 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.870+0000 7f2528c00700 1 -- 172.21.15.90:0/2607121867 --> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f252407fff0 con 0x7f2508039600 2024-04-24T09:10:40.722 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.878+0000 7f251aa00700 1 -- 172.21.15.90:0/2607121867 <== mgr.14120 v2:172.21.15.90:6800/1809038230 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f252407fff0 con 0x7f2508039600 2024-04-24T09:10:40.722 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.881+0000 7f2528c00700 1 -- 172.21.15.90:0/2607121867 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f2508039600 msgr2=0x7f250803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:40.722 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.881+0000 7f2528c00700 1 --2- 172.21.15.90:0/2607121867 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f2508039600 0x7f250803bac0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2510006fd0 tx=0x7f2510006e40 comp rx=0 tx=0).stop 2024-04-24T09:10:40.722 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.881+0000 7f2528c00700 1 -- 172.21.15.90:0/2607121867 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f252410f0a0 msgr2=0x7f25241a2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:40.723 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.881+0000 7f2528c00700 1 --2- 172.21.15.90:0/2607121867 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f252410f0a0 0x7f25241a2b40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2514000c00 tx=0x7f2514011c10 comp rx=0 tx=0).stop 2024-04-24T09:10:40.723 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.881+0000 7f2528c00700 1 -- 172.21.15.90:0/2607121867 shutdown_connections 2024-04-24T09:10:40.723 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.881+0000 7f2528c00700 1 --2- 172.21.15.90:0/2607121867 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f2508039600 0x7f250803bac0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:40.723 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.881+0000 7f2528c00700 1 --2- 172.21.15.90:0/2607121867 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f252410f0a0 0x7f25241a2b40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:40.723 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.881+0000 7f2528c00700 1 -- 172.21.15.90:0/2607121867 >> 172.21.15.90:0/2607121867 conn(0x7f252407ddc0 msgr2=0x7f252407f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:40.723 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.881+0000 7f2528c00700 1 -- 172.21.15.90:0/2607121867 shutdown_connections 2024-04-24T09:10:40.723 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:39.881+0000 7f2528c00700 1 -- 172.21.15.90:0/2607121867 wait complete. 2024-04-24T09:10:40.723 INFO:teuthology.orchestra.run.smithi090.stdout:Deploying unmanaged mgr service... 2024-04-24T09:10:41.196 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:40 smithi090 ceph-mon[27825]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:10:41.196 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:40 smithi090 ceph-mon[27825]: Saving service mon spec with placement count:5 2024-04-24T09:10:41.196 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:40 smithi090 ceph-mon[27825]: from='mgr.14120 172.21.15.90:0/738983790' entity='mgr.x' 2024-04-24T09:10:43.321 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:43 smithi090 ceph-mon[27825]: from='mgr.14120 172.21.15.90:0/738983790' entity='mgr.x' 2024-04-24T09:10:43.321 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:43 smithi090 ceph-mon[27825]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:10:43.321 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:43 smithi090 ceph-mon[27825]: Saving service mgr spec with placement count:2 2024-04-24T09:10:43.321 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:43 smithi090 ceph-mon[27825]: from='mgr.14120 172.21.15.90:0/738983790' entity='mgr.x' 2024-04-24T09:10:43.321 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:43 smithi090 ceph-mon[27825]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:10:43.321 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:43 smithi090 ceph-mon[27825]: from='mgr.14120 172.21.15.90:0/738983790' entity='mgr.x' 2024-04-24T09:10:43.331 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-24T09:10:43.332 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.271+0000 7efe20c00700 1 Processor -- start 2024-04-24T09:10:43.332 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.271+0000 7efe20c00700 1 -- start start 2024-04-24T09:10:43.332 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.272+0000 7efe20c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe1c10a640 0x7efe1c10aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:43.332 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.272+0000 7efe20c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efe1c10b030 con 0x7efe1c10a640 2024-04-24T09:10:43.332 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.272+0000 7efe1b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe1c10a640 0x7efe1c10aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:43.332 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.272+0000 7efe1b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe1c10a640 0x7efe1c10aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54278/0 (socket says 172.21.15.90:54278) 2024-04-24T09:10:43.333 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.272+0000 7efe1b400700 1 -- 172.21.15.90:0/3316540058 learned_addr learned my addr 172.21.15.90:0/3316540058 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:10:43.333 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.273+0000 7efe1b400700 1 -- 172.21.15.90:0/3316540058 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe1c10b7f0 con 0x7efe1c10a640 2024-04-24T09:10:43.333 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.273+0000 7efe1b400700 1 --2- 172.21.15.90:0/3316540058 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe1c10a640 0x7efe1c10aa60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7efe0c0098a0 tx=0x7efe0c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61b331ce704a7c01 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:43.333 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.273+0000 7efe1a000700 1 -- 172.21.15.90:0/3316540058 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe0c004160 con 0x7efe1c10a640 2024-04-24T09:10:43.333 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.273+0000 7efe1a000700 1 -- 172.21.15.90:0/3316540058 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe0c004300 con 0x7efe1c10a640 2024-04-24T09:10:43.333 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.273+0000 7efe1a000700 1 -- 172.21.15.90:0/3316540058 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe0c011b20 con 0x7efe1c10a640 2024-04-24T09:10:43.334 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.274+0000 7efe20c00700 1 -- 172.21.15.90:0/3316540058 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe1c10a640 msgr2=0x7efe1c10aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:43.334 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.274+0000 7efe20c00700 1 --2- 172.21.15.90:0/3316540058 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe1c10a640 0x7efe1c10aa60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7efe0c0098a0 tx=0x7efe0c009bb0 comp rx=0 tx=0).stop 2024-04-24T09:10:43.334 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.274+0000 7efe20c00700 1 -- 172.21.15.90:0/3316540058 shutdown_connections 2024-04-24T09:10:43.336 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.274+0000 7efe20c00700 1 --2- 172.21.15.90:0/3316540058 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe1c10a640 0x7efe1c10aa60 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:43.337 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.274+0000 7efe20c00700 1 -- 172.21.15.90:0/3316540058 >> 172.21.15.90:0/3316540058 conn(0x7efe1c07de00 msgr2=0x7efe1c1084c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:43.337 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.274+0000 7efe20c00700 1 -- 172.21.15.90:0/3316540058 shutdown_connections 2024-04-24T09:10:43.337 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.275+0000 7efe20c00700 1 -- 172.21.15.90:0/3316540058 wait complete. 2024-04-24T09:10:43.337 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.276+0000 7efe20c00700 1 Processor -- start 2024-04-24T09:10:43.337 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.276+0000 7efe20c00700 1 -- start start 2024-04-24T09:10:43.337 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.276+0000 7efe20c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe1c10a640 0x7efe1c19e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:43.337 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.276+0000 7efe20c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efe1c10eba0 con 0x7efe1c10a640 2024-04-24T09:10:43.337 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.276+0000 7efe1b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe1c10a640 0x7efe1c19e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:43.337 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.277+0000 7efe1b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe1c10a640 0x7efe1c19e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54290/0 (socket says 172.21.15.90:54290) 2024-04-24T09:10:43.337 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.277+0000 7efe1b400700 1 -- 172.21.15.90:0/161926491 learned_addr learned my addr 172.21.15.90:0/161926491 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:10:43.337 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.277+0000 7efe1b400700 1 -- 172.21.15.90:0/161926491 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe1c19eac0 con 0x7efe1c10a640 2024-04-24T09:10:43.337 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.277+0000 7efe1b400700 1 --2- 172.21.15.90:0/161926491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe1c10a640 0x7efe1c19e580 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efe0c000c00 tx=0x7efe0c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:43.337 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.277+0000 7efe13e00700 1 -- 172.21.15.90:0/161926491 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe0c003b60 con 0x7efe1c10a640 2024-04-24T09:10:43.337 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.277+0000 7efe13e00700 1 -- 172.21.15.90:0/161926491 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe0c01a910 con 0x7efe1c10a640 2024-04-24T09:10:43.337 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.278+0000 7efe20c00700 1 -- 172.21.15.90:0/161926491 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe1c19ed50 con 0x7efe1c10a640 2024-04-24T09:10:43.338 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.278+0000 7efe13e00700 1 -- 172.21.15.90:0/161926491 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe0c024930 con 0x7efe1c10a640 2024-04-24T09:10:43.338 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.278+0000 7efe20c00700 1 -- 172.21.15.90:0/161926491 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe1c19f1b0 con 0x7efe1c10a640 2024-04-24T09:10:43.338 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.279+0000 7efe20c00700 1 -- 172.21.15.90:0/161926491 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe1c064150 con 0x7efe1c10a640 2024-04-24T09:10:43.338 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.280+0000 7efe13e00700 1 -- 172.21.15.90:0/161926491 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7efe0c022020 con 0x7efe1c10a640 2024-04-24T09:10:43.338 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.280+0000 7efe13e00700 1 --2- 172.21.15.90:0/161926491 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7efe00039600 0x7efe0003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:43.338 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.280+0000 7efe13e00700 1 -- 172.21.15.90:0/161926491 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7efe0c04d9b0 con 0x7efe1c10a640 2024-04-24T09:10:43.338 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.280+0000 7efe1aa00700 1 --2- 172.21.15.90:0/161926491 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7efe00039600 0x7efe0003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:43.338 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.281+0000 7efe1aa00700 1 --2- 172.21.15.90:0/161926491 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7efe00039600 0x7efe0003bac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7efe08006fd0 tx=0x7efe08006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:43.339 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.286+0000 7efe13e00700 1 -- 172.21.15.90:0/161926491 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efe0c00b3e0 con 0x7efe1c10a640 2024-04-24T09:10:43.340 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.529+0000 7efe20c00700 1 -- 172.21.15.90:0/161926491 --> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7efe1c108320 con 0x7efe00039600 2024-04-24T09:10:43.340 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.539+0000 7efe13e00700 1 -- 172.21.15.90:0/161926491 <== mgr.14120 v2:172.21.15.90:6800/1809038230 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7efe1c108320 con 0x7efe00039600 2024-04-24T09:10:43.340 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.542+0000 7efe20c00700 1 -- 172.21.15.90:0/161926491 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7efe00039600 msgr2=0x7efe0003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:43.340 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.542+0000 7efe20c00700 1 --2- 172.21.15.90:0/161926491 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7efe00039600 0x7efe0003bac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7efe08006fd0 tx=0x7efe08006e40 comp rx=0 tx=0).stop 2024-04-24T09:10:43.340 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.542+0000 7efe20c00700 1 -- 172.21.15.90:0/161926491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe1c10a640 msgr2=0x7efe1c19e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:43.340 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.542+0000 7efe20c00700 1 --2- 172.21.15.90:0/161926491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe1c10a640 0x7efe1c19e580 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efe0c000c00 tx=0x7efe0c011c10 comp rx=0 tx=0).stop 2024-04-24T09:10:43.340 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.543+0000 7efe20c00700 1 -- 172.21.15.90:0/161926491 shutdown_connections 2024-04-24T09:10:43.340 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.543+0000 7efe20c00700 1 --2- 172.21.15.90:0/161926491 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7efe00039600 0x7efe0003bac0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:43.340 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.543+0000 7efe20c00700 1 --2- 172.21.15.90:0/161926491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe1c10a640 0x7efe1c19e580 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:43.340 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.543+0000 7efe20c00700 1 -- 172.21.15.90:0/161926491 >> 172.21.15.90:0/161926491 conn(0x7efe1c07de00 msgr2=0x7efe1c107c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:43.340 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.543+0000 7efe20c00700 1 -- 172.21.15.90:0/161926491 shutdown_connections 2024-04-24T09:10:43.340 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:42.543+0000 7efe20c00700 1 -- 172.21.15.90:0/161926491 wait complete. 2024-04-24T09:10:45.541 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:45 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/1515805631' entity='client.admin' 2024-04-24T09:10:45.542 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:45 smithi090 ceph-mon[27825]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:10:45.557 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:43.938+0000 7fdac5200700 1 Processor -- start 2024-04-24T09:10:45.558 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:43.939+0000 7fdac5200700 1 -- start start 2024-04-24T09:10:45.558 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:43.939+0000 7fdac5200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdac0108610 0x7fdac0108a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:45.558 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:43.939+0000 7fdac5200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdac0109000 con 0x7fdac0108610 2024-04-24T09:10:45.558 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:43.940+0000 7fdabde00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdac0108610 0x7fdac0108a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:45.558 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:43.940+0000 7fdabde00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdac0108610 0x7fdac0108a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54302/0 (socket says 172.21.15.90:54302) 2024-04-24T09:10:45.558 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:43.940+0000 7fdabde00700 1 -- 172.21.15.90:0/45414391 learned_addr learned my addr 172.21.15.90:0/45414391 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:10:45.558 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:43.940+0000 7fdabde00700 1 -- 172.21.15.90:0/45414391 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdac01097c0 con 0x7fdac0108610 2024-04-24T09:10:45.558 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:43.941+0000 7fdabde00700 1 --2- 172.21.15.90:0/45414391 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdac0108610 0x7fdac0108a30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdab00098a0 tx=0x7fdab0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de304e37f200b896 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:45.558 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:43.941+0000 7fdabca00700 1 -- 172.21.15.90:0/45414391 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdab0004160 con 0x7fdac0108610 2024-04-24T09:10:45.558 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:43.941+0000 7fdabca00700 1 -- 172.21.15.90:0/45414391 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdab0004300 con 0x7fdac0108610 2024-04-24T09:10:45.558 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:43.941+0000 7fdabca00700 1 -- 172.21.15.90:0/45414391 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdab0011b20 con 0x7fdac0108610 2024-04-24T09:10:45.559 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:43.942+0000 7fdac5200700 1 -- 172.21.15.90:0/45414391 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdac0108610 msgr2=0x7fdac0108a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:45.559 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:43.942+0000 7fdac5200700 1 --2- 172.21.15.90:0/45414391 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdac0108610 0x7fdac0108a30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdab00098a0 tx=0x7fdab0009bb0 comp rx=0 tx=0).stop 2024-04-24T09:10:45.559 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:43.942+0000 7fdac5200700 1 -- 172.21.15.90:0/45414391 shutdown_connections 2024-04-24T09:10:45.559 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:43.942+0000 7fdac5200700 1 --2- 172.21.15.90:0/45414391 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdac0108610 0x7fdac0108a30 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:45.559 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:43.942+0000 7fdac5200700 1 -- 172.21.15.90:0/45414391 >> 172.21.15.90:0/45414391 conn(0x7fdac0103c10 msgr2=0x7fdac0106030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:45.559 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:43.942+0000 7fdac5200700 1 -- 172.21.15.90:0/45414391 shutdown_connections 2024-04-24T09:10:45.559 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:43.942+0000 7fdac5200700 1 -- 172.21.15.90:0/45414391 wait complete. 2024-04-24T09:10:45.559 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:43.944+0000 7fdac5200700 1 Processor -- start 2024-04-24T09:10:45.559 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:43.944+0000 7fdac5200700 1 -- start start 2024-04-24T09:10:45.559 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:43.944+0000 7fdac5200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdac0108610 0x7fdac019e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:45.559 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:43.944+0000 7fdac5200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdac010cb70 con 0x7fdac0108610 2024-04-24T09:10:45.559 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:43.944+0000 7fdabde00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdac0108610 0x7fdac019e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:45.559 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:43.944+0000 7fdabde00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdac0108610 0x7fdac019e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54316/0 (socket says 172.21.15.90:54316) 2024-04-24T09:10:45.560 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:43.944+0000 7fdabde00700 1 -- 172.21.15.90:0/1515805631 learned_addr learned my addr 172.21.15.90:0/1515805631 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:10:45.560 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:43.945+0000 7fdabde00700 1 -- 172.21.15.90:0/1515805631 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdac019ecc0 con 0x7fdac0108610 2024-04-24T09:10:45.560 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:43.945+0000 7fdabde00700 1 --2- 172.21.15.90:0/1515805631 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdac0108610 0x7fdac019e780 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdab0000c00 tx=0x7fdab0011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:45.560 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:43.945+0000 7fdab6a00700 1 -- 172.21.15.90:0/1515805631 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdab0003b60 con 0x7fdac0108610 2024-04-24T09:10:45.560 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:43.945+0000 7fdab6a00700 1 -- 172.21.15.90:0/1515805631 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdab001a910 con 0x7fdac0108610 2024-04-24T09:10:45.560 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:43.945+0000 7fdac5200700 1 -- 172.21.15.90:0/1515805631 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdac019ef50 con 0x7fdac0108610 2024-04-24T09:10:45.560 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:43.945+0000 7fdab6a00700 1 -- 172.21.15.90:0/1515805631 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdab0024930 con 0x7fdac0108610 2024-04-24T09:10:45.560 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:43.946+0000 7fdac5200700 1 -- 172.21.15.90:0/1515805631 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdac019f3b0 con 0x7fdac0108610 2024-04-24T09:10:45.560 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:43.947+0000 7fdab6a00700 1 -- 172.21.15.90:0/1515805631 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fdab0022020 con 0x7fdac0108610 2024-04-24T09:10:45.560 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:43.947+0000 7fdac5200700 1 -- 172.21.15.90:0/1515805631 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdac0064150 con 0x7fdac0108610 2024-04-24T09:10:45.560 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:43.947+0000 7fdab6a00700 1 --2- 172.21.15.90:0/1515805631 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7fdaa4039600 0x7fdaa403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:45.560 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:43.947+0000 7fdab6a00700 1 -- 172.21.15.90:0/1515805631 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdab004dc20 con 0x7fdac0108610 2024-04-24T09:10:45.560 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:43.948+0000 7fdabd400700 1 --2- 172.21.15.90:0/1515805631 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7fdaa4039600 0x7fdaa403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:45.561 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:43.948+0000 7fdabd400700 1 --2- 172.21.15.90:0/1515805631 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7fdaa4039600 0x7fdaa403bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdaac006fd0 tx=0x7fdaac006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:45.561 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:43.953+0000 7fdab6a00700 1 -- 172.21.15.90:0/1515805631 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdab004d5b0 con 0x7fdac0108610 2024-04-24T09:10:45.561 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:44.195+0000 7fdac5200700 1 -- 172.21.15.90:0/1515805631 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fdac01a1d00 con 0x7fdac0108610 2024-04-24T09:10:45.561 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:44.199+0000 7fdab6a00700 1 -- 172.21.15.90:0/1515805631 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fdab000fdd0 con 0x7fdac0108610 2024-04-24T09:10:45.561 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:44.202+0000 7fdac5200700 1 -- 172.21.15.90:0/1515805631 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7fdaa4039600 msgr2=0x7fdaa403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:45.561 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:44.202+0000 7fdac5200700 1 --2- 172.21.15.90:0/1515805631 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7fdaa4039600 0x7fdaa403bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdaac006fd0 tx=0x7fdaac006e40 comp rx=0 tx=0).stop 2024-04-24T09:10:45.561 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:44.202+0000 7fdac5200700 1 -- 172.21.15.90:0/1515805631 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdac0108610 msgr2=0x7fdac019e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:45.561 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:44.202+0000 7fdac5200700 1 --2- 172.21.15.90:0/1515805631 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdac0108610 0x7fdac019e780 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdab0000c00 tx=0x7fdab0011c10 comp rx=0 tx=0).stop 2024-04-24T09:10:45.561 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:44.202+0000 7fdac5200700 1 -- 172.21.15.90:0/1515805631 shutdown_connections 2024-04-24T09:10:45.561 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:44.202+0000 7fdac5200700 1 --2- 172.21.15.90:0/1515805631 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7fdaa4039600 0x7fdaa403bac0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:45.561 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:44.202+0000 7fdac5200700 1 --2- 172.21.15.90:0/1515805631 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdac0108610 0x7fdac019e780 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:45.561 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:44.202+0000 7fdac5200700 1 -- 172.21.15.90:0/1515805631 >> 172.21.15.90:0/1515805631 conn(0x7fdac0103c10 msgr2=0x7fdac01046e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:45.561 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:44.203+0000 7fdac5200700 1 -- 172.21.15.90:0/1515805631 shutdown_connections 2024-04-24T09:10:45.562 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:44.203+0000 7fdac5200700 1 -- 172.21.15.90:0/1515805631 wait complete. 2024-04-24T09:10:46.426 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:46 smithi090 ceph-mon[27825]: from='mgr.14120 172.21.15.90:0/738983790' entity='mgr.x' 2024-04-24T09:10:47.188 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.162+0000 7fa0cbe00700 1 Processor -- start 2024-04-24T09:10:47.188 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.162+0000 7fa0cbe00700 1 -- start start 2024-04-24T09:10:47.188 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.162+0000 7fa0cbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0c4109110 0x7fa0c410b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:47.188 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.162+0000 7fa0cbe00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c407bde0 con 0x7fa0c4109110 2024-04-24T09:10:47.188 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.163+0000 7fa0c9400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0c4109110 0x7fa0c410b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:47.188 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.163+0000 7fa0c9400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0c4109110 0x7fa0c410b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38892/0 (socket says 172.21.15.90:38892) 2024-04-24T09:10:47.188 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.163+0000 7fa0c9400700 1 -- 172.21.15.90:0/1024065054 learned_addr learned my addr 172.21.15.90:0/1024065054 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:10:47.188 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.163+0000 7fa0c9400700 1 -- 172.21.15.90:0/1024065054 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0c410ba40 con 0x7fa0c4109110 2024-04-24T09:10:47.188 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.163+0000 7fa0c9400700 1 --2- 172.21.15.90:0/1024065054 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0c4109110 0x7fa0c410b500 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa0b80098a0 tx=0x7fa0b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=302171cf06e8879 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:47.189 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.164+0000 7fa0c3e00700 1 -- 172.21.15.90:0/1024065054 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0b8004160 con 0x7fa0c4109110 2024-04-24T09:10:47.189 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.164+0000 7fa0c3e00700 1 -- 172.21.15.90:0/1024065054 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0b8004300 con 0x7fa0c4109110 2024-04-24T09:10:47.189 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.164+0000 7fa0c3e00700 1 -- 172.21.15.90:0/1024065054 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0b8011b20 con 0x7fa0c4109110 2024-04-24T09:10:47.189 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.165+0000 7fa0cbe00700 1 -- 172.21.15.90:0/1024065054 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0c4109110 msgr2=0x7fa0c410b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:47.189 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.165+0000 7fa0cbe00700 1 --2- 172.21.15.90:0/1024065054 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0c4109110 0x7fa0c410b500 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa0b80098a0 tx=0x7fa0b8009bb0 comp rx=0 tx=0).stop 2024-04-24T09:10:47.189 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.165+0000 7fa0cbe00700 1 -- 172.21.15.90:0/1024065054 shutdown_connections 2024-04-24T09:10:47.189 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.165+0000 7fa0cbe00700 1 --2- 172.21.15.90:0/1024065054 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0c4109110 0x7fa0c410b500 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:47.189 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.165+0000 7fa0cbe00700 1 -- 172.21.15.90:0/1024065054 >> 172.21.15.90:0/1024065054 conn(0x7fa0c4102ae0 msgr2=0x7fa0c4104f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:47.189 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.165+0000 7fa0cbe00700 1 -- 172.21.15.90:0/1024065054 shutdown_connections 2024-04-24T09:10:47.189 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.165+0000 7fa0cbe00700 1 -- 172.21.15.90:0/1024065054 wait complete. 2024-04-24T09:10:47.189 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.166+0000 7fa0cbe00700 1 Processor -- start 2024-04-24T09:10:47.189 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.166+0000 7fa0cbe00700 1 -- start start 2024-04-24T09:10:47.189 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.166+0000 7fa0cbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0c4109110 0x7fa0c4108990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:47.189 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.166+0000 7fa0cbe00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c410c9f0 con 0x7fa0c4109110 2024-04-24T09:10:47.190 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.167+0000 7fa0c9400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0c4109110 0x7fa0c4108990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:47.190 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.167+0000 7fa0c9400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0c4109110 0x7fa0c4108990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38904/0 (socket says 172.21.15.90:38904) 2024-04-24T09:10:47.190 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.167+0000 7fa0c9400700 1 -- 172.21.15.90:0/3361098504 learned_addr learned my addr 172.21.15.90:0/3361098504 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:10:47.190 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.167+0000 7fa0c9400700 1 -- 172.21.15.90:0/3361098504 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0c4108ed0 con 0x7fa0c4109110 2024-04-24T09:10:47.190 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.167+0000 7fa0c9400700 1 --2- 172.21.15.90:0/3361098504 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0c4109110 0x7fa0c4108990 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa0b80117e0 tx=0x7fa0b8011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:47.190 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.168+0000 7fa0c2000700 1 -- 172.21.15.90:0/3361098504 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0b8003ba0 con 0x7fa0c4109110 2024-04-24T09:10:47.190 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.168+0000 7fa0c2000700 1 -- 172.21.15.90:0/3361098504 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0b801a910 con 0x7fa0c4109110 2024-04-24T09:10:47.190 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.168+0000 7fa0cbe00700 1 -- 172.21.15.90:0/3361098504 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0c41070c0 con 0x7fa0c4109110 2024-04-24T09:10:47.190 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.168+0000 7fa0c2000700 1 -- 172.21.15.90:0/3361098504 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0b80248f0 con 0x7fa0c4109110 2024-04-24T09:10:47.190 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.168+0000 7fa0cbe00700 1 -- 172.21.15.90:0/3361098504 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0c4107580 con 0x7fa0c4109110 2024-04-24T09:10:47.190 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.169+0000 7fa0c2000700 1 -- 172.21.15.90:0/3361098504 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fa0b8022020 con 0x7fa0c4109110 2024-04-24T09:10:47.190 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.169+0000 7fa0cbe00700 1 -- 172.21.15.90:0/3361098504 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0c4064150 con 0x7fa0c4109110 2024-04-24T09:10:47.190 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.169+0000 7fa0c2000700 1 --2- 172.21.15.90:0/3361098504 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7fa0ac039600 0x7fa0ac03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:47.190 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.169+0000 7fa0c2000700 1 -- 172.21.15.90:0/3361098504 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa0b804db00 con 0x7fa0c4109110 2024-04-24T09:10:47.191 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.170+0000 7fa0c8a00700 1 --2- 172.21.15.90:0/3361098504 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7fa0ac039600 0x7fa0ac03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:47.191 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.170+0000 7fa0c8a00700 1 --2- 172.21.15.90:0/3361098504 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7fa0ac039600 0x7fa0ac03bac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0006fd0 tx=0x7fa0b0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:47.191 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.175+0000 7fa0c2000700 1 -- 172.21.15.90:0/3361098504 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa0b801ab70 con 0x7fa0c4109110 2024-04-24T09:10:47.191 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.426+0000 7fa0cbe00700 1 -- 172.21.15.90:0/3361098504 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fa0c4107800 con 0x7fa0c4109110 2024-04-24T09:10:47.191 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.432+0000 7fa0c2000700 1 -- 172.21.15.90:0/3361098504 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fa0b804de20 con 0x7fa0c4109110 2024-04-24T09:10:47.191 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.436+0000 7fa0cbe00700 1 -- 172.21.15.90:0/3361098504 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7fa0ac039600 msgr2=0x7fa0ac03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:47.191 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.436+0000 7fa0cbe00700 1 --2- 172.21.15.90:0/3361098504 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7fa0ac039600 0x7fa0ac03bac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0006fd0 tx=0x7fa0b0006e40 comp rx=0 tx=0).stop 2024-04-24T09:10:47.191 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.436+0000 7fa0cbe00700 1 -- 172.21.15.90:0/3361098504 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0c4109110 msgr2=0x7fa0c4108990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:47.191 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.436+0000 7fa0cbe00700 1 --2- 172.21.15.90:0/3361098504 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0c4109110 0x7fa0c4108990 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa0b80117e0 tx=0x7fa0b8011ba0 comp rx=0 tx=0).stop 2024-04-24T09:10:47.191 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.436+0000 7fa0cbe00700 1 -- 172.21.15.90:0/3361098504 shutdown_connections 2024-04-24T09:10:47.191 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.436+0000 7fa0cbe00700 1 --2- 172.21.15.90:0/3361098504 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7fa0ac039600 0x7fa0ac03bac0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:47.191 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.436+0000 7fa0cbe00700 1 --2- 172.21.15.90:0/3361098504 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0c4109110 0x7fa0c4108990 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:47.191 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.437+0000 7fa0cbe00700 1 -- 172.21.15.90:0/3361098504 >> 172.21.15.90:0/3361098504 conn(0x7fa0c4102ae0 msgr2=0x7fa0c4103530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:47.192 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.437+0000 7fa0cbe00700 1 -- 172.21.15.90:0/3361098504 shutdown_connections 2024-04-24T09:10:47.192 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:46.437+0000 7fa0cbe00700 1 -- 172.21.15.90:0/3361098504 wait complete. 2024-04-24T09:10:47.192 INFO:teuthology.orchestra.run.smithi090.stdout:Enabling the dashboard module... 2024-04-24T09:10:47.705 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:47 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/3361098504' entity='client.admin' 2024-04-24T09:10:47.705 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:47 smithi090 ceph-mon[27825]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:10:48.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:48 smithi090 ceph-mon[27825]: from='mgr.14120 172.21.15.90:0/738983790' entity='mgr.x' 2024-04-24T09:10:48.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:48 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/1626447757' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-24T09:10:48.852 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:48 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: ignoring --setuser ceph since I am not root 2024-04-24T09:10:48.852 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:48 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: ignoring --setgroup ceph since I am not root 2024-04-24T09:10:48.852 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:48 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:48.780+0000 7f2dfa4c9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T09:10:49.162 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:49 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:49.031+0000 7f2dfa4c9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T09:10:49.162 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:49 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:49.161+0000 7f2dfa4c9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T09:10:49.851 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:49 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/1626447757' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-24T09:10:49.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:49 smithi090 ceph-mon[27825]: mgrmap e9: x(active, since 25s) 2024-04-24T09:10:49.852 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:49 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:49.557+0000 7f2dfa4c9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T09:10:49.852 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:49 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:49.669+0000 7f2dfa4c9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T09:10:50.308 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:47.767+0000 7f3e52000700 1 Processor -- start 2024-04-24T09:10:50.310 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:47.767+0000 7f3e52000700 1 -- start start 2024-04-24T09:10:50.311 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:47.767+0000 7f3e52000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e4c109110 0x7f3e4c10b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:50.311 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:47.767+0000 7f3e52000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4c07bde0 con 0x7f3e4c109110 2024-04-24T09:10:50.311 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:47.768+0000 7f3e4b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e4c109110 0x7f3e4c10b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:50.311 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:47.768+0000 7f3e4b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e4c109110 0x7f3e4c10b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38908/0 (socket says 172.21.15.90:38908) 2024-04-24T09:10:50.311 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:47.768+0000 7f3e4b400700 1 -- 172.21.15.90:0/2305470380 learned_addr learned my addr 172.21.15.90:0/2305470380 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:10:50.311 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:47.768+0000 7f3e4b400700 1 -- 172.21.15.90:0/2305470380 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e4c10ba40 con 0x7f3e4c109110 2024-04-24T09:10:50.311 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:47.768+0000 7f3e4b400700 1 --2- 172.21.15.90:0/2305470380 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e4c109110 0x7f3e4c10b500 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c0098a0 tx=0x7f3e3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb04dc748c3b2acb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:50.311 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:47.769+0000 7f3e4a000700 1 -- 172.21.15.90:0/2305470380 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e3c004160 con 0x7f3e4c109110 2024-04-24T09:10:50.311 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:47.769+0000 7f3e4a000700 1 -- 172.21.15.90:0/2305470380 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e3c004300 con 0x7f3e4c109110 2024-04-24T09:10:50.311 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:47.769+0000 7f3e4a000700 1 -- 172.21.15.90:0/2305470380 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e3c011b20 con 0x7f3e4c109110 2024-04-24T09:10:50.311 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:47.769+0000 7f3e52000700 1 -- 172.21.15.90:0/2305470380 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e4c109110 msgr2=0x7f3e4c10b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:50.311 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:47.769+0000 7f3e52000700 1 --2- 172.21.15.90:0/2305470380 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e4c109110 0x7f3e4c10b500 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c0098a0 tx=0x7f3e3c009bb0 comp rx=0 tx=0).stop 2024-04-24T09:10:50.312 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:47.770+0000 7f3e52000700 1 -- 172.21.15.90:0/2305470380 shutdown_connections 2024-04-24T09:10:50.312 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:47.770+0000 7f3e52000700 1 --2- 172.21.15.90:0/2305470380 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e4c109110 0x7f3e4c10b500 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:50.312 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:47.770+0000 7f3e52000700 1 -- 172.21.15.90:0/2305470380 >> 172.21.15.90:0/2305470380 conn(0x7f3e4c102ae0 msgr2=0x7f3e4c104f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:50.312 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:47.770+0000 7f3e52000700 1 -- 172.21.15.90:0/2305470380 shutdown_connections 2024-04-24T09:10:50.313 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:47.770+0000 7f3e52000700 1 -- 172.21.15.90:0/2305470380 wait complete. 2024-04-24T09:10:50.313 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:47.771+0000 7f3e52000700 1 Processor -- start 2024-04-24T09:10:50.313 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:47.771+0000 7f3e52000700 1 -- start start 2024-04-24T09:10:50.313 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:47.771+0000 7f3e52000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e4c109110 0x7f3e4c19e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:50.313 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:47.771+0000 7f3e52000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4c10c9f0 con 0x7f3e4c109110 2024-04-24T09:10:50.313 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:47.771+0000 7f3e4b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e4c109110 0x7f3e4c19e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:50.313 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:47.772+0000 7f3e4b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e4c109110 0x7f3e4c19e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38910/0 (socket says 172.21.15.90:38910) 2024-04-24T09:10:50.314 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:47.772+0000 7f3e4b400700 1 -- 172.21.15.90:0/1626447757 learned_addr learned my addr 172.21.15.90:0/1626447757 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:10:50.314 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:47.772+0000 7f3e4b400700 1 -- 172.21.15.90:0/1626447757 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e4c19ec20 con 0x7f3e4c109110 2024-04-24T09:10:50.314 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:47.772+0000 7f3e4b400700 1 --2- 172.21.15.90:0/1626447757 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e4c109110 0x7f3e4c19e6e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c0117e0 tx=0x7f3e3c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:50.314 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:47.772+0000 7f3e43e00700 1 -- 172.21.15.90:0/1626447757 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e3c003ba0 con 0x7f3e4c109110 2024-04-24T09:10:50.319 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:47.772+0000 7f3e43e00700 1 -- 172.21.15.90:0/1626447757 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e3c01a910 con 0x7f3e4c109110 2024-04-24T09:10:50.320 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:47.772+0000 7f3e43e00700 1 -- 172.21.15.90:0/1626447757 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e3c0248f0 con 0x7f3e4c109110 2024-04-24T09:10:50.320 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:47.772+0000 7f3e52000700 1 -- 172.21.15.90:0/1626447757 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e4c19eeb0 con 0x7f3e4c109110 2024-04-24T09:10:50.320 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:47.773+0000 7f3e52000700 1 -- 172.21.15.90:0/1626447757 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e4c19f310 con 0x7f3e4c109110 2024-04-24T09:10:50.320 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:47.774+0000 7f3e43e00700 1 -- 172.21.15.90:0/1626447757 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f3e3c022020 con 0x7f3e4c109110 2024-04-24T09:10:50.320 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:47.774+0000 7f3e52000700 1 -- 172.21.15.90:0/1626447757 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e4c064150 con 0x7f3e4c109110 2024-04-24T09:10:50.320 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:47.774+0000 7f3e43e00700 1 --2- 172.21.15.90:0/1626447757 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f3e300395b0 0x7f3e3003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:50.321 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:47.774+0000 7f3e43e00700 1 -- 172.21.15.90:0/1626447757 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3e3c04dc00 con 0x7f3e4c109110 2024-04-24T09:10:50.321 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:47.774+0000 7f3e4aa00700 1 --2- 172.21.15.90:0/1626447757 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f3e300395b0 0x7f3e3003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:50.321 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:47.775+0000 7f3e4aa00700 1 --2- 172.21.15.90:0/1626447757 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f3e300395b0 0x7f3e3003ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3e38006fd0 tx=0x7f3e38006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:50.321 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:47.780+0000 7f3e43e00700 1 -- 172.21.15.90:0/1626447757 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3e3c00b3e0 con 0x7f3e4c109110 2024-04-24T09:10:50.321 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:48.070+0000 7f3e52000700 1 -- 172.21.15.90:0/1626447757 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f3e4c1a1ce0 con 0x7f3e4c109110 2024-04-24T09:10:50.321 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:48.477+0000 7f3e43e00700 1 -- 172.21.15.90:0/1626447757 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f3e3c020c10 con 0x7f3e4c109110 2024-04-24T09:10:50.321 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:48.482+0000 7f3e43e00700 1 -- 172.21.15.90:0/1626447757 <== mon.0 v2:172.21.15.90:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f3e3c018350 con 0x7f3e4c109110 2024-04-24T09:10:50.322 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:48.485+0000 7f3e52000700 1 -- 172.21.15.90:0/1626447757 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f3e300395b0 msgr2=0x7f3e3003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:50.322 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:48.485+0000 7f3e52000700 1 --2- 172.21.15.90:0/1626447757 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f3e300395b0 0x7f3e3003ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3e38006fd0 tx=0x7f3e38006e40 comp rx=0 tx=0).stop 2024-04-24T09:10:50.322 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:48.485+0000 7f3e52000700 1 -- 172.21.15.90:0/1626447757 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e4c109110 msgr2=0x7f3e4c19e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:50.322 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:48.485+0000 7f3e52000700 1 --2- 172.21.15.90:0/1626447757 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e4c109110 0x7f3e4c19e6e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c0117e0 tx=0x7f3e3c011ba0 comp rx=0 tx=0).stop 2024-04-24T09:10:50.322 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:48.485+0000 7f3e52000700 1 -- 172.21.15.90:0/1626447757 shutdown_connections 2024-04-24T09:10:50.322 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:48.485+0000 7f3e52000700 1 --2- 172.21.15.90:0/1626447757 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f3e300395b0 0x7f3e3003ba70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:50.322 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:48.485+0000 7f3e52000700 1 --2- 172.21.15.90:0/1626447757 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e4c109110 0x7f3e4c19e6e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:50.322 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:48.485+0000 7f3e52000700 1 -- 172.21.15.90:0/1626447757 >> 172.21.15.90:0/1626447757 conn(0x7f3e4c102ae0 msgr2=0x7f3e4c1034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:50.323 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:48.485+0000 7f3e52000700 1 -- 172.21.15.90:0/1626447757 shutdown_connections 2024-04-24T09:10:50.323 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:48.485+0000 7f3e52000700 1 -- 172.21.15.90:0/1626447757 wait complete. 2024-04-24T09:10:51.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:51 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/1057237002' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-24T09:10:51.602 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:51 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:51.235+0000 7f2dfa4c9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T09:10:51.602 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:51 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:51.464+0000 7f2dfa4c9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T09:10:52.036 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:51 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:51.679+0000 7f2dfa4c9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T09:10:52.048 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-04-24T09:10:52.048 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-24T09:10:52.048 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-24T09:10:52.048 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-24T09:10:52.048 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-24T09:10:52.048 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-04-24T09:10:52.048 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:50.945+0000 7f0643400700 1 Processor -- start 2024-04-24T09:10:52.048 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:50.945+0000 7f0643400700 1 -- start start 2024-04-24T09:10:52.048 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:50.945+0000 7f0643400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f063c108610 0x7f063c108a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:52.048 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:50.945+0000 7f0643400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f063c109000 con 0x7f063c108610 2024-04-24T09:10:52.048 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:50.946+0000 7f0640a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f063c108610 0x7f063c108a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:52.048 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:50.946+0000 7f0640a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f063c108610 0x7f063c108a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38938/0 (socket says 172.21.15.90:38938) 2024-04-24T09:10:52.048 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:50.946+0000 7f0640a00700 1 -- 172.21.15.90:0/3091075669 learned_addr learned my addr 172.21.15.90:0/3091075669 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:10:52.049 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:50.946+0000 7f0640a00700 1 -- 172.21.15.90:0/3091075669 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f063c1097c0 con 0x7f063c108610 2024-04-24T09:10:52.049 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:50.947+0000 7f0640a00700 1 --2- 172.21.15.90:0/3091075669 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f063c108610 0x7f063c108a30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f06300098a0 tx=0x7f0630009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d04302b97dc71c3c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:52.049 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:50.947+0000 7f063b400700 1 -- 172.21.15.90:0/3091075669 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0630004160 con 0x7f063c108610 2024-04-24T09:10:52.052 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:50.947+0000 7f063b400700 1 -- 172.21.15.90:0/3091075669 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0630004300 con 0x7f063c108610 2024-04-24T09:10:52.052 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:50.947+0000 7f063b400700 1 -- 172.21.15.90:0/3091075669 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0630011b20 con 0x7f063c108610 2024-04-24T09:10:52.052 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:50.948+0000 7f0643400700 1 -- 172.21.15.90:0/3091075669 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f063c108610 msgr2=0x7f063c108a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:52.052 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:50.948+0000 7f0643400700 1 --2- 172.21.15.90:0/3091075669 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f063c108610 0x7f063c108a30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f06300098a0 tx=0x7f0630009bb0 comp rx=0 tx=0).stop 2024-04-24T09:10:52.053 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:50.948+0000 7f0643400700 1 -- 172.21.15.90:0/3091075669 shutdown_connections 2024-04-24T09:10:52.053 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:50.948+0000 7f0643400700 1 --2- 172.21.15.90:0/3091075669 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f063c108610 0x7f063c108a30 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:52.053 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:50.948+0000 7f0643400700 1 -- 172.21.15.90:0/3091075669 >> 172.21.15.90:0/3091075669 conn(0x7f063c103c10 msgr2=0x7f063c106030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:52.053 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:50.948+0000 7f0643400700 1 -- 172.21.15.90:0/3091075669 shutdown_connections 2024-04-24T09:10:52.053 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:50.948+0000 7f0643400700 1 -- 172.21.15.90:0/3091075669 wait complete. 2024-04-24T09:10:52.053 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:50.949+0000 7f0643400700 1 Processor -- start 2024-04-24T09:10:52.053 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:50.949+0000 7f0643400700 1 -- start start 2024-04-24T09:10:52.053 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:50.950+0000 7f0643400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f063c108610 0x7f063c19e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:52.053 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:50.950+0000 7f0643400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f063c10cb70 con 0x7f063c108610 2024-04-24T09:10:52.053 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:50.950+0000 7f0640a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f063c108610 0x7f063c19e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:52.053 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:50.950+0000 7f0640a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f063c108610 0x7f063c19e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38952/0 (socket says 172.21.15.90:38952) 2024-04-24T09:10:52.053 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:50.950+0000 7f0640a00700 1 -- 172.21.15.90:0/1057237002 learned_addr learned my addr 172.21.15.90:0/1057237002 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:10:52.053 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:50.950+0000 7f0640a00700 1 -- 172.21.15.90:0/1057237002 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f063c19ecc0 con 0x7f063c108610 2024-04-24T09:10:52.053 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:50.951+0000 7f0640a00700 1 --2- 172.21.15.90:0/1057237002 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f063c108610 0x7f063c19e780 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f06300116a0 tx=0x7f0630011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:52.053 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:50.951+0000 7f0639600700 1 -- 172.21.15.90:0/1057237002 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0630003a40 con 0x7f063c108610 2024-04-24T09:10:52.054 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:50.951+0000 7f0639600700 1 -- 172.21.15.90:0/1057237002 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0630003be0 con 0x7f063c108610 2024-04-24T09:10:52.054 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:50.951+0000 7f0643400700 1 -- 172.21.15.90:0/1057237002 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f063c19ef50 con 0x7f063c108610 2024-04-24T09:10:52.054 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:50.951+0000 7f0639600700 1 -- 172.21.15.90:0/1057237002 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f063001a720 con 0x7f063c108610 2024-04-24T09:10:52.054 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:50.951+0000 7f0643400700 1 -- 172.21.15.90:0/1057237002 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f063c19f3b0 con 0x7f063c108610 2024-04-24T09:10:52.054 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:50.952+0000 7f0643400700 1 -- 172.21.15.90:0/1057237002 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f063c064150 con 0x7f063c108610 2024-04-24T09:10:52.054 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:50.953+0000 7f0639600700 1 -- 172.21.15.90:0/1057237002 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f0630024430 con 0x7f063c108610 2024-04-24T09:10:52.054 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:50.953+0000 7f0639600700 1 --2- 172.21.15.90:0/1057237002 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f0624039610 0x7f062403bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:52.054 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:50.953+0000 7f0639600700 1 -- 172.21.15.90:0/1057237002 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f063004dd00 con 0x7f063c108610 2024-04-24T09:10:52.054 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:50.953+0000 7f063be00700 1 -- 172.21.15.90:0/1057237002 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f0624039610 msgr2=0x7f062403bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/1809038230 2024-04-24T09:10:52.054 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:50.953+0000 7f063be00700 1 --2- 172.21.15.90:0/1057237002 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f0624039610 0x7f062403bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T09:10:52.054 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:50.959+0000 7f0639600700 1 -- 172.21.15.90:0/1057237002 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0630011e20 con 0x7f063c108610 2024-04-24T09:10:52.054 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:51.154+0000 7f063be00700 1 -- 172.21.15.90:0/1057237002 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f0624039610 msgr2=0x7f062403bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/1809038230 2024-04-24T09:10:52.054 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:51.154+0000 7f063be00700 1 --2- 172.21.15.90:0/1057237002 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f0624039610 0x7f062403bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T09:10:52.054 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:51.239+0000 7f0643400700 1 -- 172.21.15.90:0/1057237002 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f063c1a1d60 con 0x7f063c108610 2024-04-24T09:10:52.054 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:51.240+0000 7f0639600700 1 -- 172.21.15.90:0/1057237002 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f063004d650 con 0x7f063c108610 2024-04-24T09:10:52.055 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:51.242+0000 7f0643400700 1 -- 172.21.15.90:0/1057237002 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f0624039610 msgr2=0x7f062403bad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T09:10:52.055 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:51.242+0000 7f0643400700 1 --2- 172.21.15.90:0/1057237002 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f0624039610 0x7f062403bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:52.055 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:51.242+0000 7f0643400700 1 -- 172.21.15.90:0/1057237002 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f063c108610 msgr2=0x7f063c19e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:52.055 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:51.242+0000 7f0643400700 1 --2- 172.21.15.90:0/1057237002 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f063c108610 0x7f063c19e780 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f06300116a0 tx=0x7f0630011ba0 comp rx=0 tx=0).stop 2024-04-24T09:10:52.055 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:51.242+0000 7f0643400700 1 -- 172.21.15.90:0/1057237002 shutdown_connections 2024-04-24T09:10:52.055 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:51.242+0000 7f0643400700 1 --2- 172.21.15.90:0/1057237002 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f0624039610 0x7f062403bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:52.055 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:51.242+0000 7f0643400700 1 --2- 172.21.15.90:0/1057237002 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f063c108610 0x7f063c19e780 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:52.055 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:51.242+0000 7f0643400700 1 -- 172.21.15.90:0/1057237002 >> 172.21.15.90:0/1057237002 conn(0x7f063c103c10 msgr2=0x7f063c104660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:52.055 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:51.243+0000 7f0643400700 1 -- 172.21.15.90:0/1057237002 shutdown_connections 2024-04-24T09:10:52.056 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:51.243+0000 7f0643400700 1 -- 172.21.15.90:0/1057237002 wait complete. 2024-04-24T09:10:52.056 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for the mgr to restart... 2024-04-24T09:10:52.056 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for mgr epoch 9... 2024-04-24T09:10:52.340 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:52 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:52.062+0000 7f2dfa4c9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T09:10:52.340 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:52 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:52.186+0000 7f2dfa4c9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T09:10:52.601 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:52 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:52.443+0000 7f2dfa4c9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T09:10:53.102 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:52 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:52.703+0000 7f2dfa4c9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T09:10:53.852 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:53 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:53.490+0000 7f2dfa4c9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T09:10:54.301 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:54 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:54.043+0000 7f2dfa4c9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T09:10:54.301 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:54 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:54.171+0000 7f2dfa4c9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T09:10:54.602 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:54 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:54.449+0000 7f2dfa4c9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T09:10:55.352 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:55 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:55.088+0000 7f2dfa4c9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T09:10:55.352 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:55 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:55.212+0000 7f2dfa4c9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T09:10:55.831 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:55 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:55.528+0000 7f2dfa4c9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T09:10:55.831 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:55 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:55.705+0000 7f2dfa4c9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T09:10:55.831 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:55 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:55.830+0000 7f2dfa4c9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T09:10:56.102 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:55 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:55.982+0000 7f2dfa4c9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T09:10:56.602 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:56 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:56.345+0000 7f2dfa4c9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T09:10:57.351 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:57 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:57.090+0000 7f2dfa4c9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T09:10:57.852 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:10:57 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:10:57.373+0000 7f2dfa4c9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T09:10:57.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:57 smithi090 ceph-mon[27825]: Active manager daemon x restarted 2024-04-24T09:10:57.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:57 smithi090 ceph-mon[27825]: Activating manager daemon x 2024-04-24T09:10:57.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:57 smithi090 ceph-mon[27825]: osdmap e3: 0 total, 0 up, 0 in 2024-04-24T09:10:57.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:57 smithi090 ceph-mon[27825]: mgrmap e10: x(active, starting, since 0.00797234s) 2024-04-24T09:10:57.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:57 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T09:10:57.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:57 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-24T09:10:57.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:57 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T09:10:57.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:57 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T09:10:57.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:57 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T09:10:57.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:57 smithi090 ceph-mon[27825]: Manager daemon x is now available 2024-04-24T09:10:58.735 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:58 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:10:58.736 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:58 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T09:10:58.736 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:58 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T09:10:58.736 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:58 smithi090 ceph-mon[27825]: mgrmap e11: x(active, since 1.0098s) 2024-04-24T09:10:59.445 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-04-24T09:10:59.445 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-24T09:10:59.445 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-24T09:10:59.445 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-04-24T09:10:59.454 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:52.628+0000 7f35ab800700 1 Processor -- start 2024-04-24T09:10:59.454 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:52.629+0000 7f35ab800700 1 -- start start 2024-04-24T09:10:59.454 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:52.629+0000 7f35ab800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35a41074e0 0x7f35a4107900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.454 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:52.629+0000 7f35ab800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f35a4107ed0 con 0x7f35a41074e0 2024-04-24T09:10:59.454 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:52.629+0000 7f35a8e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35a41074e0 0x7f35a4107900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.454 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:52.629+0000 7f35a8e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35a41074e0 0x7f35a4107900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38966/0 (socket says 172.21.15.90:38966) 2024-04-24T09:10:59.454 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:52.629+0000 7f35a8e00700 1 -- 172.21.15.90:0/3426891208 learned_addr learned my addr 172.21.15.90:0/3426891208 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:10:59.455 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:52.630+0000 7f35a8e00700 1 -- 172.21.15.90:0/3426891208 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35a41086e0 con 0x7f35a41074e0 2024-04-24T09:10:59.455 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:52.630+0000 7f35a8e00700 1 --2- 172.21.15.90:0/3426891208 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35a41074e0 0x7f35a4107900 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f35980098a0 tx=0x7f3598009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d5a3e93ca1d498c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:59.455 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:52.630+0000 7f35a3400700 1 -- 172.21.15.90:0/3426891208 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3598004160 con 0x7f35a41074e0 2024-04-24T09:10:59.455 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:52.630+0000 7f35a3400700 1 -- 172.21.15.90:0/3426891208 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3598004300 con 0x7f35a41074e0 2024-04-24T09:10:59.455 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:52.630+0000 7f35a3400700 1 -- 172.21.15.90:0/3426891208 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3598011b20 con 0x7f35a41074e0 2024-04-24T09:10:59.455 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:52.631+0000 7f35ab800700 1 -- 172.21.15.90:0/3426891208 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35a41074e0 msgr2=0x7f35a4107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:59.455 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:52.631+0000 7f35ab800700 1 --2- 172.21.15.90:0/3426891208 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35a41074e0 0x7f35a4107900 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f35980098a0 tx=0x7f3598009bb0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.455 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:52.631+0000 7f35ab800700 1 -- 172.21.15.90:0/3426891208 shutdown_connections 2024-04-24T09:10:59.456 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:52.631+0000 7f35ab800700 1 --2- 172.21.15.90:0/3426891208 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35a41074e0 0x7f35a4107900 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.456 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:52.631+0000 7f35ab800700 1 -- 172.21.15.90:0/3426891208 >> 172.21.15.90:0/3426891208 conn(0x7f35a4102ae0 msgr2=0x7f35a4104f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:59.456 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:52.631+0000 7f35ab800700 1 -- 172.21.15.90:0/3426891208 shutdown_connections 2024-04-24T09:10:59.456 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:52.631+0000 7f35ab800700 1 -- 172.21.15.90:0/3426891208 wait complete. 2024-04-24T09:10:59.456 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:52.632+0000 7f35ab800700 1 Processor -- start 2024-04-24T09:10:59.456 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:52.632+0000 7f35ab800700 1 -- start start 2024-04-24T09:10:59.456 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:52.633+0000 7f35ab800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35a41074e0 0x7f35a419c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.456 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:52.633+0000 7f35ab800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f35a410ba90 con 0x7f35a41074e0 2024-04-24T09:10:59.456 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:52.633+0000 7f35a8e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35a41074e0 0x7f35a419c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.456 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:52.633+0000 7f35a8e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35a41074e0 0x7f35a419c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38982/0 (socket says 172.21.15.90:38982) 2024-04-24T09:10:59.456 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:52.633+0000 7f35a8e00700 1 -- 172.21.15.90:0/603234095 learned_addr learned my addr 172.21.15.90:0/603234095 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:10:59.456 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:52.633+0000 7f35a8e00700 1 -- 172.21.15.90:0/603234095 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35a419ca50 con 0x7f35a41074e0 2024-04-24T09:10:59.457 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:52.634+0000 7f35a8e00700 1 --2- 172.21.15.90:0/603234095 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35a41074e0 0x7f35a419c510 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f35980117e0 tx=0x7f3598011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:59.457 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:52.634+0000 7f35a1600700 1 -- 172.21.15.90:0/603234095 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3598003bf0 con 0x7f35a41074e0 2024-04-24T09:10:59.457 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:52.634+0000 7f35a1600700 1 -- 172.21.15.90:0/603234095 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f359801a910 con 0x7f35a41074e0 2024-04-24T09:10:59.457 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:52.634+0000 7f35ab800700 1 -- 172.21.15.90:0/603234095 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35a419cce0 con 0x7f35a41074e0 2024-04-24T09:10:59.457 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:52.634+0000 7f35ab800700 1 -- 172.21.15.90:0/603234095 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35a419d140 con 0x7f35a41074e0 2024-04-24T09:10:59.457 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:52.634+0000 7f35a1600700 1 -- 172.21.15.90:0/603234095 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3598024970 con 0x7f35a41074e0 2024-04-24T09:10:59.457 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:52.636+0000 7f35a1600700 1 -- 172.21.15.90:0/603234095 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f3598022020 con 0x7f35a41074e0 2024-04-24T09:10:59.457 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:52.636+0000 7f35a1600700 1 --2- 172.21.15.90:0/603234095 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f358c039660 0x7f358c03bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.457 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:52.636+0000 7f35a1600700 1 -- 172.21.15.90:0/603234095 --> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f358c03c230 con 0x7f358c039660 2024-04-24T09:10:59.457 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:52.636+0000 7f35a1600700 1 -- 172.21.15.90:0/603234095 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f359804db60 con 0x7f35a41074e0 2024-04-24T09:10:59.458 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:52.636+0000 7f35a3e00700 1 -- 172.21.15.90:0/603234095 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f358c039660 msgr2=0x7f358c03bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/1809038230 2024-04-24T09:10:59.458 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:52.636+0000 7f35a3e00700 1 --2- 172.21.15.90:0/603234095 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f358c039660 0x7f358c03bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T09:10:59.458 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:52.837+0000 7f35a3e00700 1 -- 172.21.15.90:0/603234095 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f358c039660 msgr2=0x7f358c03bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/1809038230 2024-04-24T09:10:59.458 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:52.837+0000 7f35a3e00700 1 --2- 172.21.15.90:0/603234095 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f358c039660 0x7f358c03bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T09:10:59.458 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:53.238+0000 7f35a3e00700 1 -- 172.21.15.90:0/603234095 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f358c039660 msgr2=0x7f358c03bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/1809038230 2024-04-24T09:10:59.458 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:53.238+0000 7f35a3e00700 1 --2- 172.21.15.90:0/603234095 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f358c039660 0x7f358c03bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-24T09:10:59.458 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:54.039+0000 7f35a3e00700 1 -- 172.21.15.90:0/603234095 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f358c039660 msgr2=0x7f358c03bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/1809038230 2024-04-24T09:10:59.458 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:54.039+0000 7f35a3e00700 1 --2- 172.21.15.90:0/603234095 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f358c039660 0x7f358c03bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-24T09:10:59.458 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:55.641+0000 7f35a3e00700 1 -- 172.21.15.90:0/603234095 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f358c039660 msgr2=0x7f358c03bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/1809038230 2024-04-24T09:10:59.458 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:55.641+0000 7f35a3e00700 1 --2- 172.21.15.90:0/603234095 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f358c039660 0x7f358c03bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-24T09:10:59.458 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:57.382+0000 7f35a1600700 1 -- 172.21.15.90:0/603234095 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7f359802d5f0 con 0x7f35a41074e0 2024-04-24T09:10:59.458 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:57.382+0000 7f35a1600700 1 -- 172.21.15.90:0/603234095 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f358c039660 msgr2=0x7f358c03bb20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T09:10:59.458 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:57.382+0000 7f35a1600700 1 --2- 172.21.15.90:0/603234095 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f358c039660 0x7f358c03bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.458 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:58.383+0000 7f35a1600700 1 -- 172.21.15.90:0/603234095 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f3598018c70 con 0x7f35a41074e0 2024-04-24T09:10:59.458 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:58.383+0000 7f35a1600700 1 --2- 172.21.15.90:0/603234095 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f358c03cff0 0x7f358c03f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.459 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:58.384+0000 7f35a1600700 1 -- 172.21.15.90:0/603234095 --> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f358c03c230 con 0x7f358c03cff0 2024-04-24T09:10:59.459 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:58.385+0000 7f35a3e00700 1 --2- 172.21.15.90:0/603234095 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f358c03cff0 0x7f358c03f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.459 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:58.386+0000 7f35a3e00700 1 --2- 172.21.15.90:0/603234095 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f358c03cff0 0x7f358c03f3e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3590003a10 tx=0x7f3590009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:59.459 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:58.387+0000 7f35a1600700 1 -- 172.21.15.90:0/603234095 <== mgr.14152 v2:172.21.15.90:6800/4261662956 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f358c03c230 con 0x7f358c03cff0 2024-04-24T09:10:59.459 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:58.395+0000 7f3596a00700 1 -- 172.21.15.90:0/603234095 --> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f3580002600 con 0x7f358c03cff0 2024-04-24T09:10:59.459 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:58.396+0000 7f35a1600700 1 -- 172.21.15.90:0/603234095 <== mgr.14152 v2:172.21.15.90:6800/4261662956 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f3580002600 con 0x7f358c03cff0 2024-04-24T09:10:59.459 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:58.396+0000 7f3596a00700 1 -- 172.21.15.90:0/603234095 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f358c03cff0 msgr2=0x7f358c03f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:59.459 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:58.396+0000 7f3596a00700 1 --2- 172.21.15.90:0/603234095 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f358c03cff0 0x7f358c03f3e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3590003a10 tx=0x7f3590009270 comp rx=0 tx=0).stop 2024-04-24T09:10:59.459 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:58.396+0000 7f3596a00700 1 -- 172.21.15.90:0/603234095 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35a41074e0 msgr2=0x7f35a419c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:59.459 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:58.396+0000 7f3596a00700 1 --2- 172.21.15.90:0/603234095 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35a41074e0 0x7f35a419c510 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f35980117e0 tx=0x7f3598011ba0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.459 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:58.396+0000 7f3596a00700 1 -- 172.21.15.90:0/603234095 shutdown_connections 2024-04-24T09:10:59.459 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:58.396+0000 7f3596a00700 1 --2- 172.21.15.90:0/603234095 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f358c03cff0 0x7f358c03f3e0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.459 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:58.396+0000 7f3596a00700 1 --2- 172.21.15.90:0/603234095 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35a41074e0 0x7f35a419c510 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.459 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:58.396+0000 7f3596a00700 1 --2- 172.21.15.90:0/603234095 >> [v2:172.21.15.90:6800/1809038230,v1:172.21.15.90:6801/1809038230] conn(0x7f358c039660 0x7f358c03bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.459 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:58.396+0000 7f3596a00700 1 -- 172.21.15.90:0/603234095 >> 172.21.15.90:0/603234095 conn(0x7f35a4102ae0 msgr2=0x7f35a41038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:59.460 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:58.396+0000 7f3596a00700 1 -- 172.21.15.90:0/603234095 shutdown_connections 2024-04-24T09:10:59.460 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:10:58.396+0000 7f3596a00700 1 -- 172.21.15.90:0/603234095 wait complete. 2024-04-24T09:10:59.460 INFO:teuthology.orchestra.run.smithi090.stdout:mgr epoch 9 is available 2024-04-24T09:10:59.460 INFO:teuthology.orchestra.run.smithi090.stdout:Generating a dashboard self-signed certificate... 2024-04-24T09:10:59.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:59 smithi090 ceph-mon[27825]: [24/Apr/2024:09:10:57] ENGINE Bus STARTING 2024-04-24T09:10:59.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:59 smithi090 ceph-mon[27825]: [24/Apr/2024:09:10:57] ENGINE Serving on https://172.21.15.90:7150 2024-04-24T09:10:59.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:59 smithi090 ceph-mon[27825]: [24/Apr/2024:09:10:57] ENGINE Serving on http://172.21.15.90:8765 2024-04-24T09:10:59.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:59 smithi090 ceph-mon[27825]: [24/Apr/2024:09:10:57] ENGINE Bus STARTED 2024-04-24T09:10:59.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:59 smithi090 ceph-mon[27825]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-24T09:10:59.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:10:59 smithi090 ceph-mon[27825]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-24T09:11:00.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:00 smithi090 ceph-mon[27825]: mgrmap e12: x(active, since 2s) 2024-04-24T09:11:00.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:00 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:11:01.604 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-24T09:11:01.610 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.420+0000 7f0cb3000700 1 Processor -- start 2024-04-24T09:11:01.610 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.420+0000 7f0cb3000700 1 -- start start 2024-04-24T09:11:01.610 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.420+0000 7f0cb3000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cac108f00 0x7f0cac10b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:01.610 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.420+0000 7f0cb3000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0cac07be20 con 0x7f0cac108f00 2024-04-24T09:11:01.610 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.421+0000 7f0cb1c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cac108f00 0x7f0cac10b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:01.610 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.421+0000 7f0cb1c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cac108f00 0x7f0cac10b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39686/0 (socket says 172.21.15.90:39686) 2024-04-24T09:11:01.611 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.421+0000 7f0cb1c00700 1 -- 172.21.15.90:0/282989017 learned_addr learned my addr 172.21.15.90:0/282989017 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:11:01.611 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.421+0000 7f0cb1c00700 1 -- 172.21.15.90:0/282989017 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cac10b830 con 0x7f0cac108f00 2024-04-24T09:11:01.611 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.422+0000 7f0cb1c00700 1 --2- 172.21.15.90:0/282989017 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cac108f00 0x7f0cac10b2f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0ca00098a0 tx=0x7f0ca0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64f64eb1397901f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:01.611 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.422+0000 7f0cabe00700 1 -- 172.21.15.90:0/282989017 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ca0004160 con 0x7f0cac108f00 2024-04-24T09:11:01.611 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.422+0000 7f0cabe00700 1 -- 172.21.15.90:0/282989017 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ca0004300 con 0x7f0cac108f00 2024-04-24T09:11:01.611 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.422+0000 7f0cabe00700 1 -- 172.21.15.90:0/282989017 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ca0011b20 con 0x7f0cac108f00 2024-04-24T09:11:01.611 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.423+0000 7f0cb3000700 1 -- 172.21.15.90:0/282989017 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cac108f00 msgr2=0x7f0cac10b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:01.611 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.423+0000 7f0cb3000700 1 --2- 172.21.15.90:0/282989017 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cac108f00 0x7f0cac10b2f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0ca00098a0 tx=0x7f0ca0009bb0 comp rx=0 tx=0).stop 2024-04-24T09:11:01.611 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.423+0000 7f0cb3000700 1 -- 172.21.15.90:0/282989017 shutdown_connections 2024-04-24T09:11:01.611 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.423+0000 7f0cb3000700 1 --2- 172.21.15.90:0/282989017 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cac108f00 0x7f0cac10b2f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:01.611 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.423+0000 7f0cb3000700 1 -- 172.21.15.90:0/282989017 >> 172.21.15.90:0/282989017 conn(0x7f0cac1028d0 msgr2=0x7f0cac104cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:01.611 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.423+0000 7f0cb3000700 1 -- 172.21.15.90:0/282989017 shutdown_connections 2024-04-24T09:11:01.611 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.423+0000 7f0cb3000700 1 -- 172.21.15.90:0/282989017 wait complete. 2024-04-24T09:11:01.612 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.424+0000 7f0cb3000700 1 Processor -- start 2024-04-24T09:11:01.612 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.424+0000 7f0cb3000700 1 -- start start 2024-04-24T09:11:01.612 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.425+0000 7f0cb3000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cac108f00 0x7f0cac19c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:01.612 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.425+0000 7f0cb3000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0cac10c830 con 0x7f0cac108f00 2024-04-24T09:11:01.612 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.425+0000 7f0cb1c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cac108f00 0x7f0cac19c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:01.612 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.425+0000 7f0cb1c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cac108f00 0x7f0cac19c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39690/0 (socket says 172.21.15.90:39690) 2024-04-24T09:11:01.612 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.425+0000 7f0cb1c00700 1 -- 172.21.15.90:0/3318015886 learned_addr learned my addr 172.21.15.90:0/3318015886 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:11:01.612 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.425+0000 7f0cb1c00700 1 -- 172.21.15.90:0/3318015886 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cac19c8d0 con 0x7f0cac108f00 2024-04-24T09:11:01.612 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.426+0000 7f0cb1c00700 1 --2- 172.21.15.90:0/3318015886 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cac108f00 0x7f0cac19c390 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0011b60 tx=0x7f0ca0011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:01.612 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.426+0000 7f0caa000700 1 -- 172.21.15.90:0/3318015886 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ca0003b60 con 0x7f0cac108f00 2024-04-24T09:11:01.612 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.426+0000 7f0caa000700 1 -- 172.21.15.90:0/3318015886 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ca001a910 con 0x7f0cac108f00 2024-04-24T09:11:01.612 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.426+0000 7f0cb3000700 1 -- 172.21.15.90:0/3318015886 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cac19cb60 con 0x7f0cac108f00 2024-04-24T09:11:01.612 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.426+0000 7f0caa000700 1 -- 172.21.15.90:0/3318015886 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ca0024930 con 0x7f0cac108f00 2024-04-24T09:11:01.612 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.426+0000 7f0cb3000700 1 -- 172.21.15.90:0/3318015886 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cac19cfc0 con 0x7f0cac108f00 2024-04-24T09:11:01.612 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.427+0000 7f0cb3000700 1 -- 172.21.15.90:0/3318015886 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cac064150 con 0x7f0cac108f00 2024-04-24T09:11:01.613 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.428+0000 7f0caa000700 1 -- 172.21.15.90:0/3318015886 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f0ca0022020 con 0x7f0cac108f00 2024-04-24T09:11:01.613 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.428+0000 7f0caa000700 1 --2- 172.21.15.90:0/3318015886 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f0c94039600 0x7f0c9403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:01.613 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.428+0000 7f0caa000700 1 -- 172.21.15.90:0/3318015886 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0ca004da30 con 0x7f0cac108f00 2024-04-24T09:11:01.613 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.428+0000 7f0cb1200700 1 --2- 172.21.15.90:0/3318015886 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f0c94039600 0x7f0c9403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:01.613 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.429+0000 7f0cb1200700 1 --2- 172.21.15.90:0/3318015886 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f0c94039600 0x7f0c9403bac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0c98006fd0 tx=0x7f0c98006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:01.613 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.434+0000 7f0caa000700 1 -- 172.21.15.90:0/3318015886 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0ca000b3e0 con 0x7f0cac108f00 2024-04-24T09:11:01.613 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.667+0000 7f0cb3000700 1 -- 172.21.15.90:0/3318015886 --> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f0cac103aa0 con 0x7f0c94039600 2024-04-24T09:11:01.613 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.830+0000 7f0caa000700 1 -- 172.21.15.90:0/3318015886 <== mgr.14152 v2:172.21.15.90:6800/4261662956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f0cac103aa0 con 0x7f0c94039600 2024-04-24T09:11:01.613 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.833+0000 7f0cb3000700 1 -- 172.21.15.90:0/3318015886 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f0c94039600 msgr2=0x7f0c9403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:01.613 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.833+0000 7f0cb3000700 1 --2- 172.21.15.90:0/3318015886 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f0c94039600 0x7f0c9403bac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0c98006fd0 tx=0x7f0c98006e40 comp rx=0 tx=0).stop 2024-04-24T09:11:01.613 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.833+0000 7f0cb3000700 1 -- 172.21.15.90:0/3318015886 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cac108f00 msgr2=0x7f0cac19c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:01.613 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.833+0000 7f0cb3000700 1 --2- 172.21.15.90:0/3318015886 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cac108f00 0x7f0cac19c390 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0011b60 tx=0x7f0ca0011c40 comp rx=0 tx=0).stop 2024-04-24T09:11:01.613 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.833+0000 7f0cb3000700 1 -- 172.21.15.90:0/3318015886 shutdown_connections 2024-04-24T09:11:01.613 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.833+0000 7f0cb3000700 1 --2- 172.21.15.90:0/3318015886 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f0c94039600 0x7f0c9403bac0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:01.613 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.833+0000 7f0cb3000700 1 --2- 172.21.15.90:0/3318015886 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cac108f00 0x7f0cac19c390 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:01.614 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.833+0000 7f0cb3000700 1 -- 172.21.15.90:0/3318015886 >> 172.21.15.90:0/3318015886 conn(0x7f0cac1028d0 msgr2=0x7f0cac103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:01.614 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.834+0000 7f0cb3000700 1 -- 172.21.15.90:0/3318015886 shutdown_connections 2024-04-24T09:11:01.614 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:00.834+0000 7f0cb3000700 1 -- 172.21.15.90:0/3318015886 wait complete. 2024-04-24T09:11:01.614 INFO:teuthology.orchestra.run.smithi090.stdout:Creating initial admin user... 2024-04-24T09:11:01.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:01 smithi090 ceph-mon[27825]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:11:01.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:01 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:11:01.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:01 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:11:03.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:03 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:11:03.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:03 smithi090 ceph-mon[27825]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:11:03.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:03 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:11:03.678 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$WEMSK7PE1pYf86u6vqGDWujnHuEOe9AGwUTqoX2NsB8hDHnwbhQDK", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1713949862, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-24T09:11:03.678 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.245+0000 7fd340000700 1 Processor -- start 2024-04-24T09:11:03.678 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.245+0000 7fd340000700 1 -- start start 2024-04-24T09:11:03.678 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.246+0000 7fd340000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd338108610 0x7fd338108a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:03.678 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.246+0000 7fd340000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd338109000 con 0x7fd338108610 2024-04-24T09:11:03.678 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.246+0000 7fd33d600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd338108610 0x7fd338108a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:03.679 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.246+0000 7fd33d600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd338108610 0x7fd338108a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39694/0 (socket says 172.21.15.90:39694) 2024-04-24T09:11:03.679 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.246+0000 7fd33d600700 1 -- 172.21.15.90:0/3632024757 learned_addr learned my addr 172.21.15.90:0/3632024757 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:11:03.679 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.247+0000 7fd33d600700 1 -- 172.21.15.90:0/3632024757 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3381097c0 con 0x7fd338108610 2024-04-24T09:11:03.679 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.247+0000 7fd33d600700 1 --2- 172.21.15.90:0/3632024757 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd338108610 0x7fd338108a30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd32c0098a0 tx=0x7fd32c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=438e8f15fa6eea41 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:03.679 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.247+0000 7fd337e00700 1 -- 172.21.15.90:0/3632024757 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd32c004160 con 0x7fd338108610 2024-04-24T09:11:03.679 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.247+0000 7fd337e00700 1 -- 172.21.15.90:0/3632024757 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd32c004300 con 0x7fd338108610 2024-04-24T09:11:03.679 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.247+0000 7fd337e00700 1 -- 172.21.15.90:0/3632024757 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd32c011b20 con 0x7fd338108610 2024-04-24T09:11:03.679 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.248+0000 7fd340000700 1 -- 172.21.15.90:0/3632024757 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd338108610 msgr2=0x7fd338108a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:03.679 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.248+0000 7fd340000700 1 --2- 172.21.15.90:0/3632024757 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd338108610 0x7fd338108a30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd32c0098a0 tx=0x7fd32c009bb0 comp rx=0 tx=0).stop 2024-04-24T09:11:03.679 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.248+0000 7fd340000700 1 -- 172.21.15.90:0/3632024757 shutdown_connections 2024-04-24T09:11:03.679 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.248+0000 7fd340000700 1 --2- 172.21.15.90:0/3632024757 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd338108610 0x7fd338108a30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:03.679 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.248+0000 7fd340000700 1 -- 172.21.15.90:0/3632024757 >> 172.21.15.90:0/3632024757 conn(0x7fd338103c10 msgr2=0x7fd338106030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:03.679 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.248+0000 7fd340000700 1 -- 172.21.15.90:0/3632024757 shutdown_connections 2024-04-24T09:11:03.679 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.248+0000 7fd340000700 1 -- 172.21.15.90:0/3632024757 wait complete. 2024-04-24T09:11:03.680 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.249+0000 7fd340000700 1 Processor -- start 2024-04-24T09:11:03.680 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.250+0000 7fd340000700 1 -- start start 2024-04-24T09:11:03.680 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.250+0000 7fd340000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd338108610 0x7fd33807db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:03.680 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.250+0000 7fd340000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd33810cb70 con 0x7fd338108610 2024-04-24T09:11:03.680 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.250+0000 7fd33d600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd338108610 0x7fd33807db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:03.680 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.250+0000 7fd33d600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd338108610 0x7fd33807db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39708/0 (socket says 172.21.15.90:39708) 2024-04-24T09:11:03.680 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.250+0000 7fd33d600700 1 -- 172.21.15.90:0/905515945 learned_addr learned my addr 172.21.15.90:0/905515945 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:11:03.680 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.251+0000 7fd33d600700 1 -- 172.21.15.90:0/905515945 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd33807e0a0 con 0x7fd338108610 2024-04-24T09:11:03.680 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.251+0000 7fd33d600700 1 --2- 172.21.15.90:0/905515945 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd338108610 0x7fd33807db60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd32c000c00 tx=0x7fd32c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:03.680 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.251+0000 7fd336000700 1 -- 172.21.15.90:0/905515945 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd32c003b60 con 0x7fd338108610 2024-04-24T09:11:03.680 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.251+0000 7fd336000700 1 -- 172.21.15.90:0/905515945 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd32c01a910 con 0x7fd338108610 2024-04-24T09:11:03.680 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.251+0000 7fd336000700 1 -- 172.21.15.90:0/905515945 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd32c024930 con 0x7fd338108610 2024-04-24T09:11:03.680 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.251+0000 7fd340000700 1 -- 172.21.15.90:0/905515945 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd33807c290 con 0x7fd338108610 2024-04-24T09:11:03.680 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.252+0000 7fd340000700 1 -- 172.21.15.90:0/905515945 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd33807c750 con 0x7fd338108610 2024-04-24T09:11:03.680 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.253+0000 7fd340000700 1 -- 172.21.15.90:0/905515945 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd318005360 con 0x7fd338108610 2024-04-24T09:11:03.681 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.253+0000 7fd336000700 1 -- 172.21.15.90:0/905515945 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fd32c022020 con 0x7fd338108610 2024-04-24T09:11:03.681 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.253+0000 7fd336000700 1 --2- 172.21.15.90:0/905515945 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fd320039660 0x7fd32003bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:03.681 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.253+0000 7fd336000700 1 -- 172.21.15.90:0/905515945 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd32c04da00 con 0x7fd338108610 2024-04-24T09:11:03.681 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.254+0000 7fd33cc00700 1 --2- 172.21.15.90:0/905515945 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fd320039660 0x7fd32003bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:03.681 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.255+0000 7fd33cc00700 1 --2- 172.21.15.90:0/905515945 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fd320039660 0x7fd32003bb20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd324006fd0 tx=0x7fd324006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:03.681 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.259+0000 7fd336000700 1 -- 172.21.15.90:0/905515945 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd32c00b3e0 con 0x7fd338108610 2024-04-24T09:11:03.681 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.508+0000 7fd340000700 1 -- 172.21.15.90:0/905515945 --> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fd318002430 con 0x7fd320039660 2024-04-24T09:11:03.681 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.776+0000 7fd336000700 1 -- 172.21.15.90:0/905515945 <== mgr.14152 v2:172.21.15.90:6800/4261662956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fd318002430 con 0x7fd320039660 2024-04-24T09:11:03.681 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.780+0000 7fd340000700 1 -- 172.21.15.90:0/905515945 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fd320039660 msgr2=0x7fd32003bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:03.681 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.780+0000 7fd340000700 1 --2- 172.21.15.90:0/905515945 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fd320039660 0x7fd32003bb20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd324006fd0 tx=0x7fd324006e40 comp rx=0 tx=0).stop 2024-04-24T09:11:03.681 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.780+0000 7fd340000700 1 -- 172.21.15.90:0/905515945 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd338108610 msgr2=0x7fd33807db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:03.681 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.780+0000 7fd340000700 1 --2- 172.21.15.90:0/905515945 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd338108610 0x7fd33807db60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd32c000c00 tx=0x7fd32c011c10 comp rx=0 tx=0).stop 2024-04-24T09:11:03.681 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.781+0000 7fd340000700 1 -- 172.21.15.90:0/905515945 shutdown_connections 2024-04-24T09:11:03.681 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.781+0000 7fd340000700 1 --2- 172.21.15.90:0/905515945 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fd320039660 0x7fd32003bb20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:03.682 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.781+0000 7fd340000700 1 --2- 172.21.15.90:0/905515945 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd338108610 0x7fd33807db60 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:03.682 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.781+0000 7fd340000700 1 -- 172.21.15.90:0/905515945 >> 172.21.15.90:0/905515945 conn(0x7fd338103c10 msgr2=0x7fd338104770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:03.682 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.781+0000 7fd340000700 1 -- 172.21.15.90:0/905515945 shutdown_connections 2024-04-24T09:11:03.682 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:02.781+0000 7fd340000700 1 -- 172.21.15.90:0/905515945 wait complete. 2024-04-24T09:11:03.682 INFO:teuthology.orchestra.run.smithi090.stdout:Fetching dashboard port number... 2024-04-24T09:11:05.228 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:05 smithi090 ceph-mon[27825]: mgrmap e13: x(active, since 6s) 2024-04-24T09:11:05.228 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:05 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/3257701270' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-24T09:11:05.575 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 8443 2024-04-24T09:11:05.576 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.368+0000 7f8803a00700 1 Processor -- start 2024-04-24T09:11:05.576 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.369+0000 7f8803a00700 1 -- start start 2024-04-24T09:11:05.576 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.369+0000 7f8803a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87fc108600 0x7f87fc108a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:05.577 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.369+0000 7f8803a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f87fc108ff0 con 0x7f87fc108600 2024-04-24T09:11:05.577 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.369+0000 7f8801000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87fc108600 0x7f87fc108a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:05.577 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.369+0000 7f8801000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87fc108600 0x7f87fc108a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39718/0 (socket says 172.21.15.90:39718) 2024-04-24T09:11:05.577 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.369+0000 7f8801000700 1 -- 172.21.15.90:0/1868423110 learned_addr learned my addr 172.21.15.90:0/1868423110 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:11:05.578 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.370+0000 7f8801000700 1 -- 172.21.15.90:0/1868423110 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87fc1097b0 con 0x7f87fc108600 2024-04-24T09:11:05.578 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.370+0000 7f8801000700 1 --2- 172.21.15.90:0/1868423110 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87fc108600 0x7f87fc108a20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f87f0009e80 tx=0x7f87f000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22d0bcabe0e2f315 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:05.578 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.370+0000 7f87fb400700 1 -- 172.21.15.90:0/1868423110 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87f0017070 con 0x7f87fc108600 2024-04-24T09:11:05.578 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.370+0000 7f87fb400700 1 -- 172.21.15.90:0/1868423110 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87f000b8c0 con 0x7f87fc108600 2024-04-24T09:11:05.579 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.370+0000 7f87fb400700 1 -- 172.21.15.90:0/1868423110 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87f00045c0 con 0x7f87fc108600 2024-04-24T09:11:05.579 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.371+0000 7f8803a00700 1 -- 172.21.15.90:0/1868423110 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87fc108600 msgr2=0x7f87fc108a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:05.579 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.371+0000 7f8803a00700 1 --2- 172.21.15.90:0/1868423110 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87fc108600 0x7f87fc108a20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f87f0009e80 tx=0x7f87f000b330 comp rx=0 tx=0).stop 2024-04-24T09:11:05.579 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.371+0000 7f8803a00700 1 -- 172.21.15.90:0/1868423110 shutdown_connections 2024-04-24T09:11:05.579 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.371+0000 7f8803a00700 1 --2- 172.21.15.90:0/1868423110 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87fc108600 0x7f87fc108a20 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:05.580 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.371+0000 7f8803a00700 1 -- 172.21.15.90:0/1868423110 >> 172.21.15.90:0/1868423110 conn(0x7f87fc103c40 msgr2=0x7f87fc106060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:05.580 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.371+0000 7f8803a00700 1 -- 172.21.15.90:0/1868423110 shutdown_connections 2024-04-24T09:11:05.581 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.371+0000 7f8803a00700 1 -- 172.21.15.90:0/1868423110 wait complete. 2024-04-24T09:11:05.581 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.373+0000 7f8803a00700 1 Processor -- start 2024-04-24T09:11:05.581 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.373+0000 7f8803a00700 1 -- start start 2024-04-24T09:11:05.581 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.373+0000 7f8803a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87fc108600 0x7f87fc19c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:05.581 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.373+0000 7f8803a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f87fc10cb60 con 0x7f87fc108600 2024-04-24T09:11:05.582 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.373+0000 7f8801000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87fc108600 0x7f87fc19c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:05.582 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.373+0000 7f8801000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87fc108600 0x7f87fc19c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39726/0 (socket says 172.21.15.90:39726) 2024-04-24T09:11:05.582 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.373+0000 7f8801000700 1 -- 172.21.15.90:0/3257701270 learned_addr learned my addr 172.21.15.90:0/3257701270 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:11:05.582 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.374+0000 7f8801000700 1 -- 172.21.15.90:0/3257701270 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87fc19ca50 con 0x7f87fc108600 2024-04-24T09:11:05.582 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.374+0000 7f8801000700 1 --2- 172.21.15.90:0/3257701270 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87fc108600 0x7f87fc19c510 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f87f0009fd0 tx=0x7f87f0003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:05.582 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.374+0000 7f87f9600700 1 -- 172.21.15.90:0/3257701270 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87f0020020 con 0x7f87fc108600 2024-04-24T09:11:05.582 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.374+0000 7f87f9600700 1 -- 172.21.15.90:0/3257701270 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87f000bba0 con 0x7f87fc108600 2024-04-24T09:11:05.584 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.374+0000 7f87f9600700 1 -- 172.21.15.90:0/3257701270 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87f0017040 con 0x7f87fc108600 2024-04-24T09:11:05.584 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.374+0000 7f8803a00700 1 -- 172.21.15.90:0/3257701270 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87fc19cce0 con 0x7f87fc108600 2024-04-24T09:11:05.584 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.375+0000 7f8803a00700 1 -- 172.21.15.90:0/3257701270 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87fc19f9d0 con 0x7f87fc108600 2024-04-24T09:11:05.584 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.376+0000 7f87f9600700 1 -- 172.21.15.90:0/3257701270 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f87f0023020 con 0x7f87fc108600 2024-04-24T09:11:05.584 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.376+0000 7f87eea00700 1 -- 172.21.15.90:0/3257701270 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87fc064150 con 0x7f87fc108600 2024-04-24T09:11:05.584 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.376+0000 7f87f9600700 1 --2- 172.21.15.90:0/3257701270 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f87e4039700 0x7f87e403bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:05.584 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.376+0000 7f87f9600700 1 -- 172.21.15.90:0/3257701270 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f87f000f050 con 0x7f87fc108600 2024-04-24T09:11:05.584 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.377+0000 7f87fbe00700 1 --2- 172.21.15.90:0/3257701270 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f87e4039700 0x7f87e403bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:05.585 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.377+0000 7f87fbe00700 1 --2- 172.21.15.90:0/3257701270 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f87e4039700 0x7f87e403bbc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f87e8006fd0 tx=0x7f87e8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:05.585 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.382+0000 7f87f9600700 1 -- 172.21.15.90:0/3257701270 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f87f002a020 con 0x7f87fc108600 2024-04-24T09:11:05.585 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.613+0000 7f87eea00700 1 -- 172.21.15.90:0/3257701270 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f87fc063700 con 0x7f87fc108600 2024-04-24T09:11:05.586 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.613+0000 7f87f9600700 1 -- 172.21.15.90:0/3257701270 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f87f0019260 con 0x7f87fc108600 2024-04-24T09:11:05.586 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.616+0000 7f87eea00700 1 -- 172.21.15.90:0/3257701270 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f87e4039700 msgr2=0x7f87e403bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:05.586 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.616+0000 7f87eea00700 1 --2- 172.21.15.90:0/3257701270 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f87e4039700 0x7f87e403bbc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f87e8006fd0 tx=0x7f87e8006e40 comp rx=0 tx=0).stop 2024-04-24T09:11:05.586 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.616+0000 7f87eea00700 1 -- 172.21.15.90:0/3257701270 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87fc108600 msgr2=0x7f87fc19c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:05.586 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.616+0000 7f87eea00700 1 --2- 172.21.15.90:0/3257701270 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87fc108600 0x7f87fc19c510 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f87f0009fd0 tx=0x7f87f0003d00 comp rx=0 tx=0).stop 2024-04-24T09:11:05.587 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.616+0000 7f87eea00700 1 -- 172.21.15.90:0/3257701270 shutdown_connections 2024-04-24T09:11:05.587 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.616+0000 7f87eea00700 1 --2- 172.21.15.90:0/3257701270 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f87e4039700 0x7f87e403bbc0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:05.587 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.616+0000 7f87eea00700 1 --2- 172.21.15.90:0/3257701270 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87fc108600 0x7f87fc19c510 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:05.587 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.616+0000 7f87eea00700 1 -- 172.21.15.90:0/3257701270 >> 172.21.15.90:0/3257701270 conn(0x7f87fc103c40 msgr2=0x7f87fc104730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:05.587 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.617+0000 7f87eea00700 1 -- 172.21.15.90:0/3257701270 shutdown_connections 2024-04-24T09:11:05.587 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:04.617+0000 7f87eea00700 1 -- 172.21.15.90:0/3257701270 wait complete. 2024-04-24T09:11:05.631 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T09:11:05.631 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout disabled 2024-04-24T09:11:05.656 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T09:11:05.656 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout inactive 2024-04-24T09:11:05.657 INFO:teuthology.orchestra.run.smithi090.stdout:firewalld.service is not enabled 2024-04-24T09:11:05.657 INFO:teuthology.orchestra.run.smithi090.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-24T09:11:05.660 INFO:teuthology.orchestra.run.smithi090.stdout:Ceph Dashboard is now available at: 2024-04-24T09:11:05.660 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:11:05.660 INFO:teuthology.orchestra.run.smithi090.stdout: URL: https://smithi090.front.sepia.ceph.com:8443/ 2024-04-24T09:11:05.661 INFO:teuthology.orchestra.run.smithi090.stdout: User: admin 2024-04-24T09:11:05.661 INFO:teuthology.orchestra.run.smithi090.stdout: Password: tag1wvysmx 2024-04-24T09:11:05.661 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:11:05.661 INFO:teuthology.orchestra.run.smithi090.stdout:Saving cluster configuration to /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/config directory 2024-04-24T09:11:05.662 INFO:teuthology.orchestra.run.smithi090.stdout:Enabling autotune for osd_memory_target 2024-04-24T09:11:07.702 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.641+0000 7f46d9800700 1 Processor -- start 2024-04-24T09:11:07.703 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.641+0000 7f46d9800700 1 -- start start 2024-04-24T09:11:07.703 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.642+0000 7f46d9800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46d410a7d0 0x7f46d410abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:07.703 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.642+0000 7f46d9800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f46d410b1c0 con 0x7f46d410a7d0 2024-04-24T09:11:07.703 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.642+0000 7f46d2a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46d410a7d0 0x7f46d410abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:07.703 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.642+0000 7f46d2a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46d410a7d0 0x7f46d410abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40120/0 (socket says 172.21.15.90:40120) 2024-04-24T09:11:07.703 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.642+0000 7f46d2a00700 1 -- 172.21.15.90:0/1283255905 learned_addr learned my addr 172.21.15.90:0/1283255905 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:11:07.703 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.643+0000 7f46d2a00700 1 -- 172.21.15.90:0/1283255905 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d410b980 con 0x7f46d410a7d0 2024-04-24T09:11:07.703 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.643+0000 7f46d2a00700 1 --2- 172.21.15.90:0/1283255905 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46d410a7d0 0x7f46d410abf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f46c40098a0 tx=0x7f46c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46875200a2154496 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:07.703 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.644+0000 7f46d2000700 1 -- 172.21.15.90:0/1283255905 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46c4004160 con 0x7f46d410a7d0 2024-04-24T09:11:07.703 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.644+0000 7f46d2000700 1 -- 172.21.15.90:0/1283255905 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46c4004300 con 0x7f46d410a7d0 2024-04-24T09:11:07.703 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.644+0000 7f46d2000700 1 -- 172.21.15.90:0/1283255905 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46c4011b20 con 0x7f46d410a7d0 2024-04-24T09:11:07.703 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.644+0000 7f46d9800700 1 -- 172.21.15.90:0/1283255905 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46d410a7d0 msgr2=0x7f46d410abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:07.703 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.644+0000 7f46d9800700 1 --2- 172.21.15.90:0/1283255905 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46d410a7d0 0x7f46d410abf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f46c40098a0 tx=0x7f46c4009bb0 comp rx=0 tx=0).stop 2024-04-24T09:11:07.703 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.644+0000 7f46d9800700 1 -- 172.21.15.90:0/1283255905 shutdown_connections 2024-04-24T09:11:07.704 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.644+0000 7f46d9800700 1 --2- 172.21.15.90:0/1283255905 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46d410a7d0 0x7f46d410abf0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:07.704 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.644+0000 7f46d9800700 1 -- 172.21.15.90:0/1283255905 >> 172.21.15.90:0/1283255905 conn(0x7f46d407ddc0 msgr2=0x7f46d41084c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:07.704 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.645+0000 7f46d9800700 1 -- 172.21.15.90:0/1283255905 shutdown_connections 2024-04-24T09:11:07.704 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.645+0000 7f46d9800700 1 -- 172.21.15.90:0/1283255905 wait complete. 2024-04-24T09:11:07.704 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.646+0000 7f46d9800700 1 Processor -- start 2024-04-24T09:11:07.704 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.646+0000 7f46d9800700 1 -- start start 2024-04-24T09:11:07.704 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.646+0000 7f46d9800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46d410a7d0 0x7f46d419e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:07.704 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.646+0000 7f46d9800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f46d410ed30 con 0x7f46d410a7d0 2024-04-24T09:11:07.705 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.646+0000 7f46d2a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46d410a7d0 0x7f46d419e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:07.705 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.646+0000 7f46d2a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46d410a7d0 0x7f46d419e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40136/0 (socket says 172.21.15.90:40136) 2024-04-24T09:11:07.705 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.646+0000 7f46d2a00700 1 -- 172.21.15.90:0/3818606762 learned_addr learned my addr 172.21.15.90:0/3818606762 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:11:07.705 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.647+0000 7f46d2a00700 1 -- 172.21.15.90:0/3818606762 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d419ebf0 con 0x7f46d410a7d0 2024-04-24T09:11:07.705 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.647+0000 7f46d2a00700 1 --2- 172.21.15.90:0/3818606762 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46d410a7d0 0x7f46d419e6b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f46c40117e0 tx=0x7f46c4011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:07.705 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.647+0000 7f46cb400700 1 -- 172.21.15.90:0/3818606762 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46c4003bf0 con 0x7f46d410a7d0 2024-04-24T09:11:07.705 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.647+0000 7f46cb400700 1 -- 172.21.15.90:0/3818606762 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46c401a910 con 0x7f46d410a7d0 2024-04-24T09:11:07.705 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.647+0000 7f46d9800700 1 -- 172.21.15.90:0/3818606762 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46d419ee80 con 0x7f46d410a7d0 2024-04-24T09:11:07.705 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.647+0000 7f46cb400700 1 -- 172.21.15.90:0/3818606762 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46c4024970 con 0x7f46d410a7d0 2024-04-24T09:11:07.705 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.648+0000 7f46d9800700 1 -- 172.21.15.90:0/3818606762 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46d419f2e0 con 0x7f46d410a7d0 2024-04-24T09:11:07.705 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.649+0000 7f46cb400700 1 -- 172.21.15.90:0/3818606762 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f46c4022020 con 0x7f46d410a7d0 2024-04-24T09:11:07.705 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.649+0000 7f46d9800700 1 -- 172.21.15.90:0/3818606762 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46d4064150 con 0x7f46d410a7d0 2024-04-24T09:11:07.705 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.649+0000 7f46cb400700 1 --2- 172.21.15.90:0/3818606762 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f46b8039700 0x7f46b803bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:07.706 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.649+0000 7f46cb400700 1 -- 172.21.15.90:0/3818606762 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f46c404dc40 con 0x7f46d410a7d0 2024-04-24T09:11:07.706 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.649+0000 7f46cbe00700 1 --2- 172.21.15.90:0/3818606762 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f46b8039700 0x7f46b803bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:07.706 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.650+0000 7f46cbe00700 1 --2- 172.21.15.90:0/3818606762 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f46b8039700 0x7f46b803bbc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f46bc006fd0 tx=0x7f46bc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:07.706 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.655+0000 7f46cb400700 1 -- 172.21.15.90:0/3818606762 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f46c4010380 con 0x7f46d410a7d0 2024-04-24T09:11:07.706 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.878+0000 7f46d9800700 1 -- 172.21.15.90:0/3818606762 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f46d41a1cb0 con 0x7f46d410a7d0 2024-04-24T09:11:07.706 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.879+0000 7f46cb400700 1 -- 172.21.15.90:0/3818606762 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f46c40212c0 con 0x7f46d410a7d0 2024-04-24T09:11:07.706 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.882+0000 7f46d9800700 1 -- 172.21.15.90:0/3818606762 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f46b8039700 msgr2=0x7f46b803bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:07.706 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.882+0000 7f46d9800700 1 --2- 172.21.15.90:0/3818606762 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f46b8039700 0x7f46b803bbc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f46bc006fd0 tx=0x7f46bc006e40 comp rx=0 tx=0).stop 2024-04-24T09:11:07.706 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.882+0000 7f46d9800700 1 -- 172.21.15.90:0/3818606762 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46d410a7d0 msgr2=0x7f46d419e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:07.706 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.882+0000 7f46d9800700 1 --2- 172.21.15.90:0/3818606762 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46d410a7d0 0x7f46d419e6b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f46c40117e0 tx=0x7f46c4011ba0 comp rx=0 tx=0).stop 2024-04-24T09:11:07.706 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.882+0000 7f46d9800700 1 -- 172.21.15.90:0/3818606762 shutdown_connections 2024-04-24T09:11:07.706 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.882+0000 7f46d9800700 1 --2- 172.21.15.90:0/3818606762 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f46b8039700 0x7f46b803bbc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:07.706 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.882+0000 7f46d9800700 1 --2- 172.21.15.90:0/3818606762 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46d410a7d0 0x7f46d419e6b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:07.707 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.882+0000 7f46d9800700 1 -- 172.21.15.90:0/3818606762 >> 172.21.15.90:0/3818606762 conn(0x7f46d407ddc0 msgr2=0x7f46d4107ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:07.707 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.883+0000 7f46d9800700 1 -- 172.21.15.90:0/3818606762 shutdown_connections 2024-04-24T09:11:07.707 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:06.883+0000 7f46d9800700 1 -- 172.21.15.90:0/3818606762 wait complete. 2024-04-24T09:11:09.408 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.303+0000 7fafdde00700 1 Processor -- start 2024-04-24T09:11:09.408 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.303+0000 7fafdde00700 1 -- start start 2024-04-24T09:11:09.409 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.304+0000 7fafdde00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fafd8108410 0x7fafd8108830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:09.409 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.304+0000 7fafdde00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fafd8108e00 con 0x7fafd8108410 2024-04-24T09:11:09.409 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.305+0000 7fafdca00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fafd8108410 0x7fafd8108830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:09.409 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.305+0000 7fafdca00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fafd8108410 0x7fafd8108830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40138/0 (socket says 172.21.15.90:40138) 2024-04-24T09:11:09.409 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.305+0000 7fafdca00700 1 -- 172.21.15.90:0/1140348402 learned_addr learned my addr 172.21.15.90:0/1140348402 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:11:09.409 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.305+0000 7fafdca00700 1 -- 172.21.15.90:0/1140348402 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd81095c0 con 0x7fafd8108410 2024-04-24T09:11:09.414 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.306+0000 7fafdca00700 1 --2- 172.21.15.90:0/1140348402 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fafd8108410 0x7fafd8108830 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fafcc009e80 tx=0x7fafcc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9c8f7920acb5a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:09.414 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.306+0000 7fafd7400700 1 -- 172.21.15.90:0/1140348402 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafcc017070 con 0x7fafd8108410 2024-04-24T09:11:09.414 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.306+0000 7fafd7400700 1 -- 172.21.15.90:0/1140348402 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafcc00b8c0 con 0x7fafd8108410 2024-04-24T09:11:09.414 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.306+0000 7fafd7400700 1 -- 172.21.15.90:0/1140348402 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafcc0045c0 con 0x7fafd8108410 2024-04-24T09:11:09.414 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.307+0000 7fafdde00700 1 -- 172.21.15.90:0/1140348402 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fafd8108410 msgr2=0x7fafd8108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:09.415 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.307+0000 7fafdde00700 1 --2- 172.21.15.90:0/1140348402 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fafd8108410 0x7fafd8108830 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fafcc009e80 tx=0x7fafcc00b330 comp rx=0 tx=0).stop 2024-04-24T09:11:09.415 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.307+0000 7fafdde00700 1 -- 172.21.15.90:0/1140348402 shutdown_connections 2024-04-24T09:11:09.415 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.307+0000 7fafdde00700 1 --2- 172.21.15.90:0/1140348402 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fafd8108410 0x7fafd8108830 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:09.415 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.307+0000 7fafdde00700 1 -- 172.21.15.90:0/1140348402 >> 172.21.15.90:0/1140348402 conn(0x7fafd8103a50 msgr2=0x7fafd8105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:09.415 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.307+0000 7fafdde00700 1 -- 172.21.15.90:0/1140348402 shutdown_connections 2024-04-24T09:11:09.415 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.307+0000 7fafdde00700 1 -- 172.21.15.90:0/1140348402 wait complete. 2024-04-24T09:11:09.415 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.308+0000 7fafdde00700 1 Processor -- start 2024-04-24T09:11:09.415 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.309+0000 7fafdde00700 1 -- start start 2024-04-24T09:11:09.415 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.309+0000 7fafdde00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fafd8108410 0x7fafd819e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:09.415 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.309+0000 7fafdde00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fafd807c0d0 con 0x7fafd8108410 2024-04-24T09:11:09.415 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.309+0000 7fafdca00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fafd8108410 0x7fafd819e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:09.415 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.309+0000 7fafdca00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fafd8108410 0x7fafd819e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40150/0 (socket says 172.21.15.90:40150) 2024-04-24T09:11:09.415 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.309+0000 7fafdca00700 1 -- 172.21.15.90:0/1622181900 learned_addr learned my addr 172.21.15.90:0/1622181900 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:11:09.415 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.310+0000 7fafdca00700 1 -- 172.21.15.90:0/1622181900 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd819eb30 con 0x7fafd8108410 2024-04-24T09:11:09.416 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.310+0000 7fafdca00700 1 --2- 172.21.15.90:0/1622181900 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fafd8108410 0x7fafd819e5f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fafcc004210 tx=0x7fafcc003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:09.416 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.310+0000 7fafd5600700 1 -- 172.21.15.90:0/1622181900 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafcc020020 con 0x7fafd8108410 2024-04-24T09:11:09.416 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.310+0000 7fafd5600700 1 -- 172.21.15.90:0/1622181900 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafcc00bba0 con 0x7fafd8108410 2024-04-24T09:11:09.416 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.310+0000 7fafd5600700 1 -- 172.21.15.90:0/1622181900 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafcc017040 con 0x7fafd8108410 2024-04-24T09:11:09.416 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.310+0000 7fafdde00700 1 -- 172.21.15.90:0/1622181900 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafd819edc0 con 0x7fafd8108410 2024-04-24T09:11:09.416 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.311+0000 7fafdde00700 1 -- 172.21.15.90:0/1622181900 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafd81a1ab0 con 0x7fafd8108410 2024-04-24T09:11:09.416 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.312+0000 7fafdde00700 1 -- 172.21.15.90:0/1622181900 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafd8064150 con 0x7fafd8108410 2024-04-24T09:11:09.416 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.312+0000 7fafd5600700 1 -- 172.21.15.90:0/1622181900 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fafcc023020 con 0x7fafd8108410 2024-04-24T09:11:09.416 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.312+0000 7fafd5600700 1 --2- 172.21.15.90:0/1622181900 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fafc0039700 0x7fafc003bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:09.416 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.313+0000 7fafd5600700 1 -- 172.21.15.90:0/1622181900 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fafcc00f050 con 0x7fafd8108410 2024-04-24T09:11:09.416 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.313+0000 7fafd7e00700 1 --2- 172.21.15.90:0/1622181900 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fafc0039700 0x7fafc003bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:09.416 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.313+0000 7fafd7e00700 1 --2- 172.21.15.90:0/1622181900 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fafc0039700 0x7fafc003bbc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fafc4006fd0 tx=0x7fafc4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:09.416 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.318+0000 7fafd5600700 1 -- 172.21.15.90:0/1622181900 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fafcc02a020 con 0x7fafd8108410 2024-04-24T09:11:09.416 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.663+0000 7fafdde00700 1 -- 172.21.15.90:0/1622181900 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fafd802e320 con 0x7fafd8108410 2024-04-24T09:11:09.416 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.666+0000 7fafd5600700 1 -- 172.21.15.90:0/1622181900 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7fafcc00bd50 con 0x7fafd8108410 2024-04-24T09:11:09.417 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.669+0000 7fafdde00700 1 -- 172.21.15.90:0/1622181900 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fafc0039700 msgr2=0x7fafc003bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:09.417 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.669+0000 7fafdde00700 1 --2- 172.21.15.90:0/1622181900 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fafc0039700 0x7fafc003bbc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fafc4006fd0 tx=0x7fafc4006e40 comp rx=0 tx=0).stop 2024-04-24T09:11:09.417 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.669+0000 7fafdde00700 1 -- 172.21.15.90:0/1622181900 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fafd8108410 msgr2=0x7fafd819e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:09.417 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.669+0000 7fafdde00700 1 --2- 172.21.15.90:0/1622181900 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fafd8108410 0x7fafd819e5f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fafcc004210 tx=0x7fafcc003d00 comp rx=0 tx=0).stop 2024-04-24T09:11:09.417 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.669+0000 7fafdde00700 1 -- 172.21.15.90:0/1622181900 shutdown_connections 2024-04-24T09:11:09.417 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.669+0000 7fafdde00700 1 --2- 172.21.15.90:0/1622181900 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fafc0039700 0x7fafc003bbc0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:09.417 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.669+0000 7fafdde00700 1 --2- 172.21.15.90:0/1622181900 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fafd8108410 0x7fafd819e5f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:09.417 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.669+0000 7fafdde00700 1 -- 172.21.15.90:0/1622181900 >> 172.21.15.90:0/1622181900 conn(0x7fafd8103a50 msgr2=0x7fafd8195910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:09.417 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.670+0000 7fafdde00700 1 -- 172.21.15.90:0/1622181900 shutdown_connections 2024-04-24T09:11:09.417 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-24T09:11:08.670+0000 7fafdde00700 1 -- 172.21.15.90:0/1622181900 wait complete. 2024-04-24T09:11:09.417 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-24T09:11:09.417 INFO:teuthology.orchestra.run.smithi090.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-24T09:11:09.417 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:11:09.417 INFO:teuthology.orchestra.run.smithi090.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-24T09:11:09.418 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:11:09.418 INFO:teuthology.orchestra.run.smithi090.stdout:Or, if you are only running a single cluster on this host: 2024-04-24T09:11:09.418 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:11:09.418 INFO:teuthology.orchestra.run.smithi090.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-24T09:11:09.418 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:11:09.418 INFO:teuthology.orchestra.run.smithi090.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-24T09:11:09.418 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:11:09.418 INFO:teuthology.orchestra.run.smithi090.stdout: ceph telemetry on 2024-04-24T09:11:09.418 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:11:09.418 INFO:teuthology.orchestra.run.smithi090.stdout:For more information see: 2024-04-24T09:11:09.418 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:11:09.418 INFO:teuthology.orchestra.run.smithi090.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-24T09:11:09.418 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:11:09.418 INFO:teuthology.orchestra.run.smithi090.stdout:Bootstrap complete. 2024-04-24T09:11:09.497 INFO:tasks.cephadm:Fetching config... 2024-04-24T09:11:09.497 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-24T09:11:09.497 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-24T09:11:09.577 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-24T09:11:09.578 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-24T09:11:09.578 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-24T09:11:09.660 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-24T09:11:09.660 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-24T09:11:09.660 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-24T09:11:09.772 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-24T09:11:09.772 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-24T09:11:09.772 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-24T09:11:09.855 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-24T09:11:09.855 DEBUG:teuthology.orchestra.run.smithi090:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDja/7j8fYYTD44wRLm7SUM4mZ0M5HulNpxXPWEYnjrmIB8sx4wwfBTPb5TbjIat4ZCIhjUCI7v07EyDtl3xRuvTuSSzX6BgMXX7eM/ZMFkDud7LXypYEfUtjAqe1zlub/BH1betO9E6ZfExXVgNp4H9m+k6H9JY+bZx1005epsH+/eIw0PeEtioC86oWkbE3ieJVkP/GbhUEvaP95oc1D+y86QVNiQQbk26KnRzzwU8K0YcXceABY1qugtNMIpAbJay0Am7RZyTelvkLl5BX6+wlPnyRXkUDOV67nzuwUXCi3lUQeEgHt7jTpfx+Wzn0z4HTThNQxHdUxhG50AN5UK4TRCmTAYCkGRJ+MLa6Ldh/hS5+TVvtZP73oSlmGoU+VVimk4ug9cNT8FiJUdeGhlcE4RRlK9YQgWrhsdtJohcJHpZrS+p+YWOrTHIMA8xtWHS38QOFybuss4ksFqo7nuIlekbro3YqUnyTUCKNCsVkmMMZ4TffJRnDi1RNiPGzU= ceph-30ee9be2-021a-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-24T09:11:09.954 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:09 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/1622181900' entity='client.admin' 2024-04-24T09:11:09.989 INFO:teuthology.orchestra.run.smithi090.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDja/7j8fYYTD44wRLm7SUM4mZ0M5HulNpxXPWEYnjrmIB8sx4wwfBTPb5TbjIat4ZCIhjUCI7v07EyDtl3xRuvTuSSzX6BgMXX7eM/ZMFkDud7LXypYEfUtjAqe1zlub/BH1betO9E6ZfExXVgNp4H9m+k6H9JY+bZx1005epsH+/eIw0PeEtioC86oWkbE3ieJVkP/GbhUEvaP95oc1D+y86QVNiQQbk26KnRzzwU8K0YcXceABY1qugtNMIpAbJay0Am7RZyTelvkLl5BX6+wlPnyRXkUDOV67nzuwUXCi3lUQeEgHt7jTpfx+Wzn0z4HTThNQxHdUxhG50AN5UK4TRCmTAYCkGRJ+MLa6Ldh/hS5+TVvtZP73oSlmGoU+VVimk4ug9cNT8FiJUdeGhlcE4RRlK9YQgWrhsdtJohcJHpZrS+p+YWOrTHIMA8xtWHS38QOFybuss4ksFqo7nuIlekbro3YqUnyTUCKNCsVkmMMZ4TffJRnDi1RNiPGzU= ceph-30ee9be2-021a-11ef-bc93-c7b262605968 2024-04-24T09:11:10.028 DEBUG:teuthology.orchestra.run.smithi114:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDja/7j8fYYTD44wRLm7SUM4mZ0M5HulNpxXPWEYnjrmIB8sx4wwfBTPb5TbjIat4ZCIhjUCI7v07EyDtl3xRuvTuSSzX6BgMXX7eM/ZMFkDud7LXypYEfUtjAqe1zlub/BH1betO9E6ZfExXVgNp4H9m+k6H9JY+bZx1005epsH+/eIw0PeEtioC86oWkbE3ieJVkP/GbhUEvaP95oc1D+y86QVNiQQbk26KnRzzwU8K0YcXceABY1qugtNMIpAbJay0Am7RZyTelvkLl5BX6+wlPnyRXkUDOV67nzuwUXCi3lUQeEgHt7jTpfx+Wzn0z4HTThNQxHdUxhG50AN5UK4TRCmTAYCkGRJ+MLa6Ldh/hS5+TVvtZP73oSlmGoU+VVimk4ug9cNT8FiJUdeGhlcE4RRlK9YQgWrhsdtJohcJHpZrS+p+YWOrTHIMA8xtWHS38QOFybuss4ksFqo7nuIlekbro3YqUnyTUCKNCsVkmMMZ4TffJRnDi1RNiPGzU= ceph-30ee9be2-021a-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-24T09:11:10.069 INFO:teuthology.orchestra.run.smithi114.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDja/7j8fYYTD44wRLm7SUM4mZ0M5HulNpxXPWEYnjrmIB8sx4wwfBTPb5TbjIat4ZCIhjUCI7v07EyDtl3xRuvTuSSzX6BgMXX7eM/ZMFkDud7LXypYEfUtjAqe1zlub/BH1betO9E6ZfExXVgNp4H9m+k6H9JY+bZx1005epsH+/eIw0PeEtioC86oWkbE3ieJVkP/GbhUEvaP95oc1D+y86QVNiQQbk26KnRzzwU8K0YcXceABY1qugtNMIpAbJay0Am7RZyTelvkLl5BX6+wlPnyRXkUDOV67nzuwUXCi3lUQeEgHt7jTpfx+Wzn0z4HTThNQxHdUxhG50AN5UK4TRCmTAYCkGRJ+MLa6Ldh/hS5+TVvtZP73oSlmGoU+VVimk4ug9cNT8FiJUdeGhlcE4RRlK9YQgWrhsdtJohcJHpZrS+p+YWOrTHIMA8xtWHS38QOFybuss4ksFqo7nuIlekbro3YqUnyTUCKNCsVkmMMZ4TffJRnDi1RNiPGzU= ceph-30ee9be2-021a-11ef-bc93-c7b262605968 2024-04-24T09:11:10.081 DEBUG:teuthology.orchestra.run.smithi149:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDja/7j8fYYTD44wRLm7SUM4mZ0M5HulNpxXPWEYnjrmIB8sx4wwfBTPb5TbjIat4ZCIhjUCI7v07EyDtl3xRuvTuSSzX6BgMXX7eM/ZMFkDud7LXypYEfUtjAqe1zlub/BH1betO9E6ZfExXVgNp4H9m+k6H9JY+bZx1005epsH+/eIw0PeEtioC86oWkbE3ieJVkP/GbhUEvaP95oc1D+y86QVNiQQbk26KnRzzwU8K0YcXceABY1qugtNMIpAbJay0Am7RZyTelvkLl5BX6+wlPnyRXkUDOV67nzuwUXCi3lUQeEgHt7jTpfx+Wzn0z4HTThNQxHdUxhG50AN5UK4TRCmTAYCkGRJ+MLa6Ldh/hS5+TVvtZP73oSlmGoU+VVimk4ug9cNT8FiJUdeGhlcE4RRlK9YQgWrhsdtJohcJHpZrS+p+YWOrTHIMA8xtWHS38QOFybuss4ksFqo7nuIlekbro3YqUnyTUCKNCsVkmMMZ4TffJRnDi1RNiPGzU= ceph-30ee9be2-021a-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-24T09:11:10.126 INFO:teuthology.orchestra.run.smithi149.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDja/7j8fYYTD44wRLm7SUM4mZ0M5HulNpxXPWEYnjrmIB8sx4wwfBTPb5TbjIat4ZCIhjUCI7v07EyDtl3xRuvTuSSzX6BgMXX7eM/ZMFkDud7LXypYEfUtjAqe1zlub/BH1betO9E6ZfExXVgNp4H9m+k6H9JY+bZx1005epsH+/eIw0PeEtioC86oWkbE3ieJVkP/GbhUEvaP95oc1D+y86QVNiQQbk26KnRzzwU8K0YcXceABY1qugtNMIpAbJay0Am7RZyTelvkLl5BX6+wlPnyRXkUDOV67nzuwUXCi3lUQeEgHt7jTpfx+Wzn0z4HTThNQxHdUxhG50AN5UK4TRCmTAYCkGRJ+MLa6Ldh/hS5+TVvtZP73oSlmGoU+VVimk4ug9cNT8FiJUdeGhlcE4RRlK9YQgWrhsdtJohcJHpZrS+p+YWOrTHIMA8xtWHS38QOFybuss4ksFqo7nuIlekbro3YqUnyTUCKNCsVkmMMZ4TffJRnDi1RNiPGzU= ceph-30ee9be2-021a-11ef-bc93-c7b262605968 2024-04-24T09:11:10.139 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-24T09:11:10.530 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:11:13.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:13.452+0000 7fbbe4600700 1 -- 172.21.15.90:0/2881046557 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbbdc104790 msgr2=0x7fbbdc104bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:13.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:13.452+0000 7fbbe4600700 1 --2- 172.21.15.90:0/2881046557 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbbdc104790 0x7fbbdc104bb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0009910 tx=0x7fbbd0009c20 comp rx=0 tx=0).stop 2024-04-24T09:11:13.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:13.452+0000 7fbbe4600700 1 -- 172.21.15.90:0/2881046557 shutdown_connections 2024-04-24T09:11:13.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:13.452+0000 7fbbe4600700 1 --2- 172.21.15.90:0/2881046557 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbbdc104790 0x7fbbdc104bb0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:13.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:13.452+0000 7fbbe4600700 1 -- 172.21.15.90:0/2881046557 >> 172.21.15.90:0/2881046557 conn(0x7fbbdc0ffd30 msgr2=0x7fbbdc102170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:13.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:13.453+0000 7fbbe4600700 1 -- 172.21.15.90:0/2881046557 shutdown_connections 2024-04-24T09:11:13.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:13.453+0000 7fbbe4600700 1 -- 172.21.15.90:0/2881046557 wait complete. 2024-04-24T09:11:13.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:13.454+0000 7fbbe4600700 1 Processor -- start 2024-04-24T09:11:13.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:13.454+0000 7fbbe4600700 1 -- start start 2024-04-24T09:11:13.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:13.454+0000 7fbbe4600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbbdc104790 0x7fbbdc199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:13.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:13.454+0000 7fbbe4600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fbbdc10a820 con 0x7fbbdc104790 2024-04-24T09:11:13.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:13.454+0000 7fbbe1c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbbdc104790 0x7fbbdc199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:13.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:13.454+0000 7fbbe1c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbbdc104790 0x7fbbdc199e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40176/0 (socket says 172.21.15.90:40176) 2024-04-24T09:11:13.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:13.454+0000 7fbbe1c00700 1 -- 172.21.15.90:0/4215933997 learned_addr learned my addr 172.21.15.90:0/4215933997 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:11:13.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:13.455+0000 7fbbe1c00700 1 -- 172.21.15.90:0/4215933997 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbdc19a3b0 con 0x7fbbdc104790 2024-04-24T09:11:13.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:13.455+0000 7fbbe1c00700 1 --2- 172.21.15.90:0/4215933997 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbbdc104790 0x7fbbdc199e70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0005850 tx=0x7fbbd0005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:13.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:13.455+0000 7fbbda000700 1 -- 172.21.15.90:0/4215933997 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbbd0026070 con 0x7fbbdc104790 2024-04-24T09:11:13.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:13.456+0000 7fbbe4600700 1 -- 172.21.15.90:0/4215933997 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbdc19a640 con 0x7fbbdc104790 2024-04-24T09:11:13.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:13.456+0000 7fbbda000700 1 -- 172.21.15.90:0/4215933997 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbd00044a0 con 0x7fbbdc104790 2024-04-24T09:11:13.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:13.456+0000 7fbbda000700 1 -- 172.21.15.90:0/4215933997 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbbd0003ab0 con 0x7fbbdc104790 2024-04-24T09:11:13.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:13.456+0000 7fbbe4600700 1 -- 172.21.15.90:0/4215933997 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbdc19d330 con 0x7fbbdc104790 2024-04-24T09:11:13.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:13.457+0000 7fbbe4600700 1 -- 172.21.15.90:0/4215933997 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbdc10afa0 con 0x7fbbdc104790 2024-04-24T09:11:13.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:13.457+0000 7fbbda000700 1 -- 172.21.15.90:0/4215933997 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fbbd0029070 con 0x7fbbdc104790 2024-04-24T09:11:13.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:13.457+0000 7fbbda000700 1 --2- 172.21.15.90:0/4215933997 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fbbc40396b0 0x7fbbc403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:13.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:13.457+0000 7fbbda000700 1 -- 172.21.15.90:0/4215933997 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbbd001e070 con 0x7fbbdc104790 2024-04-24T09:11:13.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:13.458+0000 7fbbe1200700 1 --2- 172.21.15.90:0/4215933997 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fbbc40396b0 0x7fbbc403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:13.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:13.458+0000 7fbbe1200700 1 --2- 172.21.15.90:0/4215933997 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fbbc40396b0 0x7fbbc403bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbbc8006fd0 tx=0x7fbbc8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:13.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:13.468+0000 7fbbda000700 1 -- 172.21.15.90:0/4215933997 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbbd0013070 con 0x7fbbdc104790 2024-04-24T09:11:13.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:13.694+0000 7fbbe4600700 1 -- 172.21.15.90:0/4215933997 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fbbdc02e0f0 con 0x7fbbdc104790 2024-04-24T09:11:13.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:13.698+0000 7fbbda000700 1 -- 172.21.15.90:0/4215933997 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fbbd0030020 con 0x7fbbdc104790 2024-04-24T09:11:13.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:13.702+0000 7fbbe4600700 1 -- 172.21.15.90:0/4215933997 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fbbc40396b0 msgr2=0x7fbbc403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:13.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:13.702+0000 7fbbe4600700 1 --2- 172.21.15.90:0/4215933997 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fbbc40396b0 0x7fbbc403bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbbc8006fd0 tx=0x7fbbc8006e40 comp rx=0 tx=0).stop 2024-04-24T09:11:13.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:13.702+0000 7fbbe4600700 1 -- 172.21.15.90:0/4215933997 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbbdc104790 msgr2=0x7fbbdc199e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:13.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:13.702+0000 7fbbe4600700 1 --2- 172.21.15.90:0/4215933997 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbbdc104790 0x7fbbdc199e70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0005850 tx=0x7fbbd0005030 comp rx=0 tx=0).stop 2024-04-24T09:11:13.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:13.702+0000 7fbbe4600700 1 -- 172.21.15.90:0/4215933997 shutdown_connections 2024-04-24T09:11:13.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:13.702+0000 7fbbe4600700 1 --2- 172.21.15.90:0/4215933997 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fbbc40396b0 0x7fbbc403bb70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:13.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:13.702+0000 7fbbe4600700 1 --2- 172.21.15.90:0/4215933997 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbbdc104790 0x7fbbdc199e70 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:13.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:13.702+0000 7fbbe4600700 1 -- 172.21.15.90:0/4215933997 >> 172.21.15.90:0/4215933997 conn(0x7fbbdc0ffd30 msgr2=0x7fbbdc100760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:13.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:13.702+0000 7fbbe4600700 1 -- 172.21.15.90:0/4215933997 shutdown_connections 2024-04-24T09:11:13.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:13.702+0000 7fbbe4600700 1 -- 172.21.15.90:0/4215933997 wait complete. 2024-04-24T09:11:14.483 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-24T09:11:14.483 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-24T09:11:15.071 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:14 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/4215933997' entity='client.admin' 2024-04-24T09:11:15.595 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:11:16.769 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:16 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:11:16.770 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:16 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:11:16.770 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:16 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi090", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:11:16.770 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:16 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:11:16.770 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:16 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:11:16.770 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:16 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:11:17.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:17.546+0000 7efd83e00700 1 -- 172.21.15.90:0/1890150423 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd84076b20 msgr2=0x7efd84074f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:17.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:17.546+0000 7efd83e00700 1 --2- 172.21.15.90:0/1890150423 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd84076b20 0x7efd84074f80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7efd7400b0a0 tx=0x7efd7400b3b0 comp rx=0 tx=0).stop 2024-04-24T09:11:17.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:17.546+0000 7efd83e00700 1 -- 172.21.15.90:0/1890150423 shutdown_connections 2024-04-24T09:11:17.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:17.546+0000 7efd83e00700 1 --2- 172.21.15.90:0/1890150423 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd84076b20 0x7efd84074f80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:17.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:17.546+0000 7efd83e00700 1 -- 172.21.15.90:0/1890150423 >> 172.21.15.90:0/1890150423 conn(0x7efd840fd9a0 msgr2=0x7efd840ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:17.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:17.546+0000 7efd83e00700 1 -- 172.21.15.90:0/1890150423 shutdown_connections 2024-04-24T09:11:17.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:17.546+0000 7efd83e00700 1 -- 172.21.15.90:0/1890150423 wait complete. 2024-04-24T09:11:17.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:17.547+0000 7efd83e00700 1 Processor -- start 2024-04-24T09:11:17.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:17.548+0000 7efd83e00700 1 -- start start 2024-04-24T09:11:17.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:17.548+0000 7efd83e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd84076b20 0x7efd8419e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:17.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:17.548+0000 7efd83e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efd8410eb00 con 0x7efd84076b20 2024-04-24T09:11:17.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:17.548+0000 7efd82a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd84076b20 0x7efd8419e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:17.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:17.548+0000 7efd82a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd84076b20 0x7efd8419e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48848/0 (socket says 172.21.15.90:48848) 2024-04-24T09:11:17.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:17.548+0000 7efd82a00700 1 -- 172.21.15.90:0/4205953363 learned_addr learned my addr 172.21.15.90:0/4205953363 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:11:17.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:17.549+0000 7efd82a00700 1 -- 172.21.15.90:0/4205953363 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd8419e540 con 0x7efd84076b20 2024-04-24T09:11:17.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:17.549+0000 7efd82a00700 1 --2- 172.21.15.90:0/4205953363 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd84076b20 0x7efd8419e000 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efd74007890 tx=0x7efd74009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:17.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:17.549+0000 7efd7be00700 1 -- 172.21.15.90:0/4205953363 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd74017040 con 0x7efd84076b20 2024-04-24T09:11:17.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:17.549+0000 7efd7be00700 1 -- 172.21.15.90:0/4205953363 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd7400bd50 con 0x7efd84076b20 2024-04-24T09:11:17.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:17.549+0000 7efd83e00700 1 -- 172.21.15.90:0/4205953363 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd8419e7d0 con 0x7efd84076b20 2024-04-24T09:11:17.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:17.549+0000 7efd7be00700 1 -- 172.21.15.90:0/4205953363 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd7401d7a0 con 0x7efd84076b20 2024-04-24T09:11:17.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:17.549+0000 7efd83e00700 1 -- 172.21.15.90:0/4205953363 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd841a14c0 con 0x7efd84076b20 2024-04-24T09:11:17.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:17.550+0000 7efd83e00700 1 -- 172.21.15.90:0/4205953363 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd8410f1b0 con 0x7efd84076b20 2024-04-24T09:11:17.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:17.551+0000 7efd7be00700 1 -- 172.21.15.90:0/4205953363 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7efd74024020 con 0x7efd84076b20 2024-04-24T09:11:17.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:17.552+0000 7efd7be00700 1 --2- 172.21.15.90:0/4205953363 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7efd64039700 0x7efd6403bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:17.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:17.552+0000 7efd7be00700 1 -- 172.21.15.90:0/4205953363 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7efd74022020 con 0x7efd84076b20 2024-04-24T09:11:17.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:17.552+0000 7efd72000700 1 --2- 172.21.15.90:0/4205953363 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7efd64039700 0x7efd6403bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:17.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:17.553+0000 7efd72000700 1 --2- 172.21.15.90:0/4205953363 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7efd64039700 0x7efd6403bbc0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7efd6c006fd0 tx=0x7efd6c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:17.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:17.557+0000 7efd7be00700 1 -- 172.21.15.90:0/4205953363 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efd7402b020 con 0x7efd84076b20 2024-04-24T09:11:17.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:17.784+0000 7efd83e00700 1 -- 172.21.15.90:0/4205953363 --> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7efd8402e0f0 con 0x7efd64039700 2024-04-24T09:11:17.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:17.792+0000 7efd7be00700 1 -- 172.21.15.90:0/4205953363 <== mgr.14152 v2:172.21.15.90:6800/4261662956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efd8402e0f0 con 0x7efd64039700 2024-04-24T09:11:17.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:17.794+0000 7efd83e00700 1 -- 172.21.15.90:0/4205953363 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7efd64039700 msgr2=0x7efd6403bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:17.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:17.794+0000 7efd83e00700 1 --2- 172.21.15.90:0/4205953363 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7efd64039700 0x7efd6403bbc0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7efd6c006fd0 tx=0x7efd6c006e40 comp rx=0 tx=0).stop 2024-04-24T09:11:17.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:17.794+0000 7efd83e00700 1 -- 172.21.15.90:0/4205953363 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd84076b20 msgr2=0x7efd8419e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:17.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:17.794+0000 7efd83e00700 1 --2- 172.21.15.90:0/4205953363 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd84076b20 0x7efd8419e000 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efd74007890 tx=0x7efd74009790 comp rx=0 tx=0).stop 2024-04-24T09:11:17.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:17.794+0000 7efd83e00700 1 -- 172.21.15.90:0/4205953363 shutdown_connections 2024-04-24T09:11:17.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:17.794+0000 7efd83e00700 1 --2- 172.21.15.90:0/4205953363 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7efd64039700 0x7efd6403bbc0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:17.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:17.794+0000 7efd83e00700 1 --2- 172.21.15.90:0/4205953363 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd84076b20 0x7efd8419e000 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:17.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:17.794+0000 7efd83e00700 1 -- 172.21.15.90:0/4205953363 >> 172.21.15.90:0/4205953363 conn(0x7efd840fd9a0 msgr2=0x7efd840fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:17.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:17.795+0000 7efd83e00700 1 -- 172.21.15.90:0/4205953363 shutdown_connections 2024-04-24T09:11:17.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:17.795+0000 7efd83e00700 1 -- 172.21.15.90:0/4205953363 wait complete. 2024-04-24T09:11:18.637 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi114 2024-04-24T09:11:18.638 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-24T09:11:18.638 DEBUG:teuthology.orchestra.run.smithi114:> dd of=/etc/ceph/ceph.conf 2024-04-24T09:11:18.656 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-24T09:11:18.656 DEBUG:teuthology.orchestra.run.smithi114:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T09:11:18.714 INFO:tasks.cephadm:Adding host smithi114 to orchestrator... 2024-04-24T09:11:18.714 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph orch host add smithi114 2024-04-24T09:11:18.771 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:18 smithi090 ceph-mon[27825]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:11:18.771 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:18 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:11:18.771 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:18 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:11:18.771 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:18 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:11:18.771 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:18 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:11:19.102 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:11:19.851 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:19 smithi090 ceph-mon[27825]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:11:19.851 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:19 smithi090 ceph-mon[27825]: Updating smithi090:/etc/ceph/ceph.conf 2024-04-24T09:11:19.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:19 smithi090 ceph-mon[27825]: Updating smithi090:/var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T09:11:20.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:20 smithi090 ceph-mon[27825]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-04-24T09:11:20.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:20 smithi090 ceph-mon[27825]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:11:20.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:20 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:11:20.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:20 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:11:20.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:20 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:11:21.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:21.065+0000 7fa316a00700 1 -- 172.21.15.90:0/709243058 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa3101085d0 msgr2=0x7fa3101089b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:21.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:21.065+0000 7fa316a00700 1 --2- 172.21.15.90:0/709243058 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa3101085d0 0x7fa3101089b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa300009910 tx=0x7fa300009c20 comp rx=0 tx=0).stop 2024-04-24T09:11:21.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:21.065+0000 7fa316a00700 1 -- 172.21.15.90:0/709243058 shutdown_connections 2024-04-24T09:11:21.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:21.065+0000 7fa316a00700 1 --2- 172.21.15.90:0/709243058 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa3101085d0 0x7fa3101089b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:21.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:21.065+0000 7fa316a00700 1 -- 172.21.15.90:0/709243058 >> 172.21.15.90:0/709243058 conn(0x7fa3100fe030 msgr2=0x7fa310100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:21.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:21.065+0000 7fa316a00700 1 -- 172.21.15.90:0/709243058 shutdown_connections 2024-04-24T09:11:21.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:21.066+0000 7fa316a00700 1 -- 172.21.15.90:0/709243058 wait complete. 2024-04-24T09:11:21.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:21.067+0000 7fa316a00700 1 Processor -- start 2024-04-24T09:11:21.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:21.067+0000 7fa316a00700 1 -- start start 2024-04-24T09:11:21.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:21.067+0000 7fa316a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa3101085d0 0x7fa310197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:21.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:21.067+0000 7fa316a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa31010cb90 con 0x7fa3101085d0 2024-04-24T09:11:21.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:21.068+0000 7fa30fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa3101085d0 0x7fa310197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:21.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:21.068+0000 7fa30fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa3101085d0 0x7fa310197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48868/0 (socket says 172.21.15.90:48868) 2024-04-24T09:11:21.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:21.068+0000 7fa30fe00700 1 -- 172.21.15.90:0/1763188683 learned_addr learned my addr 172.21.15.90:0/1763188683 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:11:21.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:21.068+0000 7fa30fe00700 1 -- 172.21.15.90:0/1763188683 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3101984d0 con 0x7fa3101085d0 2024-04-24T09:11:21.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:21.068+0000 7fa30fe00700 1 --2- 172.21.15.90:0/1763188683 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa3101085d0 0x7fa310197f90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa300005850 tx=0x7fa300005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:21.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:21.069+0000 7fa30cc00700 1 -- 172.21.15.90:0/1763188683 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa300026070 con 0x7fa3101085d0 2024-04-24T09:11:21.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:21.069+0000 7fa30cc00700 1 -- 172.21.15.90:0/1763188683 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa30000bc30 con 0x7fa3101085d0 2024-04-24T09:11:21.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:21.069+0000 7fa316a00700 1 -- 172.21.15.90:0/1763188683 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa310198760 con 0x7fa3101085d0 2024-04-24T09:11:21.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:21.069+0000 7fa30cc00700 1 -- 172.21.15.90:0/1763188683 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa300003a20 con 0x7fa3101085d0 2024-04-24T09:11:21.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:21.069+0000 7fa316a00700 1 -- 172.21.15.90:0/1763188683 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa31019c860 con 0x7fa3101085d0 2024-04-24T09:11:21.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:21.070+0000 7fa30cc00700 1 -- 172.21.15.90:0/1763188683 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fa300029070 con 0x7fa3101085d0 2024-04-24T09:11:21.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:21.070+0000 7fa316a00700 1 -- 172.21.15.90:0/1763188683 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa310103c40 con 0x7fa3101085d0 2024-04-24T09:11:21.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:21.071+0000 7fa30cc00700 1 --2- 172.21.15.90:0/1763188683 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa2f40396b0 0x7fa2f403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:21.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:21.071+0000 7fa30cc00700 1 -- 172.21.15.90:0/1763188683 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa30001e070 con 0x7fa3101085d0 2024-04-24T09:11:21.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:21.071+0000 7fa30f400700 1 --2- 172.21.15.90:0/1763188683 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa2f40396b0 0x7fa2f403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:21.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:21.071+0000 7fa30f400700 1 --2- 172.21.15.90:0/1763188683 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa2f40396b0 0x7fa2f403bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc006fd0 tx=0x7fa2fc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:21.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:21.076+0000 7fa30cc00700 1 -- 172.21.15.90:0/1763188683 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa300013070 con 0x7fa3101085d0 2024-04-24T09:11:21.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:21.311+0000 7fa316a00700 1 -- 172.21.15.90:0/1763188683 --> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi114", "target": ["mon-mgr", ""]}) -- 0x7fa3101992f0 con 0x7fa2f40396b0 2024-04-24T09:11:21.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:21 smithi090 ceph-mon[27825]: Updating smithi090:/var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T09:11:22.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:22 smithi090 ceph-mon[27825]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi114", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:11:22.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:22 smithi090 ceph-mon[27825]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:11:23.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:23 smithi090 ceph-mon[27825]: Deploying cephadm binary to smithi114 2024-04-24T09:11:24.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:24 smithi090 ceph-mon[27825]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:11:25.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:25.934+0000 7fa30cc00700 1 -- 172.21.15.90:0/1763188683 <== mgr.14152 v2:172.21.15.90:6800/4261662956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fa3101992f0 con 0x7fa2f40396b0 2024-04-24T09:11:25.935 INFO:teuthology.orchestra.run.smithi090.stdout:Added host 'smithi114' with addr '172.21.15.114' 2024-04-24T09:11:25.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:25.937+0000 7fa316a00700 1 -- 172.21.15.90:0/1763188683 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa2f40396b0 msgr2=0x7fa2f403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:25.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:25.937+0000 7fa316a00700 1 --2- 172.21.15.90:0/1763188683 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa2f40396b0 0x7fa2f403bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc006fd0 tx=0x7fa2fc006e40 comp rx=0 tx=0).stop 2024-04-24T09:11:25.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:25.937+0000 7fa316a00700 1 -- 172.21.15.90:0/1763188683 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa3101085d0 msgr2=0x7fa310197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:25.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:25.937+0000 7fa316a00700 1 --2- 172.21.15.90:0/1763188683 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa3101085d0 0x7fa310197f90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa300005850 tx=0x7fa300005030 comp rx=0 tx=0).stop 2024-04-24T09:11:25.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:25.937+0000 7fa316a00700 1 -- 172.21.15.90:0/1763188683 shutdown_connections 2024-04-24T09:11:25.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:25.937+0000 7fa316a00700 1 --2- 172.21.15.90:0/1763188683 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa2f40396b0 0x7fa2f403bb70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:25.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:25.937+0000 7fa316a00700 1 --2- 172.21.15.90:0/1763188683 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa3101085d0 0x7fa310197f90 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:25.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:25.937+0000 7fa316a00700 1 -- 172.21.15.90:0/1763188683 >> 172.21.15.90:0/1763188683 conn(0x7fa3100fe030 msgr2=0x7fa3100ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:25.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:25.938+0000 7fa316a00700 1 -- 172.21.15.90:0/1763188683 shutdown_connections 2024-04-24T09:11:25.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:25.938+0000 7fa316a00700 1 -- 172.21.15.90:0/1763188683 wait complete. 2024-04-24T09:11:26.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:26 smithi090 ceph-mon[27825]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:11:26.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:26 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:11:26.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:26 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:11:26.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:26 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:11:26.708 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-24T09:11:27.142 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:11:27.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:27 smithi090 ceph-mon[27825]: Added host smithi114 2024-04-24T09:11:27.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:27 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:11:28.781 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:28 smithi090 ceph-mon[27825]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:11:29.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:29.303+0000 7f0a89a00700 1 -- 172.21.15.90:0/3436540348 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a841085d0 msgr2=0x7f0a841089b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:29.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:29.303+0000 7f0a89a00700 1 --2- 172.21.15.90:0/3436540348 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a841085d0 0x7f0a841089b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0a7400b0a0 tx=0x7f0a7400b3b0 comp rx=0 tx=0).stop 2024-04-24T09:11:29.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:29.303+0000 7f0a89a00700 1 -- 172.21.15.90:0/3436540348 shutdown_connections 2024-04-24T09:11:29.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:29.303+0000 7f0a89a00700 1 --2- 172.21.15.90:0/3436540348 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a841085d0 0x7f0a841089b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:29.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:29.303+0000 7f0a89a00700 1 -- 172.21.15.90:0/3436540348 >> 172.21.15.90:0/3436540348 conn(0x7f0a840fe030 msgr2=0x7f0a84100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:29.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:29.303+0000 7f0a89a00700 1 -- 172.21.15.90:0/3436540348 shutdown_connections 2024-04-24T09:11:29.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:29.303+0000 7f0a89a00700 1 -- 172.21.15.90:0/3436540348 wait complete. 2024-04-24T09:11:29.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:29.304+0000 7f0a89a00700 1 Processor -- start 2024-04-24T09:11:29.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:29.305+0000 7f0a89a00700 1 -- start start 2024-04-24T09:11:29.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:29.305+0000 7f0a89a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a841085d0 0x7f0a84074df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:29.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:29.305+0000 7f0a89a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8410cb90 con 0x7f0a841085d0 2024-04-24T09:11:29.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:29.305+0000 7f0a82a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a841085d0 0x7f0a84074df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:29.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:29.306+0000 7f0a82a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a841085d0 0x7f0a84074df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56028/0 (socket says 172.21.15.90:56028) 2024-04-24T09:11:29.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:29.306+0000 7f0a82a00700 1 -- 172.21.15.90:0/710212332 learned_addr learned my addr 172.21.15.90:0/710212332 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:11:29.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:29.306+0000 7f0a82a00700 1 -- 172.21.15.90:0/710212332 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a84076ce0 con 0x7f0a841085d0 2024-04-24T09:11:29.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:29.306+0000 7f0a82a00700 1 --2- 172.21.15.90:0/710212332 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a841085d0 0x7f0a84074df0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0a74000f80 tx=0x7f0a74003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:29.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:29.306+0000 7f0a7be00700 1 -- 172.21.15.90:0/710212332 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a74017040 con 0x7f0a841085d0 2024-04-24T09:11:29.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:29.306+0000 7f0a7be00700 1 -- 172.21.15.90:0/710212332 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a74022070 con 0x7f0a841085d0 2024-04-24T09:11:29.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:29.307+0000 7f0a89a00700 1 -- 172.21.15.90:0/710212332 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a84075330 con 0x7f0a841085d0 2024-04-24T09:11:29.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:29.307+0000 7f0a7be00700 1 -- 172.21.15.90:0/710212332 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a7401d440 con 0x7f0a841085d0 2024-04-24T09:11:29.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:29.307+0000 7f0a89a00700 1 -- 172.21.15.90:0/710212332 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a840757f0 con 0x7f0a841085d0 2024-04-24T09:11:29.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:29.308+0000 7f0a89a00700 1 -- 172.21.15.90:0/710212332 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a84103c40 con 0x7f0a841085d0 2024-04-24T09:11:29.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:29.308+0000 7f0a7be00700 1 -- 172.21.15.90:0/710212332 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f0a74004000 con 0x7f0a841085d0 2024-04-24T09:11:29.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:29.308+0000 7f0a7be00700 1 --2- 172.21.15.90:0/710212332 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f0a68039700 0x7f0a6803bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:29.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:29.308+0000 7f0a7be00700 1 -- 172.21.15.90:0/710212332 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0a7400f050 con 0x7f0a841085d0 2024-04-24T09:11:29.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:29.309+0000 7f0a7a000700 1 --2- 172.21.15.90:0/710212332 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f0a68039700 0x7f0a6803bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:29.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:29.309+0000 7f0a7a000700 1 --2- 172.21.15.90:0/710212332 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f0a68039700 0x7f0a6803bbc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c006fd0 tx=0x7f0a6c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:29.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:29.314+0000 7f0a7be00700 1 -- 172.21.15.90:0/710212332 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0a74007e20 con 0x7f0a841085d0 2024-04-24T09:11:29.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:29.548+0000 7f0a89a00700 1 -- 172.21.15.90:0/710212332 --> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0a84076390 con 0x7f0a68039700 2024-04-24T09:11:29.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:29.550+0000 7f0a7be00700 1 -- 172.21.15.90:0/710212332 <== mgr.14152 v2:172.21.15.90:6800/4261662956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7f0a84076390 con 0x7f0a68039700 2024-04-24T09:11:29.550 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:11:29.551 INFO:teuthology.orchestra.run.smithi090.stdout:[{"addr": "172.21.15.90", "hostname": "smithi090", "labels": [], "status": ""}, {"addr": "172.21.15.114", "hostname": "smithi114", "labels": [], "status": ""}] 2024-04-24T09:11:29.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:29.553+0000 7f0a89a00700 1 -- 172.21.15.90:0/710212332 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f0a68039700 msgr2=0x7f0a6803bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:29.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:29.553+0000 7f0a89a00700 1 --2- 172.21.15.90:0/710212332 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f0a68039700 0x7f0a6803bbc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c006fd0 tx=0x7f0a6c006e40 comp rx=0 tx=0).stop 2024-04-24T09:11:29.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:29.553+0000 7f0a89a00700 1 -- 172.21.15.90:0/710212332 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a841085d0 msgr2=0x7f0a84074df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:29.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:29.553+0000 7f0a89a00700 1 --2- 172.21.15.90:0/710212332 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a841085d0 0x7f0a84074df0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0a74000f80 tx=0x7f0a74003ce0 comp rx=0 tx=0).stop 2024-04-24T09:11:29.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:29.553+0000 7f0a89a00700 1 -- 172.21.15.90:0/710212332 shutdown_connections 2024-04-24T09:11:29.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:29.553+0000 7f0a89a00700 1 --2- 172.21.15.90:0/710212332 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f0a68039700 0x7f0a6803bbc0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:29.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:29.553+0000 7f0a89a00700 1 --2- 172.21.15.90:0/710212332 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a841085d0 0x7f0a84074df0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:29.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:29.553+0000 7f0a89a00700 1 -- 172.21.15.90:0/710212332 >> 172.21.15.90:0/710212332 conn(0x7f0a840fe030 msgr2=0x7f0a840ff730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:29.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:29.554+0000 7f0a89a00700 1 -- 172.21.15.90:0/710212332 shutdown_connections 2024-04-24T09:11:29.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:29.554+0000 7f0a89a00700 1 -- 172.21.15.90:0/710212332 wait complete. 2024-04-24T09:11:29.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:29 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:11:29.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:29 smithi090 ceph-mon[27825]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:11:29.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:29 smithi090 ceph-mon[27825]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T09:11:30.359 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi149 2024-04-24T09:11:30.360 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-04-24T09:11:30.360 DEBUG:teuthology.orchestra.run.smithi149:> dd of=/etc/ceph/ceph.conf 2024-04-24T09:11:30.379 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-04-24T09:11:30.379 DEBUG:teuthology.orchestra.run.smithi149:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T09:11:30.436 INFO:tasks.cephadm:Adding host smithi149 to orchestrator... 2024-04-24T09:11:30.436 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph orch host add smithi149 2024-04-24T09:11:30.867 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:11:32.529 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:32 smithi090 ceph-mon[27825]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:11:32.529 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:32 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:11:32.529 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:32 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:11:32.529 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:32 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:11:32.529 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:32 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:11:32.529 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:32 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi114", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:11:32.529 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:32 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:11:32.529 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:32 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:11:33.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:33.110+0000 7f84a3e00700 1 -- 172.21.15.90:0/1311085613 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84a4069f10 msgr2=0x7f84a406a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:33.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:33.110+0000 7f84a3e00700 1 --2- 172.21.15.90:0/1311085613 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84a4069f10 0x7f84a406a2f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8494009910 tx=0x7f8494009c20 comp rx=0 tx=0).stop 2024-04-24T09:11:33.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:33.110+0000 7f84a3e00700 1 -- 172.21.15.90:0/1311085613 shutdown_connections 2024-04-24T09:11:33.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:33.110+0000 7f84a3e00700 1 --2- 172.21.15.90:0/1311085613 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84a4069f10 0x7f84a406a2f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:33.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:33.110+0000 7f84a3e00700 1 -- 172.21.15.90:0/1311085613 >> 172.21.15.90:0/1311085613 conn(0x7f84a4076da0 msgr2=0x7f84a40771b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:33.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:33.110+0000 7f84a3e00700 1 -- 172.21.15.90:0/1311085613 shutdown_connections 2024-04-24T09:11:33.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:33.110+0000 7f84a3e00700 1 -- 172.21.15.90:0/1311085613 wait complete. 2024-04-24T09:11:33.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:33.111+0000 7f84a3e00700 1 Processor -- start 2024-04-24T09:11:33.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:33.111+0000 7f84a3e00700 1 -- start start 2024-04-24T09:11:33.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:33.112+0000 7f84a3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84a4069f10 0x7f84a419a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:33.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:33.112+0000 7f84a3e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f84a410ebd0 con 0x7f84a4069f10 2024-04-24T09:11:33.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:33.112+0000 7f84a2a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84a4069f10 0x7f84a419a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:33.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:33.112+0000 7f84a2a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84a4069f10 0x7f84a419a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56042/0 (socket says 172.21.15.90:56042) 2024-04-24T09:11:33.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:33.112+0000 7f84a2a00700 1 -- 172.21.15.90:0/3655141332 learned_addr learned my addr 172.21.15.90:0/3655141332 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:11:33.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:33.113+0000 7f84a2a00700 1 -- 172.21.15.90:0/3655141332 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84a419a5f0 con 0x7f84a4069f10 2024-04-24T09:11:33.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:33.113+0000 7f84a2a00700 1 --2- 172.21.15.90:0/3655141332 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84a4069f10 0x7f84a419a0b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8494006010 tx=0x7f8494005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:33.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:33.113+0000 7f84a8a00700 1 -- 172.21.15.90:0/3655141332 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8494026070 con 0x7f84a4069f10 2024-04-24T09:11:33.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:33.113+0000 7f84a3e00700 1 -- 172.21.15.90:0/3655141332 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84a419a880 con 0x7f84a4069f10 2024-04-24T09:11:33.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:33.113+0000 7f84a8a00700 1 -- 172.21.15.90:0/3655141332 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f849400bc30 con 0x7f84a4069f10 2024-04-24T09:11:33.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:33.113+0000 7f84a8a00700 1 -- 172.21.15.90:0/3655141332 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84940039d0 con 0x7f84a4069f10 2024-04-24T09:11:33.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:33.113+0000 7f84a3e00700 1 -- 172.21.15.90:0/3655141332 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84a419ace0 con 0x7f84a4069f10 2024-04-24T09:11:33.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:33.114+0000 7f84a3e00700 1 -- 172.21.15.90:0/3655141332 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84a4101880 con 0x7f84a4069f10 2024-04-24T09:11:33.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:33.114+0000 7f84a8a00700 1 -- 172.21.15.90:0/3655141332 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f8494029070 con 0x7f84a4069f10 2024-04-24T09:11:33.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:33.115+0000 7f84a8a00700 1 --2- 172.21.15.90:0/3655141332 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f8488039650 0x7f848803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:33.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:33.115+0000 7f84a8a00700 1 -- 172.21.15.90:0/3655141332 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f849401e070 con 0x7f84a4069f10 2024-04-24T09:11:33.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:33.115+0000 7f849a000700 1 --2- 172.21.15.90:0/3655141332 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f8488039650 0x7f848803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:33.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:33.116+0000 7f849a000700 1 --2- 172.21.15.90:0/3655141332 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f8488039650 0x7f848803bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f848c00ad30 tx=0x7f848c0093f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:33.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:33.120+0000 7f84a8a00700 1 -- 172.21.15.90:0/3655141332 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8494013070 con 0x7f84a4069f10 2024-04-24T09:11:33.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:33.362+0000 7f84a3e00700 1 -- 172.21.15.90:0/3655141332 --> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi149", "target": ["mon-mgr", ""]}) -- 0x7f84a419b410 con 0x7f8488039650 2024-04-24T09:11:33.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:33 smithi090 ceph-mon[27825]: Updating smithi114:/etc/ceph/ceph.conf 2024-04-24T09:11:33.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:33 smithi090 ceph-mon[27825]: Updating smithi114:/var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T09:11:33.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:33 smithi090 ceph-mon[27825]: Updating smithi114:/etc/ceph/ceph.client.admin.keyring 2024-04-24T09:11:33.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:33 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:11:33.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:33 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:11:33.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:33 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:11:34.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:34 smithi090 ceph-mon[27825]: Updating smithi114:/var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T09:11:34.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:34 smithi090 ceph-mon[27825]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi149", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:11:34.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:34 smithi090 ceph-mon[27825]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:11:35.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:35 smithi090 ceph-mon[27825]: Deploying cephadm binary to smithi149 2024-04-24T09:11:36.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:36 smithi090 ceph-mon[27825]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:11:38.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:38.058+0000 7f84a8a00700 1 -- 172.21.15.90:0/3655141332 <== mgr.14152 v2:172.21.15.90:6800/4261662956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f84a419b410 con 0x7f8488039650 2024-04-24T09:11:38.058 INFO:teuthology.orchestra.run.smithi090.stdout:Added host 'smithi149' with addr '172.21.15.149' 2024-04-24T09:11:38.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:38.062+0000 7f84a3e00700 1 -- 172.21.15.90:0/3655141332 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f8488039650 msgr2=0x7f848803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:38.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:38.062+0000 7f84a3e00700 1 --2- 172.21.15.90:0/3655141332 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f8488039650 0x7f848803bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f848c00ad30 tx=0x7f848c0093f0 comp rx=0 tx=0).stop 2024-04-24T09:11:38.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:38.062+0000 7f84a3e00700 1 -- 172.21.15.90:0/3655141332 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84a4069f10 msgr2=0x7f84a419a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:38.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:38.062+0000 7f84a3e00700 1 --2- 172.21.15.90:0/3655141332 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84a4069f10 0x7f84a419a0b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8494006010 tx=0x7f8494005030 comp rx=0 tx=0).stop 2024-04-24T09:11:38.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:38.062+0000 7f84a3e00700 1 -- 172.21.15.90:0/3655141332 shutdown_connections 2024-04-24T09:11:38.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:38.062+0000 7f84a3e00700 1 --2- 172.21.15.90:0/3655141332 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f8488039650 0x7f848803bb10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:38.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:38.062+0000 7f84a3e00700 1 --2- 172.21.15.90:0/3655141332 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84a4069f10 0x7f84a419a0b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:38.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:38.062+0000 7f84a3e00700 1 -- 172.21.15.90:0/3655141332 >> 172.21.15.90:0/3655141332 conn(0x7f84a4076da0 msgr2=0x7f84a41098d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:38.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:38.062+0000 7f84a3e00700 1 -- 172.21.15.90:0/3655141332 shutdown_connections 2024-04-24T09:11:38.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:38.062+0000 7f84a3e00700 1 -- 172.21.15.90:0/3655141332 wait complete. 2024-04-24T09:11:38.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:38 smithi090 ceph-mon[27825]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:11:38.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:38 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:11:38.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:38 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:11:38.959 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-24T09:11:39.389 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:11:39.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:39 smithi090 ceph-mon[27825]: Added host smithi149 2024-04-24T09:11:39.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:39 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:11:39.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:39 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:11:40.947 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:40 smithi090 ceph-mon[27825]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:11:41.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:41.410+0000 7f342d800700 1 -- 172.21.15.90:0/2677067139 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34281083f0 msgr2=0x7f34281087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:41.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:41.410+0000 7f342d800700 1 --2- 172.21.15.90:0/2677067139 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34281083f0 0x7f34281087d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3418009910 tx=0x7f3418009c20 comp rx=0 tx=0).stop 2024-04-24T09:11:41.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:41.410+0000 7f342d800700 1 -- 172.21.15.90:0/2677067139 shutdown_connections 2024-04-24T09:11:41.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:41.410+0000 7f342d800700 1 --2- 172.21.15.90:0/2677067139 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34281083f0 0x7f34281087d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:41.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:41.410+0000 7f342d800700 1 -- 172.21.15.90:0/2677067139 >> 172.21.15.90:0/2677067139 conn(0x7f3428076da0 msgr2=0x7f34280771b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:41.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:41.410+0000 7f342d800700 1 -- 172.21.15.90:0/2677067139 shutdown_connections 2024-04-24T09:11:41.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:41.410+0000 7f342d800700 1 -- 172.21.15.90:0/2677067139 wait complete. 2024-04-24T09:11:41.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:41.411+0000 7f342d800700 1 Processor -- start 2024-04-24T09:11:41.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:41.412+0000 7f342d800700 1 -- start start 2024-04-24T09:11:41.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:41.412+0000 7f342d800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34281083f0 0x7f342819a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:41.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:41.412+0000 7f342d800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f342810c970 con 0x7f34281083f0 2024-04-24T09:11:41.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:41.412+0000 7f3427e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34281083f0 0x7f342819a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:41.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:41.412+0000 7f3427e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34281083f0 0x7f342819a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36782/0 (socket says 172.21.15.90:36782) 2024-04-24T09:11:41.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:41.412+0000 7f3427e00700 1 -- 172.21.15.90:0/2269864941 learned_addr learned my addr 172.21.15.90:0/2269864941 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:11:41.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:41.413+0000 7f3427e00700 1 -- 172.21.15.90:0/2269864941 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f342819a570 con 0x7f34281083f0 2024-04-24T09:11:41.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:41.413+0000 7f3427e00700 1 --2- 172.21.15.90:0/2269864941 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34281083f0 0x7f342819a030 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3418006010 tx=0x7f3418005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:41.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:41.413+0000 7f3424c00700 1 -- 172.21.15.90:0/2269864941 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3418026070 con 0x7f34281083f0 2024-04-24T09:11:41.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:41.413+0000 7f342d800700 1 -- 172.21.15.90:0/2269864941 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f342819a860 con 0x7f34281083f0 2024-04-24T09:11:41.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:41.413+0000 7f3424c00700 1 -- 172.21.15.90:0/2269864941 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f341800bbd0 con 0x7f34281083f0 2024-04-24T09:11:41.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:41.413+0000 7f3424c00700 1 -- 172.21.15.90:0/2269864941 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3418003910 con 0x7f34281083f0 2024-04-24T09:11:41.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:41.414+0000 7f342d800700 1 -- 172.21.15.90:0/2269864941 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f342819e900 con 0x7f34281083f0 2024-04-24T09:11:41.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:41.415+0000 7f3424c00700 1 -- 172.21.15.90:0/2269864941 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f3418029070 con 0x7f34281083f0 2024-04-24T09:11:41.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:41.415+0000 7f342d800700 1 -- 172.21.15.90:0/2269864941 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3428103aa0 con 0x7f34281083f0 2024-04-24T09:11:41.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:41.415+0000 7f3424c00700 1 --2- 172.21.15.90:0/2269864941 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f340c039650 0x7f340c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:41.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:41.415+0000 7f3424c00700 1 -- 172.21.15.90:0/2269864941 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f341801e070 con 0x7f34281083f0 2024-04-24T09:11:41.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:41.415+0000 7f3427400700 1 --2- 172.21.15.90:0/2269864941 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f340c039650 0x7f340c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:41.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:41.416+0000 7f3427400700 1 --2- 172.21.15.90:0/2269864941 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f340c039650 0x7f340c03bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3414006fd0 tx=0x7f3414006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:41.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:41.425+0000 7f3424c00700 1 -- 172.21.15.90:0/2269864941 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3418013070 con 0x7f34281083f0 2024-04-24T09:11:41.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:41.669+0000 7f342d800700 1 -- 172.21.15.90:0/2269864941 --> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f342819b390 con 0x7f340c039650 2024-04-24T09:11:41.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:41.671+0000 7f3424c00700 1 -- 172.21.15.90:0/2269864941 <== mgr.14152 v2:172.21.15.90:6800/4261662956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7f342819b390 con 0x7f340c039650 2024-04-24T09:11:41.671 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:11:41.671 INFO:teuthology.orchestra.run.smithi090.stdout:[{"addr": "172.21.15.90", "hostname": "smithi090", "labels": [], "status": ""}, {"addr": "172.21.15.114", "hostname": "smithi114", "labels": [], "status": ""}, {"addr": "172.21.15.149", "hostname": "smithi149", "labels": [], "status": ""}] 2024-04-24T09:11:41.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:41.674+0000 7f342d800700 1 -- 172.21.15.90:0/2269864941 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f340c039650 msgr2=0x7f340c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:41.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:41.674+0000 7f342d800700 1 --2- 172.21.15.90:0/2269864941 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f340c039650 0x7f340c03bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3414006fd0 tx=0x7f3414006e40 comp rx=0 tx=0).stop 2024-04-24T09:11:41.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:41.674+0000 7f342d800700 1 -- 172.21.15.90:0/2269864941 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34281083f0 msgr2=0x7f342819a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:41.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:41.674+0000 7f342d800700 1 --2- 172.21.15.90:0/2269864941 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34281083f0 0x7f342819a030 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3418006010 tx=0x7f3418005030 comp rx=0 tx=0).stop 2024-04-24T09:11:41.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:41.674+0000 7f342d800700 1 -- 172.21.15.90:0/2269864941 shutdown_connections 2024-04-24T09:11:41.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:41.674+0000 7f342d800700 1 --2- 172.21.15.90:0/2269864941 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f340c039650 0x7f340c03bb10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:41.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:41.674+0000 7f342d800700 1 --2- 172.21.15.90:0/2269864941 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34281083f0 0x7f342819a030 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:41.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:41.674+0000 7f342d800700 1 -- 172.21.15.90:0/2269864941 >> 172.21.15.90:0/2269864941 conn(0x7f3428076da0 msgr2=0x7f342810aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:41.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:41.675+0000 7f342d800700 1 -- 172.21.15.90:0/2269864941 shutdown_connections 2024-04-24T09:11:41.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:41.675+0000 7f342d800700 1 -- 172.21.15.90:0/2269864941 wait complete. 2024-04-24T09:11:42.019 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:41 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:11:42.019 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:41 smithi090 ceph-mon[27825]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:11:42.353 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-24T09:11:42.353 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-24T09:11:42.766 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:11:42.851 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:42 smithi090 ceph-mon[27825]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T09:11:44.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:43 smithi090 ceph-mon[27825]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:11:44.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:44.990+0000 7f1c61600700 1 -- 172.21.15.90:0/401136298 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c5c10dab0 msgr2=0x7f1c5c10de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:44.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:44.990+0000 7f1c61600700 1 --2- 172.21.15.90:0/401136298 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c5c10dab0 0x7f1c5c10de90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c00b0a0 tx=0x7f1c4c00b3b0 comp rx=0 tx=0).stop 2024-04-24T09:11:44.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:44.990+0000 7f1c61600700 1 -- 172.21.15.90:0/401136298 shutdown_connections 2024-04-24T09:11:44.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:44.990+0000 7f1c61600700 1 --2- 172.21.15.90:0/401136298 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c5c10dab0 0x7f1c5c10de90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:44.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:44.990+0000 7f1c61600700 1 -- 172.21.15.90:0/401136298 >> 172.21.15.90:0/401136298 conn(0x7f1c5c078380 msgr2=0x7f1c5c07a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:44.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:44.990+0000 7f1c61600700 1 -- 172.21.15.90:0/401136298 shutdown_connections 2024-04-24T09:11:44.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:44.991+0000 7f1c61600700 1 -- 172.21.15.90:0/401136298 wait complete. 2024-04-24T09:11:44.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:44.992+0000 7f1c61600700 1 Processor -- start 2024-04-24T09:11:44.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:44.992+0000 7f1c61600700 1 -- start start 2024-04-24T09:11:44.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:44.993+0000 7f1c61600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c5c10dab0 0x7f1c5c19e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:44.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:44.993+0000 7f1c61600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5c113170 con 0x7f1c5c10dab0 2024-04-24T09:11:44.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:44.993+0000 7f1c5aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c5c10dab0 0x7f1c5c19e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:44.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:44.993+0000 7f1c5aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c5c10dab0 0x7f1c5c19e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36800/0 (socket says 172.21.15.90:36800) 2024-04-24T09:11:44.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:44.993+0000 7f1c5aa00700 1 -- 172.21.15.90:0/2016073856 learned_addr learned my addr 172.21.15.90:0/2016073856 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:11:44.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:44.993+0000 7f1c5aa00700 1 -- 172.21.15.90:0/2016073856 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c5c19ea90 con 0x7f1c5c10dab0 2024-04-24T09:11:44.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:44.994+0000 7f1c5aa00700 1 --2- 172.21.15.90:0/2016073856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c5c10dab0 0x7f1c5c19e550 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c015040 tx=0x7f1c4c00bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:44.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:44.994+0000 7f1c53400700 1 -- 172.21.15.90:0/2016073856 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c4c017050 con 0x7f1c5c10dab0 2024-04-24T09:11:44.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:44.994+0000 7f1c53400700 1 -- 172.21.15.90:0/2016073856 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c4c023070 con 0x7f1c5c10dab0 2024-04-24T09:11:44.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:44.994+0000 7f1c61600700 1 -- 172.21.15.90:0/2016073856 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c5c19ed20 con 0x7f1c5c10dab0 2024-04-24T09:11:44.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:44.994+0000 7f1c53400700 1 -- 172.21.15.90:0/2016073856 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c4c004000 con 0x7f1c5c10dab0 2024-04-24T09:11:44.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:44.994+0000 7f1c61600700 1 -- 172.21.15.90:0/2016073856 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c5c1a2e20 con 0x7f1c5c10dab0 2024-04-24T09:11:44.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:44.995+0000 7f1c53400700 1 -- 172.21.15.90:0/2016073856 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f1c4c01e450 con 0x7f1c5c10dab0 2024-04-24T09:11:44.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:44.995+0000 7f1c61600700 1 -- 172.21.15.90:0/2016073856 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c5c10a200 con 0x7f1c5c10dab0 2024-04-24T09:11:44.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:44.996+0000 7f1c53400700 1 --2- 172.21.15.90:0/2016073856 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f1c400396f0 0x7f1c4003bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:44.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:44.996+0000 7f1c53400700 1 -- 172.21.15.90:0/2016073856 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1c4c00f050 con 0x7f1c5c10dab0 2024-04-24T09:11:44.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:44.996+0000 7f1c5a000700 1 --2- 172.21.15.90:0/2016073856 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f1c400396f0 0x7f1c4003bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:44.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:44.997+0000 7f1c5a000700 1 --2- 172.21.15.90:0/2016073856 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f1c400396f0 0x7f1c4003bbb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1c48006fd0 tx=0x7f1c48006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:45.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:45.001+0000 7f1c53400700 1 -- 172.21.15.90:0/2016073856 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1c4c0265c0 con 0x7f1c5c10dab0 2024-04-24T09:11:45.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:44 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:11:45.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:44 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:11:45.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:44 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:11:45.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:44 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:11:45.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:44 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:11:45.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:44 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:11:45.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:44 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:11:45.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:44 smithi090 ceph-mon[27825]: Updating smithi149:/etc/ceph/ceph.conf 2024-04-24T09:11:45.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:44 smithi090 ceph-mon[27825]: Updating smithi149:/var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T09:11:45.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:44 smithi090 ceph-mon[27825]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-04-24T09:11:45.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:45.254+0000 7f1c61600700 1 -- 172.21.15.90:0/2016073856 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f1c5c19f8b0 con 0x7f1c5c10dab0 2024-04-24T09:11:46.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:46.144+0000 7f1c53400700 1 -- 172.21.15.90:0/2016073856 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f1c4c026360 con 0x7f1c5c10dab0 2024-04-24T09:11:46.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:46.148+0000 7f1c61600700 1 -- 172.21.15.90:0/2016073856 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f1c400396f0 msgr2=0x7f1c4003bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:46.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:46.148+0000 7f1c61600700 1 --2- 172.21.15.90:0/2016073856 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f1c400396f0 0x7f1c4003bbb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1c48006fd0 tx=0x7f1c48006e40 comp rx=0 tx=0).stop 2024-04-24T09:11:46.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:46.148+0000 7f1c61600700 1 -- 172.21.15.90:0/2016073856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c5c10dab0 msgr2=0x7f1c5c19e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:46.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:46.148+0000 7f1c61600700 1 --2- 172.21.15.90:0/2016073856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c5c10dab0 0x7f1c5c19e550 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c015040 tx=0x7f1c4c00bdb0 comp rx=0 tx=0).stop 2024-04-24T09:11:46.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:46.148+0000 7f1c61600700 1 -- 172.21.15.90:0/2016073856 shutdown_connections 2024-04-24T09:11:46.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:46.148+0000 7f1c61600700 1 --2- 172.21.15.90:0/2016073856 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f1c400396f0 0x7f1c4003bbb0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:46.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:46.148+0000 7f1c61600700 1 --2- 172.21.15.90:0/2016073856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c5c10dab0 0x7f1c5c19e550 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:46.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:46.148+0000 7f1c61600700 1 -- 172.21.15.90:0/2016073856 >> 172.21.15.90:0/2016073856 conn(0x7f1c5c078380 msgr2=0x7f1c5c078e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:46.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:46.149+0000 7f1c61600700 1 -- 172.21.15.90:0/2016073856 shutdown_connections 2024-04-24T09:11:46.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:11:46.149+0000 7f1c61600700 1 -- 172.21.15.90:0/2016073856 wait complete. 2024-04-24T09:11:46.150 INFO:teuthology.orchestra.run.smithi090.stderr:adjusted tunables profile to default 2024-04-24T09:11:46.512 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:46 smithi090 ceph-mon[27825]: Updating smithi149:/var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T09:11:46.513 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:46 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:11:46.513 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:46 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:11:46.513 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:46 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:11:46.513 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:46 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/2016073856' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-24T09:11:46.513 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:46 smithi090 ceph-mon[27825]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:11:46.909 INFO:tasks.cephadm:Adding mon.a on smithi090 2024-04-24T09:11:46.909 INFO:tasks.cephadm:Adding mon.b on smithi114 2024-04-24T09:11:46.909 INFO:tasks.cephadm:Adding mon.c on smithi149 2024-04-24T09:11:46.909 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi090:172.21.15.90=a;smithi114:172.21.15.114=b;smithi149:172.21.15.149=c' 2024-04-24T09:11:47.153 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T09:11:47.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:47 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/2016073856' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-24T09:11:47.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:47 smithi090 ceph-mon[27825]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T09:11:48.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:48 smithi090 ceph-mon[27825]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:11:48.641 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:48.640+0000 7fbbb02c0700 1 -- 172.21.15.149:0/2196621902 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbba81085d0 msgr2=0x7fbba81089b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:48.641 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:48.640+0000 7fbbb02c0700 1 --2- 172.21.15.149:0/2196621902 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbba81085d0 0x7fbba81089b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbba4009910 tx=0x7fbba4009c20 comp rx=0 tx=0).stop 2024-04-24T09:11:48.642 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:48.640+0000 7fbbb02c0700 1 -- 172.21.15.149:0/2196621902 shutdown_connections 2024-04-24T09:11:48.642 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:48.640+0000 7fbbb02c0700 1 --2- 172.21.15.149:0/2196621902 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbba81085d0 0x7fbba81089b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:48.642 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:48.640+0000 7fbbb02c0700 1 -- 172.21.15.149:0/2196621902 >> 172.21.15.149:0/2196621902 conn(0x7fbba80fe030 msgr2=0x7fbba8100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:48.642 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:48.640+0000 7fbbb02c0700 1 -- 172.21.15.149:0/2196621902 shutdown_connections 2024-04-24T09:11:48.642 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:48.640+0000 7fbbb02c0700 1 -- 172.21.15.149:0/2196621902 wait complete. 2024-04-24T09:11:48.642 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:48.641+0000 7fbbb02c0700 1 Processor -- start 2024-04-24T09:11:48.642 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:48.641+0000 7fbbb02c0700 1 -- start start 2024-04-24T09:11:48.642 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:48.641+0000 7fbbb02c0700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbba81085d0 0x7fbba8197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:48.642 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:48.641+0000 7fbbb02c0700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fbba810cb90 con 0x7fbba81085d0 2024-04-24T09:11:48.644 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:48.642+0000 7fbbae05c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbba81085d0 0x7fbba8197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:48.644 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:48.642+0000 7fbbae05c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbba81085d0 0x7fbba8197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.149:42458/0 (socket says 172.21.15.149:42458) 2024-04-24T09:11:48.644 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:48.642+0000 7fbbae05c700 1 -- 172.21.15.149:0/908175546 learned_addr learned my addr 172.21.15.149:0/908175546 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-04-24T09:11:48.644 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:48.642+0000 7fbbae05c700 1 -- 172.21.15.149:0/908175546 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbba8198510 con 0x7fbba81085d0 2024-04-24T09:11:48.644 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:48.643+0000 7fbbae05c700 1 --2- 172.21.15.149:0/908175546 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbba81085d0 0x7fbba8197fd0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbba4005850 tx=0x7fbba4005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:48.644 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:48.643+0000 7fbb9f7fe700 1 -- 172.21.15.149:0/908175546 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbba4026070 con 0x7fbba81085d0 2024-04-24T09:11:48.644 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:48.643+0000 7fbbb02c0700 1 -- 172.21.15.149:0/908175546 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbba81987a0 con 0x7fbba81085d0 2024-04-24T09:11:48.644 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:48.643+0000 7fbbb02c0700 1 -- 172.21.15.149:0/908175546 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbba819c8a0 con 0x7fbba81085d0 2024-04-24T09:11:48.644 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:48.643+0000 7fbb9f7fe700 1 -- 172.21.15.149:0/908175546 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbba400b750 con 0x7fbba81085d0 2024-04-24T09:11:48.644 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:48.643+0000 7fbb9f7fe700 1 -- 172.21.15.149:0/908175546 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbba4030060 con 0x7fbba81085d0 2024-04-24T09:11:48.645 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:48.644+0000 7fbbb02c0700 1 -- 172.21.15.149:0/908175546 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb8c005330 con 0x7fbba81085d0 2024-04-24T09:11:48.645 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:48.644+0000 7fbb9f7fe700 1 -- 172.21.15.149:0/908175546 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fbba4029070 con 0x7fbba81085d0 2024-04-24T09:11:48.645 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:48.645+0000 7fbb9f7fe700 1 --2- 172.21.15.149:0/908175546 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fbb94039650 0x7fbb9403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:48.645 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:48.645+0000 7fbb9f7fe700 1 -- 172.21.15.149:0/908175546 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fbba401e070 con 0x7fbba81085d0 2024-04-24T09:11:48.646 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:48.645+0000 7fbbad85b700 1 --2- 172.21.15.149:0/908175546 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fbb94039650 0x7fbb9403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:48.646 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:48.646+0000 7fbbad85b700 1 --2- 172.21.15.149:0/908175546 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fbb94039650 0x7fbb9403bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbb98006fd0 tx=0x7fbb98006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:48.660 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:48.660+0000 7fbb9f7fe700 1 -- 172.21.15.149:0/908175546 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbba4013070 con 0x7fbba81085d0 2024-04-24T09:11:48.866 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:48.866+0000 7fbbb02c0700 1 -- 172.21.15.149:0/908175546 --> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi090:172.21.15.90=a;smithi114:172.21.15.114=b;smithi149:172.21.15.149=c", "target": ["mon-mgr", ""]}) -- 0x7fbb8c000c80 con 0x7fbb94039650 2024-04-24T09:11:48.881 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:48.880+0000 7fbb9f7fe700 1 -- 172.21.15.149:0/908175546 <== mgr.14152 v2:172.21.15.90:6800/4261662956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fbb8c000c80 con 0x7fbb94039650 2024-04-24T09:11:48.881 INFO:teuthology.orchestra.run.smithi149.stdout:Scheduled mon update... 2024-04-24T09:11:48.883 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:48.883+0000 7fbbb02c0700 1 -- 172.21.15.149:0/908175546 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fbb94039650 msgr2=0x7fbb9403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:48.883 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:48.883+0000 7fbbb02c0700 1 --2- 172.21.15.149:0/908175546 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fbb94039650 0x7fbb9403bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbb98006fd0 tx=0x7fbb98006e40 comp rx=0 tx=0).stop 2024-04-24T09:11:48.883 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:48.883+0000 7fbbb02c0700 1 -- 172.21.15.149:0/908175546 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbba81085d0 msgr2=0x7fbba8197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:48.884 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:48.883+0000 7fbbb02c0700 1 --2- 172.21.15.149:0/908175546 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbba81085d0 0x7fbba8197fd0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbba4005850 tx=0x7fbba4005030 comp rx=0 tx=0).stop 2024-04-24T09:11:48.884 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:48.883+0000 7fbbb02c0700 1 -- 172.21.15.149:0/908175546 shutdown_connections 2024-04-24T09:11:48.884 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:48.883+0000 7fbbb02c0700 1 --2- 172.21.15.149:0/908175546 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fbb94039650 0x7fbb9403bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:48.884 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:48.883+0000 7fbbb02c0700 1 --2- 172.21.15.149:0/908175546 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbba81085d0 0x7fbba8197fd0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:48.884 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:48.883+0000 7fbbb02c0700 1 -- 172.21.15.149:0/908175546 >> 172.21.15.149:0/908175546 conn(0x7fbba80fe030 msgr2=0x7fbba80ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:48.884 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:48.883+0000 7fbbb02c0700 1 -- 172.21.15.149:0/908175546 shutdown_connections 2024-04-24T09:11:48.884 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:48.883+0000 7fbbb02c0700 1 -- 172.21.15.149:0/908175546 wait complete. 2024-04-24T09:11:49.521 DEBUG:teuthology.orchestra.run.smithi114:mon.b> sudo journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mon.b.service 2024-04-24T09:11:49.523 DEBUG:teuthology.orchestra.run.smithi149:mon.c> sudo journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mon.c.service 2024-04-24T09:11:49.525 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-24T09:11:49.525 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T09:11:50.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:49 smithi090 ceph-mon[27825]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi090:172.21.15.90=a;smithi114:172.21.15.114=b;smithi149:172.21.15.149=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:11:50.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:49 smithi090 ceph-mon[27825]: Saving service mon spec with placement smithi090:172.21.15.90=a;smithi114:172.21.15.114=b;smithi149:172.21.15.149=c;count:3 2024-04-24T09:11:50.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:49 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:11:50.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:49 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:11:50.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:49 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:11:50.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:49 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:11:50.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:49 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:11:50.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:49 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T09:11:50.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:49 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:11:50.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:49 smithi090 ceph-mon[27825]: Deploying daemon mon.c on smithi149 2024-04-24T09:11:50.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:49 smithi090 ceph-mon[27825]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:11:51.584 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T09:11:52.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:52 smithi090 ceph-mon[27825]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:11:53.346 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:53.345+0000 7f9b4673b700 1 -- 172.21.15.149:0/3092959404 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b401075d0 msgr2=0x7f9b401079b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:53.347 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:53.345+0000 7f9b4673b700 1 --2- 172.21.15.149:0/3092959404 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b401075d0 0x7f9b401079b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9b30009910 tx=0x7f9b30009c20 comp rx=0 tx=0).stop 2024-04-24T09:11:53.347 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:53.345+0000 7f9b4673b700 1 -- 172.21.15.149:0/3092959404 shutdown_connections 2024-04-24T09:11:53.347 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:53.345+0000 7f9b4673b700 1 --2- 172.21.15.149:0/3092959404 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b401075d0 0x7f9b401079b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:53.347 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:53.345+0000 7f9b4673b700 1 -- 172.21.15.149:0/3092959404 >> 172.21.15.149:0/3092959404 conn(0x7f9b400fd030 msgr2=0x7f9b400ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:53.347 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:53.345+0000 7f9b4673b700 1 -- 172.21.15.149:0/3092959404 shutdown_connections 2024-04-24T09:11:53.347 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:53.345+0000 7f9b4673b700 1 -- 172.21.15.149:0/3092959404 wait complete. 2024-04-24T09:11:53.347 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:53.346+0000 7f9b4673b700 1 Processor -- start 2024-04-24T09:11:53.347 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:53.346+0000 7f9b4673b700 1 -- start start 2024-04-24T09:11:53.347 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:53.346+0000 7f9b4673b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b401075d0 0x7f9b40197030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:53.347 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:53.346+0000 7f9b4673b700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4010bb90 con 0x7f9b401075d0 2024-04-24T09:11:53.348 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:53.347+0000 7f9b3ffff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b401075d0 0x7f9b40197030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:53.349 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:53.347+0000 7f9b3ffff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b401075d0 0x7f9b40197030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.149:42484/0 (socket says 172.21.15.149:42484) 2024-04-24T09:11:53.349 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:53.347+0000 7f9b3ffff700 1 -- 172.21.15.149:0/1565004435 learned_addr learned my addr 172.21.15.149:0/1565004435 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-04-24T09:11:53.349 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:53.347+0000 7f9b3ffff700 1 -- 172.21.15.149:0/1565004435 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b40197570 con 0x7f9b401075d0 2024-04-24T09:11:53.349 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:53.348+0000 7f9b3ffff700 1 --2- 172.21.15.149:0/1565004435 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b401075d0 0x7f9b40197030 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9b30005850 tx=0x7f9b30005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:53.349 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:53.348+0000 7f9b3d7fa700 1 -- 172.21.15.149:0/1565004435 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b30026070 con 0x7f9b401075d0 2024-04-24T09:11:53.349 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:53.348+0000 7f9b4673b700 1 -- 172.21.15.149:0/1565004435 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b40197800 con 0x7f9b401075d0 2024-04-24T09:11:53.349 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:53.348+0000 7f9b3d7fa700 1 -- 172.21.15.149:0/1565004435 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b3000bc30 con 0x7f9b401075d0 2024-04-24T09:11:53.349 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:53.348+0000 7f9b3d7fa700 1 -- 172.21.15.149:0/1565004435 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b30003a20 con 0x7f9b401075d0 2024-04-24T09:11:53.349 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:53.348+0000 7f9b4673b700 1 -- 172.21.15.149:0/1565004435 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b40197c60 con 0x7f9b401075d0 2024-04-24T09:11:53.349 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:53.349+0000 7f9b4673b700 1 -- 172.21.15.149:0/1565004435 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b40102c40 con 0x7f9b401075d0 2024-04-24T09:11:53.350 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:53.350+0000 7f9b3d7fa700 1 -- 172.21.15.149:0/1565004435 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f9b30029070 con 0x7f9b401075d0 2024-04-24T09:11:53.350 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:53.350+0000 7f9b3d7fa700 1 --2- 172.21.15.149:0/1565004435 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f9b28039650 0x7f9b2803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:53.350 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:53.350+0000 7f9b3d7fa700 1 -- 172.21.15.149:0/1565004435 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9b3001e070 con 0x7f9b401075d0 2024-04-24T09:11:53.350 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:53.350+0000 7f9b3f7fe700 1 --2- 172.21.15.149:0/1565004435 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f9b28039650 0x7f9b2803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:53.351 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:53.351+0000 7f9b3f7fe700 1 --2- 172.21.15.149:0/1565004435 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f9b28039650 0x7f9b2803bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9b34006fd0 tx=0x7f9b34006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:53.365 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:53.365+0000 7f9b3d7fa700 1 -- 172.21.15.149:0/1565004435 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9b30013070 con 0x7f9b401075d0 2024-04-24T09:11:53.663 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:53.663+0000 7f9b4673b700 1 -- 172.21.15.149:0/1565004435 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f9b40198390 con 0x7f9b401075d0 2024-04-24T09:11:53.665 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:53.664+0000 7f9b3d7fa700 1 -- 172.21.15.149:0/1565004435 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f9b30030030 con 0x7f9b401075d0 2024-04-24T09:11:53.665 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T09:11:53.665 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":1,"fsid":"30ee9be2-021a-11ef-bc93-c7b262605968","modified":"2024-04-24T09:09:44.652569Z","created":"2024-04-24T09:09:44.652569Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T09:11:53.667 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:53.667+0000 7f9b4673b700 1 -- 172.21.15.149:0/1565004435 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f9b28039650 msgr2=0x7f9b2803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:53.667 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:53.667+0000 7f9b4673b700 1 --2- 172.21.15.149:0/1565004435 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f9b28039650 0x7f9b2803bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9b34006fd0 tx=0x7f9b34006e40 comp rx=0 tx=0).stop 2024-04-24T09:11:53.667 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:53.667+0000 7f9b4673b700 1 -- 172.21.15.149:0/1565004435 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b401075d0 msgr2=0x7f9b40197030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:53.667 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:53.667+0000 7f9b4673b700 1 --2- 172.21.15.149:0/1565004435 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b401075d0 0x7f9b40197030 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9b30005850 tx=0x7f9b30005030 comp rx=0 tx=0).stop 2024-04-24T09:11:53.667 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:53.667+0000 7f9b4673b700 1 -- 172.21.15.149:0/1565004435 shutdown_connections 2024-04-24T09:11:53.667 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:53.667+0000 7f9b4673b700 1 --2- 172.21.15.149:0/1565004435 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f9b28039650 0x7f9b2803bb10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:53.668 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:53.667+0000 7f9b4673b700 1 --2- 172.21.15.149:0/1565004435 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b401075d0 0x7f9b40197030 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:53.668 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:53.667+0000 7f9b4673b700 1 -- 172.21.15.149:0/1565004435 >> 172.21.15.149:0/1565004435 conn(0x7f9b400fd030 msgr2=0x7f9b400fd410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:53.668 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:53.667+0000 7f9b4673b700 1 -- 172.21.15.149:0/1565004435 shutdown_connections 2024-04-24T09:11:53.668 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:11:53.667+0000 7f9b4673b700 1 -- 172.21.15.149:0/1565004435 wait complete. 2024-04-24T09:11:53.668 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 1 2024-04-24T09:11:54.275 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:11:54 smithi149 systemd[1]: Started Ceph mon.c for 30ee9be2-021a-11ef-bc93-c7b262605968. 2024-04-24T09:11:55.672 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-24T09:11:55.673 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T09:11:55.916 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T09:11:59.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:11:59 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T09:11:59.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:11:59 smithi149 ceph-mon[32592]: mon.a calling monitor election 2024-04-24T09:11:59.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:11:59 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T09:11:59.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:11:59 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T09:11:59.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:11:59 smithi149 ceph-mon[32592]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:11:59.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:11:59 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T09:11:59.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:11:59 smithi149 ceph-mon[32592]: mon.c calling monitor election 2024-04-24T09:11:59.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:11:59 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T09:11:59.363 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:11:59 smithi149 ceph-mon[32592]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:11:59.363 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:11:59 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T09:11:59.363 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:11:59 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T09:11:59.363 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:11:59 smithi149 ceph-mon[32592]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-24T09:11:59.363 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:11:59 smithi149 ceph-mon[32592]: monmap epoch 2 2024-04-24T09:11:59.363 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:11:59 smithi149 ceph-mon[32592]: fsid 30ee9be2-021a-11ef-bc93-c7b262605968 2024-04-24T09:11:59.363 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:11:59 smithi149 ceph-mon[32592]: last_changed 2024-04-24T09:11:54.053393+0000 2024-04-24T09:11:59.363 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:11:59 smithi149 ceph-mon[32592]: created 2024-04-24T09:09:44.652569+0000 2024-04-24T09:11:59.363 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:11:59 smithi149 ceph-mon[32592]: min_mon_release 19 (squid) 2024-04-24T09:11:59.363 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:11:59 smithi149 ceph-mon[32592]: election_strategy: 1 2024-04-24T09:11:59.363 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:11:59 smithi149 ceph-mon[32592]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-04-24T09:11:59.363 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:11:59 smithi149 ceph-mon[32592]: 1: [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon.c 2024-04-24T09:11:59.363 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:11:59 smithi149 ceph-mon[32592]: fsmap 2024-04-24T09:11:59.363 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:11:59 smithi149 ceph-mon[32592]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T09:11:59.363 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:11:59 smithi149 ceph-mon[32592]: mgrmap e13: x(active, since 61s) 2024-04-24T09:11:59.364 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:11:59 smithi149 ceph-mon[32592]: overall HEALTH_OK 2024-04-24T09:11:59.364 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:11:59 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:11:59.364 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:11:59 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:11:59.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:59 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T09:11:59.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:59 smithi090 ceph-mon[27825]: mon.a calling monitor election 2024-04-24T09:11:59.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:59 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T09:11:59.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:59 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T09:11:59.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:59 smithi090 ceph-mon[27825]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:11:59.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:59 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T09:11:59.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:59 smithi090 ceph-mon[27825]: mon.c calling monitor election 2024-04-24T09:11:59.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:59 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T09:11:59.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:59 smithi090 ceph-mon[27825]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:11:59.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:59 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T09:11:59.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:59 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T09:11:59.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:59 smithi090 ceph-mon[27825]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-24T09:11:59.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:59 smithi090 ceph-mon[27825]: monmap epoch 2 2024-04-24T09:11:59.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:59 smithi090 ceph-mon[27825]: fsid 30ee9be2-021a-11ef-bc93-c7b262605968 2024-04-24T09:11:59.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:59 smithi090 ceph-mon[27825]: last_changed 2024-04-24T09:11:54.053393+0000 2024-04-24T09:11:59.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:59 smithi090 ceph-mon[27825]: created 2024-04-24T09:09:44.652569+0000 2024-04-24T09:11:59.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:59 smithi090 ceph-mon[27825]: min_mon_release 19 (squid) 2024-04-24T09:11:59.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:59 smithi090 ceph-mon[27825]: election_strategy: 1 2024-04-24T09:11:59.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:59 smithi090 ceph-mon[27825]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-04-24T09:11:59.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:59 smithi090 ceph-mon[27825]: 1: [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon.c 2024-04-24T09:11:59.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:59 smithi090 ceph-mon[27825]: fsmap 2024-04-24T09:11:59.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:59 smithi090 ceph-mon[27825]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T09:11:59.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:59 smithi090 ceph-mon[27825]: mgrmap e13: x(active, since 61s) 2024-04-24T09:11:59.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:59 smithi090 ceph-mon[27825]: overall HEALTH_OK 2024-04-24T09:11:59.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:59 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:11:59.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:11:59 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:00.385 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.383+0000 7f48eb68e700 1 -- 172.21.15.149:0/125257322 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48d00035c0 msgr2=0x7f48d0005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:00.385 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.383+0000 7f48eb68e700 1 --2- 172.21.15.149:0/125257322 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48d00035c0 0x7f48d0005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f48e4101410 tx=0x7f48d400feb0 comp rx=0 tx=0).stop 2024-04-24T09:12:00.385 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.383+0000 7f48eb68e700 1 -- 172.21.15.149:0/125257322 shutdown_connections 2024-04-24T09:12:00.385 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.383+0000 7f48eb68e700 1 --2- 172.21.15.149:0/125257322 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48d00035c0 0x7f48d0005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:00.385 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.383+0000 7f48eb68e700 1 -- 172.21.15.149:0/125257322 >> 172.21.15.149:0/125257322 conn(0x7f48e40fcfd0 msgr2=0x7f48e40ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:00.385 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.384+0000 7f48eb68e700 1 -- 172.21.15.149:0/125257322 shutdown_connections 2024-04-24T09:12:00.385 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.384+0000 7f48eb68e700 1 -- 172.21.15.149:0/125257322 wait complete. 2024-04-24T09:12:00.385 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.384+0000 7f48eb68e700 1 Processor -- start 2024-04-24T09:12:00.385 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.385+0000 7f48eb68e700 1 -- start start 2024-04-24T09:12:00.387 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.385+0000 7f48eb68e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48d00035c0 0x7f48e419a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:00.387 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.385+0000 7f48eb68e700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48e419a910 0x7f48e419ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:00.387 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.385+0000 7f48eb68e700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f48e4109580 con 0x7f48d00035c0 2024-04-24T09:12:00.387 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.385+0000 7f48eb68e700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f48e4109400 con 0x7f48e419a910 2024-04-24T09:12:00.387 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.385+0000 7f48e8c29700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48e419a910 0x7f48e419ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:00.387 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.385+0000 7f48e8c29700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48e419a910 0x7f48e419ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:59316/0 (socket says 172.21.15.149:59316) 2024-04-24T09:12:00.387 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.385+0000 7f48e8c29700 1 -- 172.21.15.149:0/744423326 learned_addr learned my addr 172.21.15.149:0/744423326 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-04-24T09:12:00.387 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.385+0000 7f48e8c29700 1 -- 172.21.15.149:0/744423326 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48e419a910 msgr2=0x7f48e419ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-24T09:12:00.387 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.385+0000 7f48e8c29700 1 -- 172.21.15.149:0/744423326 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48e419a910 msgr2=0x7f48e419ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-24T09:12:00.387 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.385+0000 7f48e8c29700 1 --2- 172.21.15.149:0/744423326 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48e419a910 0x7f48e419ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T09:12:00.387 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.385+0000 7f48e8c29700 1 --2- 172.21.15.149:0/744423326 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48e419a910 0x7f48e419ec50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T09:12:00.387 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.385+0000 7f48e942a700 1 --2- 172.21.15.149:0/744423326 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48d00035c0 0x7f48e419a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:00.387 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.386+0000 7f48e942a700 1 -- 172.21.15.149:0/744423326 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48e419a910 msgr2=0x7f48e419ec50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T09:12:00.388 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.386+0000 7f48e942a700 1 --2- 172.21.15.149:0/744423326 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48e419a910 0x7f48e419ec50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:00.388 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.386+0000 7f48e942a700 1 -- 172.21.15.149:0/744423326 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48e419af50 con 0x7f48d00035c0 2024-04-24T09:12:00.388 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.386+0000 7f48e942a700 1 --2- 172.21.15.149:0/744423326 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48d00035c0 0x7f48e419a3d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f48d40059d0 tx=0x7f48d40047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:00.388 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.387+0000 7f48da7fc700 1 -- 172.21.15.149:0/744423326 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f48d4013b00 con 0x7f48d00035c0 2024-04-24T09:12:00.388 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.387+0000 7f48eb68e700 1 -- 172.21.15.149:0/744423326 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48e419f190 con 0x7f48d00035c0 2024-04-24T09:12:00.388 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.387+0000 7f48da7fc700 1 -- 172.21.15.149:0/744423326 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48d4004500 con 0x7f48d00035c0 2024-04-24T09:12:00.388 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.387+0000 7f48da7fc700 1 -- 172.21.15.149:0/744423326 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f48d4003e30 con 0x7f48d00035c0 2024-04-24T09:12:00.389 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.387+0000 7f48eb68e700 1 -- 172.21.15.149:0/744423326 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48e419f770 con 0x7f48d00035c0 2024-04-24T09:12:00.389 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.388+0000 7f48eb68e700 1 -- 172.21.15.149:0/744423326 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48e4102c20 con 0x7f48d00035c0 2024-04-24T09:12:00.389 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.389+0000 7f48da7fc700 1 -- 172.21.15.149:0/744423326 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f48d4023070 con 0x7f48d00035c0 2024-04-24T09:12:00.390 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.389+0000 7f48da7fc700 1 --2- 172.21.15.149:0/744423326 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f48c8039460 0x7f48c803b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:00.390 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.389+0000 7f48da7fc700 1 -- 172.21.15.149:0/744423326 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f48d4020050 con 0x7f48d00035c0 2024-04-24T09:12:00.390 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.389+0000 7f48e8c29700 1 --2- 172.21.15.149:0/744423326 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f48c8039460 0x7f48c803b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:00.391 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.390+0000 7f48e8c29700 1 --2- 172.21.15.149:0/744423326 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f48c8039460 0x7f48c803b920 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f48e0009180 tx=0x7f48e0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:00.403 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.403+0000 7f48da7fc700 1 -- 172.21.15.149:0/744423326 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f48d4031020 con 0x7f48d00035c0 2024-04-24T09:12:00.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:00 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:00.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:00 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T09:12:00.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:00 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:00.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:00 smithi090 ceph-mon[27825]: Deploying daemon mon.b on smithi114 2024-04-24T09:12:00.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:00 smithi090 ceph-mon[27825]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:00.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:00 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T09:12:00.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:00 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:00.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:00 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T09:12:00.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:00 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:00.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:00 smithi149 ceph-mon[32592]: Deploying daemon mon.b on smithi114 2024-04-24T09:12:00.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:00 smithi149 ceph-mon[32592]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:00.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:00 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T09:12:00.687 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.687+0000 7f48eb68e700 1 -- 172.21.15.149:0/744423326 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f48e419fa50 con 0x7f48d00035c0 2024-04-24T09:12:00.688 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.688+0000 7f48da7fc700 1 -- 172.21.15.149:0/744423326 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f48d4028030 con 0x7f48d00035c0 2024-04-24T09:12:00.688 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T09:12:00.688 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":2,"fsid":"30ee9be2-021a-11ef-bc93-c7b262605968","modified":"2024-04-24T09:11:54.053393Z","created":"2024-04-24T09:09:44.652569Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:3300","nonce":0},{"type":"v1","addr":"172.21.15.149:6789","nonce":0}]},"addr":"172.21.15.149:6789/0","public_addr":"172.21.15.149:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-24T09:12:00.690 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.690+0000 7f48eb68e700 1 -- 172.21.15.149:0/744423326 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f48c8039460 msgr2=0x7f48c803b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:00.690 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.690+0000 7f48eb68e700 1 --2- 172.21.15.149:0/744423326 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f48c8039460 0x7f48c803b920 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f48e0009180 tx=0x7f48e0008040 comp rx=0 tx=0).stop 2024-04-24T09:12:00.691 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.690+0000 7f48eb68e700 1 -- 172.21.15.149:0/744423326 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48d00035c0 msgr2=0x7f48e419a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:00.691 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.690+0000 7f48eb68e700 1 --2- 172.21.15.149:0/744423326 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48d00035c0 0x7f48e419a3d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f48d40059d0 tx=0x7f48d40047a0 comp rx=0 tx=0).stop 2024-04-24T09:12:00.691 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.690+0000 7f48eb68e700 1 -- 172.21.15.149:0/744423326 shutdown_connections 2024-04-24T09:12:00.691 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.690+0000 7f48eb68e700 1 --2- 172.21.15.149:0/744423326 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f48c8039460 0x7f48c803b920 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:00.691 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.690+0000 7f48eb68e700 1 --2- 172.21.15.149:0/744423326 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48d00035c0 0x7f48e419a3d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:00.691 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.690+0000 7f48eb68e700 1 --2- 172.21.15.149:0/744423326 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48e419a910 0x7f48e419ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:00.691 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.690+0000 7f48eb68e700 1 -- 172.21.15.149:0/744423326 >> 172.21.15.149:0/744423326 conn(0x7f48e40fcfd0 msgr2=0x7f48e406a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:00.691 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.690+0000 7f48eb68e700 1 -- 172.21.15.149:0/744423326 shutdown_connections 2024-04-24T09:12:00.691 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:00.690+0000 7f48eb68e700 1 -- 172.21.15.149:0/744423326 wait complete. 2024-04-24T09:12:00.691 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 2 2024-04-24T09:12:01.267 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:01 smithi149 ceph-mon[32592]: from='client.? 172.21.15.149:0/744423326' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T09:12:01.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:01 smithi090 ceph-mon[27825]: from='client.? 172.21.15.149:0/744423326' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T09:12:02.297 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-24T09:12:02.297 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T09:12:02.543 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T09:12:02.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:02 smithi090 ceph-mon[27825]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:02.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:02 smithi149 ceph-mon[32592]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:03.070 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: DB Session ID: 7SNO4A95W038MX2U5JBW 2024-04-24T09:12:03.070 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: CURRENT file: CURRENT 2024-04-24T09:12:03.070 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: IDENTITY file: IDENTITY 2024-04-24T09:12:03.070 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-24T09:12:03.070 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-04-24T09:12:03.070 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-04-24T09:12:03.070 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.error_if_exists: 0 2024-04-24T09:12:03.070 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.create_if_missing: 0 2024-04-24T09:12:03.070 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.paranoid_checks: 1 2024-04-24T09:12:03.070 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-24T09:12:03.070 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-24T09:12:03.071 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-24T09:12:03.071 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.env: 0x55bce2b02cc0 2024-04-24T09:12:03.071 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.fs: PosixFileSystem 2024-04-24T09:12:03.071 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.info_log: 0x55bce3bc3c20 2024-04-24T09:12:03.071 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.max_file_opening_threads: 16 2024-04-24T09:12:03.071 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.statistics: (nil) 2024-04-24T09:12:03.071 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.use_fsync: 0 2024-04-24T09:12:03.071 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.max_log_file_size: 0 2024-04-24T09:12:03.071 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-24T09:12:03.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-24T09:12:03.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.keep_log_file_num: 1000 2024-04-24T09:12:03.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.recycle_log_file_num: 0 2024-04-24T09:12:03.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.allow_fallocate: 1 2024-04-24T09:12:03.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.allow_mmap_reads: 0 2024-04-24T09:12:03.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.allow_mmap_writes: 0 2024-04-24T09:12:03.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.use_direct_reads: 0 2024-04-24T09:12:03.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-24T09:12:03.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.create_missing_column_families: 0 2024-04-24T09:12:03.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.db_log_dir: 2024-04-24T09:12:03.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.wal_dir: 2024-04-24T09:12:03.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-24T09:12:03.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-24T09:12:03.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-24T09:12:03.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-24T09:12:03.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-24T09:12:03.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-24T09:12:03.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.advise_random_on_open: 1 2024-04-24T09:12:03.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.db_write_buffer_size: 0 2024-04-24T09:12:03.428 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.write_buffer_manager: 0x55bce3b921e0 2024-04-24T09:12:03.428 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-24T09:12:03.428 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-24T09:12:03.428 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-24T09:12:03.428 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.rate_limiter: (nil) 2024-04-24T09:12:03.428 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-24T09:12:03.428 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.wal_recovery_mode: 2 2024-04-24T09:12:03.428 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.enable_thread_tracking: 0 2024-04-24T09:12:03.428 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.enable_pipelined_write: 0 2024-04-24T09:12:03.428 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.unordered_write: 0 2024-04-24T09:12:03.428 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-24T09:12:03.428 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-24T09:12:03.428 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-24T09:12:03.428 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-24T09:12:03.428 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.row_cache: None 2024-04-24T09:12:03.429 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.wal_filter: None 2024-04-24T09:12:03.429 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-24T09:12:03.429 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.allow_ingest_behind: 0 2024-04-24T09:12:03.429 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.two_write_queues: 0 2024-04-24T09:12:03.429 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.manual_wal_flush: 0 2024-04-24T09:12:03.431 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.wal_compression: 0 2024-04-24T09:12:03.431 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.atomic_flush: 0 2024-04-24T09:12:03.431 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-24T09:12:03.431 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-24T09:12:03.431 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-24T09:12:03.431 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.log_readahead_size: 0 2024-04-24T09:12:03.431 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-24T09:12:03.431 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.best_efforts_recovery: 0 2024-04-24T09:12:03.431 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-24T09:12:03.431 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-24T09:12:03.431 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.allow_data_in_errors: 0 2024-04-24T09:12:03.431 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.db_host_id: __hostname__ 2024-04-24T09:12:03.432 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-24T09:12:03.432 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.max_background_jobs: 2 2024-04-24T09:12:03.432 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.max_background_compactions: -1 2024-04-24T09:12:03.432 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.max_subcompactions: 1 2024-04-24T09:12:03.432 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-24T09:12:03.432 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-24T09:12:03.432 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-24T09:12:03.432 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.max_total_wal_size: 0 2024-04-24T09:12:03.432 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-24T09:12:03.432 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-24T09:12:03.432 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-24T09:12:03.433 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-24T09:12:03.433 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.max_open_files: -1 2024-04-24T09:12:03.433 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.bytes_per_sync: 0 2024-04-24T09:12:03.433 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-24T09:12:03.433 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-24T09:12:03.433 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.compaction_readahead_size: 0 2024-04-24T09:12:03.433 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.max_background_flushes: -1 2024-04-24T09:12:03.433 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Compression algorithms supported: 2024-04-24T09:12:03.433 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-24T09:12:03.433 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: kZSTD supported: 0 2024-04-24T09:12:03.433 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: kXpressCompression supported: 0 2024-04-24T09:12:03.433 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: kLZ4HCCompression supported: 1 2024-04-24T09:12:03.433 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: kZlibCompression supported: 1 2024-04-24T09:12:03.434 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: kSnappyCompression supported: 1 2024-04-24T09:12:03.434 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: kLZ4Compression supported: 1 2024-04-24T09:12:03.434 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: kBZip2Compression supported: 0 2024-04-24T09:12:03.434 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-24T09:12:03.434 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-24T09:12:03.434 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-04-24T09:12:03.434 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-24T09:12:03.434 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-24T09:12:03.434 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.merge_operator: 2024-04-24T09:12:03.434 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.compaction_filter: None 2024-04-24T09:12:03.434 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.compaction_filter_factory: None 2024-04-24T09:12:03.434 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.sst_partitioner_factory: None 2024-04-24T09:12:03.434 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-24T09:12:03.435 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-24T09:12:03.435 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55bce3bc3de0) 2024-04-24T09:12:03.435 INFO:journalctl@ceph.mon.b.smithi114.stdout: cache_index_and_filter_blocks: 1 2024-04-24T09:12:03.435 INFO:journalctl@ceph.mon.b.smithi114.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-24T09:12:03.435 INFO:journalctl@ceph.mon.b.smithi114.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-24T09:12:03.435 INFO:journalctl@ceph.mon.b.smithi114.stdout: pin_top_level_index_and_filter: 1 2024-04-24T09:12:03.435 INFO:journalctl@ceph.mon.b.smithi114.stdout: index_type: 0 2024-04-24T09:12:03.435 INFO:journalctl@ceph.mon.b.smithi114.stdout: data_block_index_type: 0 2024-04-24T09:12:03.435 INFO:journalctl@ceph.mon.b.smithi114.stdout: index_shortening: 1 2024-04-24T09:12:03.435 INFO:journalctl@ceph.mon.b.smithi114.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-24T09:12:03.435 INFO:journalctl@ceph.mon.b.smithi114.stdout: checksum: 4 2024-04-24T09:12:03.435 INFO:journalctl@ceph.mon.b.smithi114.stdout: no_block_cache: 0 2024-04-24T09:12:03.435 INFO:journalctl@ceph.mon.b.smithi114.stdout: block_cache: 0x55bce3bc5090 2024-04-24T09:12:03.435 INFO:journalctl@ceph.mon.b.smithi114.stdout: block_cache_name: BinnedLRUCache 2024-04-24T09:12:03.435 INFO:journalctl@ceph.mon.b.smithi114.stdout: block_cache_options: 2024-04-24T09:12:03.436 INFO:journalctl@ceph.mon.b.smithi114.stdout: capacity : 536870912 2024-04-24T09:12:03.436 INFO:journalctl@ceph.mon.b.smithi114.stdout: num_shard_bits : 4 2024-04-24T09:12:03.436 INFO:journalctl@ceph.mon.b.smithi114.stdout: strict_capacity_limit : 0 2024-04-24T09:12:03.436 INFO:journalctl@ceph.mon.b.smithi114.stdout: high_pri_pool_ratio: 0.000 2024-04-24T09:12:03.436 INFO:journalctl@ceph.mon.b.smithi114.stdout: block_cache_compressed: (nil) 2024-04-24T09:12:03.436 INFO:journalctl@ceph.mon.b.smithi114.stdout: persistent_cache: (nil) 2024-04-24T09:12:03.436 INFO:journalctl@ceph.mon.b.smithi114.stdout: block_size: 4096 2024-04-24T09:12:03.436 INFO:journalctl@ceph.mon.b.smithi114.stdout: block_size_deviation: 10 2024-04-24T09:12:03.436 INFO:journalctl@ceph.mon.b.smithi114.stdout: block_restart_interval: 16 2024-04-24T09:12:03.436 INFO:journalctl@ceph.mon.b.smithi114.stdout: index_block_restart_interval: 1 2024-04-24T09:12:03.436 INFO:journalctl@ceph.mon.b.smithi114.stdout: metadata_block_size: 4096 2024-04-24T09:12:03.436 INFO:journalctl@ceph.mon.b.smithi114.stdout: partition_filters: 0 2024-04-24T09:12:03.436 INFO:journalctl@ceph.mon.b.smithi114.stdout: use_delta_encoding: 1 2024-04-24T09:12:03.436 INFO:journalctl@ceph.mon.b.smithi114.stdout: filter_policy: bloomfilter 2024-04-24T09:12:03.436 INFO:journalctl@ceph.mon.b.smithi114.stdout: whole_key_filtering: 1 2024-04-24T09:12:03.437 INFO:journalctl@ceph.mon.b.smithi114.stdout: verify_compression: 0 2024-04-24T09:12:03.437 INFO:journalctl@ceph.mon.b.smithi114.stdout: read_amp_bytes_per_bit: 0 2024-04-24T09:12:03.437 INFO:journalctl@ceph.mon.b.smithi114.stdout: format_version: 5 2024-04-24T09:12:03.437 INFO:journalctl@ceph.mon.b.smithi114.stdout: enable_index_compression: 1 2024-04-24T09:12:03.437 INFO:journalctl@ceph.mon.b.smithi114.stdout: block_align: 0 2024-04-24T09:12:03.437 INFO:journalctl@ceph.mon.b.smithi114.stdout: max_auto_readahead_size: 262144 2024-04-24T09:12:03.437 INFO:journalctl@ceph.mon.b.smithi114.stdout: prepopulate_block_cache: 0 2024-04-24T09:12:03.437 INFO:journalctl@ceph.mon.b.smithi114.stdout: initial_auto_readahead_size: 8192 2024-04-24T09:12:03.437 INFO:journalctl@ceph.mon.b.smithi114.stdout: num_file_reads_for_auto_readahead: 2 2024-04-24T09:12:03.438 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.write_buffer_size: 33554432 2024-04-24T09:12:03.438 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.max_write_buffer_number: 2 2024-04-24T09:12:03.438 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.compression: NoCompression 2024-04-24T09:12:03.438 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.bottommost_compression: Disabled 2024-04-24T09:12:03.438 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.prefix_extractor: nullptr 2024-04-24T09:12:03.439 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-24T09:12:03.439 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.num_levels: 7 2024-04-24T09:12:03.439 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-24T09:12:03.439 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-24T09:12:03.439 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-24T09:12:03.439 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-24T09:12:03.439 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-24T09:12:03.439 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-24T09:12:03.439 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-24T09:12:03.439 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-24T09:12:03.439 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-24T09:12:03.440 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-24T09:12:03.440 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-24T09:12:03.440 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-24T09:12:03.440 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-24T09:12:03.440 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.compression_opts.level: 32767 2024-04-24T09:12:03.440 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.compression_opts.strategy: 0 2024-04-24T09:12:03.440 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-24T09:12:03.440 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-24T09:12:03.440 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-24T09:12:03.440 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-24T09:12:03.440 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.compression_opts.enabled: false 2024-04-24T09:12:03.440 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-24T09:12:03.440 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-24T09:12:03.440 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-24T09:12:03.441 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-24T09:12:03.441 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.target_file_size_base: 67108864 2024-04-24T09:12:03.441 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-24T09:12:03.441 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-24T09:12:03.441 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-24T09:12:03.441 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-24T09:12:03.441 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-24T09:12:03.441 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-24T09:12:03.442 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-24T09:12:03.442 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-24T09:12:03.442 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-24T09:12:03.442 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-24T09:12:03.442 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-24T09:12:03.443 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-24T09:12:03.443 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-24T09:12:03.443 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-24T09:12:03.443 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.arena_block_size: 1048576 2024-04-24T09:12:03.443 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-24T09:12:03.443 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-24T09:12:03.443 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.disable_auto_compactions: 0 2024-04-24T09:12:03.443 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-24T09:12:03.443 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-24T09:12:03.443 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-24T09:12:03.443 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-24T09:12:03.443 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-24T09:12:03.443 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-24T09:12:03.444 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-24T09:12:03.444 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-24T09:12:03.444 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-24T09:12:03.444 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-24T09:12:03.444 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-24T09:12:03.444 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.inplace_update_support: 0 2024-04-24T09:12:03.444 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-24T09:12:03.444 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-24T09:12:03.444 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-24T09:12:03.444 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-24T09:12:03.444 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.bloom_locality: 0 2024-04-24T09:12:03.444 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.max_successive_merges: 0 2024-04-24T09:12:03.444 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-24T09:12:03.444 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.paranoid_file_checks: 0 2024-04-24T09:12:03.445 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.force_consistency_checks: 1 2024-04-24T09:12:03.445 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.report_bg_io_stats: 0 2024-04-24T09:12:03.445 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.ttl: 2592000 2024-04-24T09:12:03.445 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-24T09:12:03.445 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-24T09:12:03.445 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-24T09:12:03.445 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.enable_blob_files: false 2024-04-24T09:12:03.445 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.min_blob_size: 0 2024-04-24T09:12:03.445 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.blob_file_size: 268435456 2024-04-24T09:12:03.445 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-24T09:12:03.445 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-24T09:12:03.446 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-24T09:12:03.446 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-24T09:12:03.446 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-24T09:12:03.446 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.blob_file_starting_level: 0 2024-04-24T09:12:03.446 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-24T09:12:03.446 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-24T09:12:03.446 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-24T09:12:03.446 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 5eef95e3-4d99-4e32-85f7-2dd8bbb0279c 2024-04-24T09:12:03.446 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713949923071150, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-24T09:12:03.446 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-24T09:12:03.446 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713949923071558, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713949923, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "5eef95e3-4d99-4e32-85f7-2dd8bbb0279c", "db_session_id": "7SNO4A95W038MX2U5JBW", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-24T09:12:03.446 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713949923071637, "job": 1, "event": "recovery_finished"} 2024-04-24T09:12:03.447 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-24T09:12:03.447 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-24T09:12:03.447 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55bce3cb2000 2024-04-24T09:12:03.447 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: DB pointer 0x55bce3c9c000 2024-04-24T09:12:03.447 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-24T09:12:03.447 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-24T09:12:03.447 INFO:journalctl@ceph.mon.b.smithi114.stdout: ** DB Stats ** 2024-04-24T09:12:03.447 INFO:journalctl@ceph.mon.b.smithi114.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T09:12:03.447 INFO:journalctl@ceph.mon.b.smithi114.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-24T09:12:03.447 INFO:journalctl@ceph.mon.b.smithi114.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T09:12:03.447 INFO:journalctl@ceph.mon.b.smithi114.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T09:12:03.447 INFO:journalctl@ceph.mon.b.smithi114.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-24T09:12:03.447 INFO:journalctl@ceph.mon.b.smithi114.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T09:12:03.447 INFO:journalctl@ceph.mon.b.smithi114.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T09:12:03.448 INFO:journalctl@ceph.mon.b.smithi114.stdout: 2024-04-24T09:12:03.448 INFO:journalctl@ceph.mon.b.smithi114.stdout: ** Compaction Stats [default] ** 2024-04-24T09:12:03.448 INFO:journalctl@ceph.mon.b.smithi114.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T09:12:03.448 INFO:journalctl@ceph.mon.b.smithi114.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-24T09:12:03.448 INFO:journalctl@ceph.mon.b.smithi114.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-24T09:12:03.448 INFO:journalctl@ceph.mon.b.smithi114.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-24T09:12:03.448 INFO:journalctl@ceph.mon.b.smithi114.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-24T09:12:03.448 INFO:journalctl@ceph.mon.b.smithi114.stdout: 2024-04-24T09:12:03.448 INFO:journalctl@ceph.mon.b.smithi114.stdout: ** Compaction Stats [default] ** 2024-04-24T09:12:03.448 INFO:journalctl@ceph.mon.b.smithi114.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T09:12:03.448 INFO:journalctl@ceph.mon.b.smithi114.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-24T09:12:03.448 INFO:journalctl@ceph.mon.b.smithi114.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-24T09:12:03.448 INFO:journalctl@ceph.mon.b.smithi114.stdout: 2024-04-24T09:12:03.448 INFO:journalctl@ceph.mon.b.smithi114.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-24T09:12:03.449 INFO:journalctl@ceph.mon.b.smithi114.stdout: 2024-04-24T09:12:03.449 INFO:journalctl@ceph.mon.b.smithi114.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T09:12:03.449 INFO:journalctl@ceph.mon.b.smithi114.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-24T09:12:03.449 INFO:journalctl@ceph.mon.b.smithi114.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-24T09:12:03.449 INFO:journalctl@ceph.mon.b.smithi114.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-24T09:12:03.449 INFO:journalctl@ceph.mon.b.smithi114.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-24T09:12:03.449 INFO:journalctl@ceph.mon.b.smithi114.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-24T09:12:03.449 INFO:journalctl@ceph.mon.b.smithi114.stdout: Cumulative compaction: 0.00 GB write, 0.68 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T09:12:03.449 INFO:journalctl@ceph.mon.b.smithi114.stdout: Interval compaction: 0.00 GB write, 0.68 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T09:12:03.449 INFO:journalctl@ceph.mon.b.smithi114.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-24T09:12:03.449 INFO:journalctl@ceph.mon.b.smithi114.stdout: Block cache BinnedLRUCache@0x55bce3bc5090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-04-24T09:12:03.449 INFO:journalctl@ceph.mon.b.smithi114.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-24T09:12:03.449 INFO:journalctl@ceph.mon.b.smithi114.stdout: 2024-04-24T09:12:03.449 INFO:journalctl@ceph.mon.b.smithi114.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-24T09:12:03.450 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-04-24T09:12:03.450 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: using public_addr v2:172.21.15.114:0/0 -> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] 2024-04-24T09:12:03.450 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: starting mon.b rank -1 at public addrs [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] at bind addrs [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 30ee9be2-021a-11ef-bc93-c7b262605968 2024-04-24T09:12:03.450 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: mon.b@-1(???) e0 preinit fsid 30ee9be2-021a-11ef-bc93-c7b262605968 2024-04-24T09:12:03.450 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: mon.b@-1(synchronizing).mds e1 new map 2024-04-24T09:12:03.450 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: mon.b@-1(synchronizing).mds e1 print_map 2024-04-24T09:12:03.450 INFO:journalctl@ceph.mon.b.smithi114.stdout: e1 2024-04-24T09:12:03.450 INFO:journalctl@ceph.mon.b.smithi114.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T09:12:03.450 INFO:journalctl@ceph.mon.b.smithi114.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-24T09:12:03.450 INFO:journalctl@ceph.mon.b.smithi114.stdout: legacy client fscid: -1 2024-04-24T09:12:03.450 INFO:journalctl@ceph.mon.b.smithi114.stdout: 2024-04-24T09:12:03.450 INFO:journalctl@ceph.mon.b.smithi114.stdout: No filesystems configured 2024-04-24T09:12:03.450 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-24T09:12:03.451 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-24T09:12:03.451 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-24T09:12:03.451 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-24T09:12:03.451 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-24T09:12:03.451 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-24T09:12:03.451 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-24T09:12:03.451 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T09:12:03.451 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T09:12:03.451 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T09:12:03.451 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: Deploying cephadm binary to smithi114 2024-04-24T09:12:03.451 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:03.451 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:03.451 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:03.452 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:12:03.452 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:03.452 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: Added host smithi114 2024-04-24T09:12:03.452 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:03.452 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:03.452 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:03.452 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:03.452 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T09:12:03.453 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:03.453 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:03.453 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:03.453 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:03.453 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:03.453 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi114", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:12:03.453 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:03.453 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:12:03.454 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: Updating smithi114:/etc/ceph/ceph.conf 2024-04-24T09:12:03.454 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: Updating smithi114:/var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T09:12:03.454 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: Updating smithi114:/etc/ceph/ceph.client.admin.keyring 2024-04-24T09:12:03.454 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:03.454 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:03.454 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:03.454 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: Updating smithi114:/var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T09:12:03.454 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi149", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:12:03.455 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:03.455 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: Deploying cephadm binary to smithi149 2024-04-24T09:12:03.455 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:03.455 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:03.455 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:03.455 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:12:03.455 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: Added host smithi149 2024-04-24T09:12:03.455 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:03.455 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:03.456 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:03.456 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:03.456 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:03.456 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T09:12:03.456 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:03.456 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:03.457 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:03.457 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:03.457 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:03.457 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:12:03.457 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:03.457 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:12:03.457 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: Updating smithi149:/etc/ceph/ceph.conf 2024-04-24T09:12:03.457 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: Updating smithi149:/var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T09:12:03.458 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-04-24T09:12:03.458 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: Updating smithi149:/var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T09:12:03.458 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:03.458 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:03.458 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:03.458 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/2016073856' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-24T09:12:03.458 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:03.459 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/2016073856' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-24T09:12:03.459 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T09:12:03.459 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:03.459 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi090:172.21.15.90=a;smithi114:172.21.15.114=b;smithi149:172.21.15.149=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:12:03.459 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: Saving service mon spec with placement smithi090:172.21.15.90=a;smithi114:172.21.15.114=b;smithi149:172.21.15.149=c;count:3 2024-04-24T09:12:03.459 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:03.459 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:12:03.459 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:03.459 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:12:03.460 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:03.460 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T09:12:03.460 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:03.460 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: Deploying daemon mon.c on smithi149 2024-04-24T09:12:03.460 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:03.460 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:03.460 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T09:12:03.461 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: mon.a calling monitor election 2024-04-24T09:12:03.461 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T09:12:03.461 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T09:12:03.461 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:03.461 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T09:12:03.461 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: mon.c calling monitor election 2024-04-24T09:12:03.462 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T09:12:03.462 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:03.462 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T09:12:03.462 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T09:12:03.462 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-24T09:12:03.462 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: monmap epoch 2 2024-04-24T09:12:03.462 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: fsid 30ee9be2-021a-11ef-bc93-c7b262605968 2024-04-24T09:12:03.462 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: last_changed 2024-04-24T09:11:54.053393+0000 2024-04-24T09:12:03.463 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: created 2024-04-24T09:09:44.652569+0000 2024-04-24T09:12:03.463 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: min_mon_release 19 (squid) 2024-04-24T09:12:03.463 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: election_strategy: 1 2024-04-24T09:12:03.463 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-04-24T09:12:03.463 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: 1: [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon.c 2024-04-24T09:12:03.463 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: fsmap 2024-04-24T09:12:03.463 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T09:12:03.463 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: mgrmap e13: x(active, since 61s) 2024-04-24T09:12:03.463 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: overall HEALTH_OK 2024-04-24T09:12:03.463 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:03.463 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:03.463 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:03.463 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T09:12:03.464 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:03.464 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: Deploying daemon mon.b on smithi114 2024-04-24T09:12:03.464 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:03.464 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T09:12:03.464 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: from='client.? 172.21.15.149:0/744423326' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T09:12:03.464 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:03.464 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 ceph-mon[32501]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-24T09:12:03.464 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 bash[32482]: 5d44667e66fbb1239da3e5ddc4a304f70e5efd0468274673dd9e281550c82713 2024-04-24T09:12:03.464 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:03 smithi114 systemd[1]: Started Ceph mon.b for 30ee9be2-021a-11ef-bc93-c7b262605968. 2024-04-24T09:12:08.385 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.384+0000 7fbf59a5f700 1 -- 172.21.15.149:0/55499662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf3c0035c0 msgr2=0x7fbf3c005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:08.385 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.384+0000 7fbf59a5f700 1 --2- 172.21.15.149:0/55499662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf3c0035c0 0x7fbf3c005a50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbf54102550 tx=0x7fbf4400fe60 comp rx=0 tx=0).stop 2024-04-24T09:12:08.385 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.384+0000 7fbf59a5f700 1 -- 172.21.15.149:0/55499662 shutdown_connections 2024-04-24T09:12:08.385 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.384+0000 7fbf59a5f700 1 --2- 172.21.15.149:0/55499662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf3c0035c0 0x7fbf3c005a50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:08.385 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.384+0000 7fbf59a5f700 1 -- 172.21.15.149:0/55499662 >> 172.21.15.149:0/55499662 conn(0x7fbf540fe0f0 msgr2=0x7fbf54100510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:08.385 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.384+0000 7fbf59a5f700 1 -- 172.21.15.149:0/55499662 shutdown_connections 2024-04-24T09:12:08.385 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.384+0000 7fbf59a5f700 1 -- 172.21.15.149:0/55499662 wait complete. 2024-04-24T09:12:08.385 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.385+0000 7fbf59a5f700 1 Processor -- start 2024-04-24T09:12:08.387 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.385+0000 7fbf59a5f700 1 -- start start 2024-04-24T09:12:08.387 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.385+0000 7fbf59a5f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf3c0035c0 0x7fbf54198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:08.387 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.385+0000 7fbf59a5f700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf54198740 0x7fbf5419cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:08.387 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.385+0000 7fbf59a5f700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf5419d010 0x7fbf5419d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:08.387 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.385+0000 7fbf59a5f700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5410a670 con 0x7fbf3c0035c0 2024-04-24T09:12:08.387 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.385+0000 7fbf59a5f700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5410a4f0 con 0x7fbf54198740 2024-04-24T09:12:08.387 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.385+0000 7fbf59a5f700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5410a7f0 con 0x7fbf5419d010 2024-04-24T09:12:08.387 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.385+0000 7fbf537fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf5419d010 0x7fbf5419d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:08.387 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.386+0000 7fbf537fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf5419d010 0x7fbf5419d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:39958/0 (socket says 172.21.15.149:39958) 2024-04-24T09:12:08.387 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.386+0000 7fbf537fe700 1 -- 172.21.15.149:0/974084649 learned_addr learned my addr 172.21.15.149:0/974084649 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-04-24T09:12:08.388 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.386+0000 7fbf537fe700 1 -- 172.21.15.149:0/974084649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf5419d010 msgr2=0x7fbf5419d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-04-24T09:12:08.388 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.386+0000 7fbf537fe700 1 -- 172.21.15.149:0/974084649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf5419d010 msgr2=0x7fbf5419d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-24T09:12:08.388 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.386+0000 7fbf537fe700 1 --2- 172.21.15.149:0/974084649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf5419d010 0x7fbf5419d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T09:12:08.388 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.386+0000 7fbf537fe700 1 --2- 172.21.15.149:0/974084649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf5419d010 0x7fbf5419d4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T09:12:08.388 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.386+0000 7fbf527fc700 1 --2- 172.21.15.149:0/974084649 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf54198740 0x7fbf5419cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:08.388 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.386+0000 7fbf52ffd700 1 --2- 172.21.15.149:0/974084649 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf3c0035c0 0x7fbf54198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:08.388 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.386+0000 7fbf527fc700 1 -- 172.21.15.149:0/974084649 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf54198740 msgr2=0x7fbf5419cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-24T09:12:08.388 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.386+0000 7fbf527fc700 1 -- 172.21.15.149:0/974084649 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf54198740 msgr2=0x7fbf5419cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-24T09:12:08.388 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.386+0000 7fbf527fc700 1 --2- 172.21.15.149:0/974084649 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf54198740 0x7fbf5419cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T09:12:08.388 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.386+0000 7fbf527fc700 1 --2- 172.21.15.149:0/974084649 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf54198740 0x7fbf5419cad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T09:12:08.388 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.386+0000 7fbf52ffd700 1 -- 172.21.15.149:0/974084649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf5419d010 msgr2=0x7fbf5419d4c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T09:12:08.388 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.386+0000 7fbf52ffd700 1 --2- 172.21.15.149:0/974084649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf5419d010 0x7fbf5419d4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:08.388 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.386+0000 7fbf52ffd700 1 -- 172.21.15.149:0/974084649 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf54198740 msgr2=0x7fbf5419cad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T09:12:08.388 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.386+0000 7fbf52ffd700 1 --2- 172.21.15.149:0/974084649 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf54198740 0x7fbf5419cad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:08.389 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.386+0000 7fbf52ffd700 1 -- 172.21.15.149:0/974084649 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf541a4d80 con 0x7fbf3c0035c0 2024-04-24T09:12:08.389 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.387+0000 7fbf52ffd700 1 --2- 172.21.15.149:0/974084649 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf3c0035c0 0x7fbf54198200 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbf54102550 tx=0x7fbf44005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:08.390 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.387+0000 7fbf58a5d700 1 -- 172.21.15.149:0/974084649 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4402b020 con 0x7fbf3c0035c0 2024-04-24T09:12:08.390 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.387+0000 7fbf58a5d700 1 -- 172.21.15.149:0/974084649 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf44004110 con 0x7fbf3c0035c0 2024-04-24T09:12:08.390 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.387+0000 7fbf59a5f700 1 -- 172.21.15.149:0/974084649 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf541a4f50 con 0x7fbf3c0035c0 2024-04-24T09:12:08.390 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.387+0000 7fbf58a5d700 1 -- 172.21.15.149:0/974084649 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf44022040 con 0x7fbf3c0035c0 2024-04-24T09:12:08.390 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.388+0000 7fbf59a5f700 1 -- 172.21.15.149:0/974084649 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf541a53b0 con 0x7fbf3c0035c0 2024-04-24T09:12:08.390 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.389+0000 7fbf59a5f700 1 -- 172.21.15.149:0/974084649 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf54064150 con 0x7fbf3c0035c0 2024-04-24T09:12:08.390 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.389+0000 7fbf58a5d700 1 -- 172.21.15.149:0/974084649 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fbf4402e020 con 0x7fbf3c0035c0 2024-04-24T09:12:08.390 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.389+0000 7fbf58a5d700 1 --2- 172.21.15.149:0/974084649 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fbf40039ae0 0x7fbf4003bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:08.390 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.389+0000 7fbf58a5d700 1 -- 172.21.15.149:0/974084649 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fbf4404f120 con 0x7fbf3c0035c0 2024-04-24T09:12:08.391 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.390+0000 7fbf527fc700 1 --2- 172.21.15.149:0/974084649 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fbf40039ae0 0x7fbf4003bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:08.391 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.391+0000 7fbf527fc700 1 --2- 172.21.15.149:0/974084649 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fbf40039ae0 0x7fbf4003bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbf48009850 tx=0x7fbf48008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:08.406 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.406+0000 7fbf58a5d700 1 -- 172.21.15.149:0/974084649 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbf44035020 con 0x7fbf3c0035c0 2024-04-24T09:12:08.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:08 smithi090 ceph-mon[27825]: mon.c calling monitor election 2024-04-24T09:12:08.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:08 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T09:12:08.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:08 smithi090 ceph-mon[27825]: mon.a calling monitor election 2024-04-24T09:12:08.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:08 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T09:12:08.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:08 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T09:12:08.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:08 smithi090 ceph-mon[27825]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:08.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:08 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T09:12:08.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:08 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T09:12:08.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:08 smithi090 ceph-mon[27825]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:08.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:08 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T09:12:08.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:08 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T09:12:08.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:08 smithi090 ceph-mon[27825]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:08.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:08 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T09:12:08.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:08 smithi090 ceph-mon[27825]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-24T09:12:08.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:08 smithi090 ceph-mon[27825]: monmap epoch 3 2024-04-24T09:12:08.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:08 smithi090 ceph-mon[27825]: fsid 30ee9be2-021a-11ef-bc93-c7b262605968 2024-04-24T09:12:08.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:08 smithi090 ceph-mon[27825]: last_changed 2024-04-24T09:12:03.096033+0000 2024-04-24T09:12:08.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:08 smithi090 ceph-mon[27825]: created 2024-04-24T09:09:44.652569+0000 2024-04-24T09:12:08.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:08 smithi090 ceph-mon[27825]: min_mon_release 19 (squid) 2024-04-24T09:12:08.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:08 smithi090 ceph-mon[27825]: election_strategy: 1 2024-04-24T09:12:08.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:08 smithi090 ceph-mon[27825]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-04-24T09:12:08.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:08 smithi090 ceph-mon[27825]: 1: [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon.c 2024-04-24T09:12:08.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:08 smithi090 ceph-mon[27825]: 2: [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] mon.b 2024-04-24T09:12:08.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:08 smithi090 ceph-mon[27825]: fsmap 2024-04-24T09:12:08.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:08 smithi090 ceph-mon[27825]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T09:12:08.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:08 smithi090 ceph-mon[27825]: mgrmap e13: x(active, since 70s) 2024-04-24T09:12:08.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:08 smithi090 ceph-mon[27825]: overall HEALTH_OK 2024-04-24T09:12:08.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:08 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:08.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:08 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:08.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:08 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:08.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:08 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:08.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:08 smithi149 ceph-mon[32592]: mon.c calling monitor election 2024-04-24T09:12:08.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:08 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T09:12:08.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:08 smithi149 ceph-mon[32592]: mon.a calling monitor election 2024-04-24T09:12:08.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:08 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T09:12:08.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:08 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T09:12:08.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:08 smithi149 ceph-mon[32592]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:08.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:08 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T09:12:08.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:08 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T09:12:08.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:08 smithi149 ceph-mon[32592]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:08.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:08 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T09:12:08.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:08 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T09:12:08.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:08 smithi149 ceph-mon[32592]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:08.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:08 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T09:12:08.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:08 smithi149 ceph-mon[32592]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-24T09:12:08.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:08 smithi149 ceph-mon[32592]: monmap epoch 3 2024-04-24T09:12:08.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:08 smithi149 ceph-mon[32592]: fsid 30ee9be2-021a-11ef-bc93-c7b262605968 2024-04-24T09:12:08.613 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:08 smithi149 ceph-mon[32592]: last_changed 2024-04-24T09:12:03.096033+0000 2024-04-24T09:12:08.613 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:08 smithi149 ceph-mon[32592]: created 2024-04-24T09:09:44.652569+0000 2024-04-24T09:12:08.613 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:08 smithi149 ceph-mon[32592]: min_mon_release 19 (squid) 2024-04-24T09:12:08.613 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:08 smithi149 ceph-mon[32592]: election_strategy: 1 2024-04-24T09:12:08.613 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:08 smithi149 ceph-mon[32592]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-04-24T09:12:08.613 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:08 smithi149 ceph-mon[32592]: 1: [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon.c 2024-04-24T09:12:08.613 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:08 smithi149 ceph-mon[32592]: 2: [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] mon.b 2024-04-24T09:12:08.613 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:08 smithi149 ceph-mon[32592]: fsmap 2024-04-24T09:12:08.613 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:08 smithi149 ceph-mon[32592]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T09:12:08.613 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:08 smithi149 ceph-mon[32592]: mgrmap e13: x(active, since 70s) 2024-04-24T09:12:08.613 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:08 smithi149 ceph-mon[32592]: overall HEALTH_OK 2024-04-24T09:12:08.613 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:08 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:08.613 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:08 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:08.613 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:08 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:08.613 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:08 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:08.704 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.703+0000 7fbf59a5f700 1 -- 172.21.15.149:0/974084649 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fbf5402e1a0 con 0x7fbf3c0035c0 2024-04-24T09:12:08.705 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.704+0000 7fbf58a5d700 1 -- 172.21.15.149:0/974084649 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7fbf4402f3b0 con 0x7fbf3c0035c0 2024-04-24T09:12:08.705 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-04-24T09:12:08.705 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":3,"fsid":"30ee9be2-021a-11ef-bc93-c7b262605968","modified":"2024-04-24T09:12:03.096033Z","created":"2024-04-24T09:09:44.652569Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:3300","nonce":0},{"type":"v1","addr":"172.21.15.149:6789","nonce":0}]},"addr":"172.21.15.149:6789/0","public_addr":"172.21.15.149:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:3300","nonce":0},{"type":"v1","addr":"172.21.15.114:6789","nonce":0}]},"addr":"172.21.15.114:6789/0","public_addr":"172.21.15.114:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-24T09:12:08.707 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.707+0000 7fbf59a5f700 1 -- 172.21.15.149:0/974084649 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fbf40039ae0 msgr2=0x7fbf4003bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:08.707 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.707+0000 7fbf59a5f700 1 --2- 172.21.15.149:0/974084649 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fbf40039ae0 0x7fbf4003bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbf48009850 tx=0x7fbf48008040 comp rx=0 tx=0).stop 2024-04-24T09:12:08.708 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.707+0000 7fbf59a5f700 1 -- 172.21.15.149:0/974084649 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf3c0035c0 msgr2=0x7fbf54198200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:08.708 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.707+0000 7fbf59a5f700 1 --2- 172.21.15.149:0/974084649 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf3c0035c0 0x7fbf54198200 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbf54102550 tx=0x7fbf44005720 comp rx=0 tx=0).stop 2024-04-24T09:12:08.708 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.707+0000 7fbf59a5f700 1 -- 172.21.15.149:0/974084649 shutdown_connections 2024-04-24T09:12:08.708 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.707+0000 7fbf59a5f700 1 --2- 172.21.15.149:0/974084649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf5419d010 0x7fbf5419d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:08.708 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.707+0000 7fbf59a5f700 1 --2- 172.21.15.149:0/974084649 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fbf40039ae0 0x7fbf4003bfa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:08.708 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.707+0000 7fbf59a5f700 1 --2- 172.21.15.149:0/974084649 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf3c0035c0 0x7fbf54198200 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:08.708 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.707+0000 7fbf59a5f700 1 --2- 172.21.15.149:0/974084649 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf54198740 0x7fbf5419cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:08.708 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.707+0000 7fbf59a5f700 1 -- 172.21.15.149:0/974084649 >> 172.21.15.149:0/974084649 conn(0x7fbf540fe0f0 msgr2=0x7fbf540fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:08.708 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.707+0000 7fbf59a5f700 1 -- 172.21.15.149:0/974084649 shutdown_connections 2024-04-24T09:12:08.708 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:08.707+0000 7fbf59a5f700 1 -- 172.21.15.149:0/974084649 wait complete. 2024-04-24T09:12:08.709 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 3 2024-04-24T09:12:09.296 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:09 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:12:09.296 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:09 smithi149 ceph-mon[32592]: from='client.? 172.21.15.149:0/974084649' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T09:12:09.296 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:09 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T09:12:09.308 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:09 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:12:09.308 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:09 smithi090 ceph-mon[27825]: from='client.? 172.21.15.149:0/974084649' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T09:12:09.308 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:09 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T09:12:09.322 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-24T09:12:09.322 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-24T09:12:09.766 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:12:10.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:10 smithi114 ceph-mon[32501]: mon.c calling monitor election 2024-04-24T09:12:10.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:10 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T09:12:10.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:10 smithi114 ceph-mon[32501]: mon.a calling monitor election 2024-04-24T09:12:10.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:10 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T09:12:10.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:10 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T09:12:10.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:10 smithi114 ceph-mon[32501]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:10.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:10 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T09:12:10.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:10 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T09:12:10.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:10 smithi114 ceph-mon[32501]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:10.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:10 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T09:12:10.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:10 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T09:12:10.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:10 smithi114 ceph-mon[32501]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:10.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:10 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T09:12:10.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:10 smithi114 ceph-mon[32501]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-24T09:12:10.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:10 smithi114 ceph-mon[32501]: monmap epoch 3 2024-04-24T09:12:10.428 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:10 smithi114 ceph-mon[32501]: fsid 30ee9be2-021a-11ef-bc93-c7b262605968 2024-04-24T09:12:10.428 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:10 smithi114 ceph-mon[32501]: last_changed 2024-04-24T09:12:03.096033+0000 2024-04-24T09:12:10.428 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:10 smithi114 ceph-mon[32501]: created 2024-04-24T09:09:44.652569+0000 2024-04-24T09:12:10.428 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:10 smithi114 ceph-mon[32501]: min_mon_release 19 (squid) 2024-04-24T09:12:10.428 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:10 smithi114 ceph-mon[32501]: election_strategy: 1 2024-04-24T09:12:10.428 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:10 smithi114 ceph-mon[32501]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-04-24T09:12:10.428 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:10 smithi114 ceph-mon[32501]: 1: [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon.c 2024-04-24T09:12:10.428 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:10 smithi114 ceph-mon[32501]: 2: [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] mon.b 2024-04-24T09:12:10.428 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:10 smithi114 ceph-mon[32501]: fsmap 2024-04-24T09:12:10.428 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:10 smithi114 ceph-mon[32501]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T09:12:10.428 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:10 smithi114 ceph-mon[32501]: mgrmap e13: x(active, since 70s) 2024-04-24T09:12:10.428 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:10 smithi114 ceph-mon[32501]: overall HEALTH_OK 2024-04-24T09:12:10.428 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:10 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:10.428 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:10 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:10.429 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:10 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:10.429 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:10 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:10.429 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:10 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:12:10.429 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:10 smithi114 ceph-mon[32501]: from='client.? 172.21.15.149:0/974084649' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T09:12:10.429 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:10 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T09:12:11.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:11 smithi090 ceph-mon[27825]: mon.b calling monitor election 2024-04-24T09:12:11.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:11 smithi090 ceph-mon[27825]: mon.b calling monitor election 2024-04-24T09:12:11.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:11 smithi090 ceph-mon[27825]: mon.c calling monitor election 2024-04-24T09:12:11.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:11 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:12:11.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:11 smithi090 ceph-mon[27825]: mon.a calling monitor election 2024-04-24T09:12:11.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:11 smithi090 ceph-mon[27825]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-24T09:12:11.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:11 smithi090 ceph-mon[27825]: monmap epoch 3 2024-04-24T09:12:11.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:11 smithi090 ceph-mon[27825]: fsid 30ee9be2-021a-11ef-bc93-c7b262605968 2024-04-24T09:12:11.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:11 smithi090 ceph-mon[27825]: last_changed 2024-04-24T09:12:03.096033+0000 2024-04-24T09:12:11.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:11 smithi090 ceph-mon[27825]: created 2024-04-24T09:09:44.652569+0000 2024-04-24T09:12:11.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:11 smithi090 ceph-mon[27825]: min_mon_release 19 (squid) 2024-04-24T09:12:11.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:11 smithi090 ceph-mon[27825]: election_strategy: 1 2024-04-24T09:12:11.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:11 smithi090 ceph-mon[27825]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-04-24T09:12:11.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:11 smithi090 ceph-mon[27825]: 1: [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon.c 2024-04-24T09:12:11.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:11 smithi090 ceph-mon[27825]: 2: [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] mon.b 2024-04-24T09:12:11.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:11 smithi090 ceph-mon[27825]: fsmap 2024-04-24T09:12:11.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:11 smithi090 ceph-mon[27825]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T09:12:11.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:11 smithi090 ceph-mon[27825]: mgrmap e13: x(active, since 72s) 2024-04-24T09:12:11.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:11 smithi090 ceph-mon[27825]: overall HEALTH_OK 2024-04-24T09:12:11.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:11 smithi090 ceph-mon[27825]: Updating smithi090:/etc/ceph/ceph.conf 2024-04-24T09:12:11.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:11 smithi090 ceph-mon[27825]: Updating smithi114:/etc/ceph/ceph.conf 2024-04-24T09:12:11.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:11 smithi090 ceph-mon[27825]: Updating smithi149:/etc/ceph/ceph.conf 2024-04-24T09:12:11.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:11 smithi090 ceph-mon[27825]: Updating smithi114:/var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T09:12:11.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:11 smithi090 ceph-mon[27825]: Updating smithi149:/var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T09:12:11.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:11 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:11.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:11 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:11.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:11 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:11.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:11 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:11.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:11 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T09:12:11.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:11 smithi149 ceph-mon[32592]: mon.b calling monitor election 2024-04-24T09:12:11.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:11 smithi149 ceph-mon[32592]: mon.b calling monitor election 2024-04-24T09:12:11.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:11 smithi149 ceph-mon[32592]: mon.c calling monitor election 2024-04-24T09:12:11.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:11 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:12:11.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:11 smithi149 ceph-mon[32592]: mon.a calling monitor election 2024-04-24T09:12:11.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:11 smithi149 ceph-mon[32592]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-24T09:12:11.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:11 smithi149 ceph-mon[32592]: monmap epoch 3 2024-04-24T09:12:11.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:11 smithi149 ceph-mon[32592]: fsid 30ee9be2-021a-11ef-bc93-c7b262605968 2024-04-24T09:12:11.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:11 smithi149 ceph-mon[32592]: last_changed 2024-04-24T09:12:03.096033+0000 2024-04-24T09:12:11.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:11 smithi149 ceph-mon[32592]: created 2024-04-24T09:09:44.652569+0000 2024-04-24T09:12:11.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:11 smithi149 ceph-mon[32592]: min_mon_release 19 (squid) 2024-04-24T09:12:11.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:11 smithi149 ceph-mon[32592]: election_strategy: 1 2024-04-24T09:12:11.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:11 smithi149 ceph-mon[32592]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-04-24T09:12:11.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:11 smithi149 ceph-mon[32592]: 1: [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon.c 2024-04-24T09:12:11.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:11 smithi149 ceph-mon[32592]: 2: [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] mon.b 2024-04-24T09:12:11.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:11 smithi149 ceph-mon[32592]: fsmap 2024-04-24T09:12:11.363 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:11 smithi149 ceph-mon[32592]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T09:12:11.363 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:11 smithi149 ceph-mon[32592]: mgrmap e13: x(active, since 72s) 2024-04-24T09:12:11.363 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:11 smithi149 ceph-mon[32592]: overall HEALTH_OK 2024-04-24T09:12:11.363 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:11 smithi149 ceph-mon[32592]: Updating smithi090:/etc/ceph/ceph.conf 2024-04-24T09:12:11.363 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:11 smithi149 ceph-mon[32592]: Updating smithi114:/etc/ceph/ceph.conf 2024-04-24T09:12:11.363 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:11 smithi149 ceph-mon[32592]: Updating smithi149:/etc/ceph/ceph.conf 2024-04-24T09:12:11.363 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:11 smithi149 ceph-mon[32592]: Updating smithi114:/var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T09:12:11.363 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:11 smithi149 ceph-mon[32592]: Updating smithi149:/var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T09:12:11.363 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:11 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:11.363 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:11 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:11.363 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:11 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:11.363 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:11 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:11.363 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:11 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T09:12:11.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:11 smithi114 ceph-mon[32501]: mon.b calling monitor election 2024-04-24T09:12:11.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:11 smithi114 ceph-mon[32501]: mon.b calling monitor election 2024-04-24T09:12:11.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:11 smithi114 ceph-mon[32501]: mon.c calling monitor election 2024-04-24T09:12:11.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:11 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:12:11.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:11 smithi114 ceph-mon[32501]: mon.a calling monitor election 2024-04-24T09:12:11.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:11 smithi114 ceph-mon[32501]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-24T09:12:11.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:11 smithi114 ceph-mon[32501]: monmap epoch 3 2024-04-24T09:12:11.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:11 smithi114 ceph-mon[32501]: fsid 30ee9be2-021a-11ef-bc93-c7b262605968 2024-04-24T09:12:11.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:11 smithi114 ceph-mon[32501]: last_changed 2024-04-24T09:12:03.096033+0000 2024-04-24T09:12:11.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:11 smithi114 ceph-mon[32501]: created 2024-04-24T09:09:44.652569+0000 2024-04-24T09:12:11.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:11 smithi114 ceph-mon[32501]: min_mon_release 19 (squid) 2024-04-24T09:12:11.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:11 smithi114 ceph-mon[32501]: election_strategy: 1 2024-04-24T09:12:11.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:11 smithi114 ceph-mon[32501]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-04-24T09:12:11.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:11 smithi114 ceph-mon[32501]: 1: [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon.c 2024-04-24T09:12:11.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:11 smithi114 ceph-mon[32501]: 2: [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] mon.b 2024-04-24T09:12:11.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:11 smithi114 ceph-mon[32501]: fsmap 2024-04-24T09:12:11.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:11 smithi114 ceph-mon[32501]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T09:12:11.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:11 smithi114 ceph-mon[32501]: mgrmap e13: x(active, since 72s) 2024-04-24T09:12:11.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:11 smithi114 ceph-mon[32501]: overall HEALTH_OK 2024-04-24T09:12:11.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:11 smithi114 ceph-mon[32501]: Updating smithi090:/etc/ceph/ceph.conf 2024-04-24T09:12:11.428 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:11 smithi114 ceph-mon[32501]: Updating smithi114:/etc/ceph/ceph.conf 2024-04-24T09:12:11.428 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:11 smithi114 ceph-mon[32501]: Updating smithi149:/etc/ceph/ceph.conf 2024-04-24T09:12:11.428 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:11 smithi114 ceph-mon[32501]: Updating smithi114:/var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T09:12:11.428 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:11 smithi114 ceph-mon[32501]: Updating smithi149:/var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T09:12:11.428 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:11 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:11.428 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:11 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:11.428 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:11 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:11.428 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:11 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:11.428 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:11 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T09:12:11.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:11.983+0000 7f6a2e000700 1 -- 172.21.15.90:0/418922303 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a28074f20 msgr2=0x7f6a28075300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:11.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:11.983+0000 7f6a2e000700 1 --2- 172.21.15.90:0/418922303 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a28074f20 0x7f6a28075300 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6a18009910 tx=0x7f6a18009c20 comp rx=0 tx=0).stop 2024-04-24T09:12:11.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:11.984+0000 7f6a2e000700 1 -- 172.21.15.90:0/418922303 shutdown_connections 2024-04-24T09:12:11.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:11.984+0000 7f6a2e000700 1 --2- 172.21.15.90:0/418922303 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a28074f20 0x7f6a28075300 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:11.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:11.984+0000 7f6a2e000700 1 -- 172.21.15.90:0/418922303 >> 172.21.15.90:0/418922303 conn(0x7f6a280fe030 msgr2=0x7f6a28100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:11.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:11.984+0000 7f6a2e000700 1 -- 172.21.15.90:0/418922303 shutdown_connections 2024-04-24T09:12:11.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:11.984+0000 7f6a2e000700 1 -- 172.21.15.90:0/418922303 wait complete. 2024-04-24T09:12:11.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:11.985+0000 7f6a2e000700 1 Processor -- start 2024-04-24T09:12:11.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:11.985+0000 7f6a2e000700 1 -- start start 2024-04-24T09:12:11.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:11.985+0000 7f6a2e000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a28074f20 0x7f6a2819a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:11.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:11.985+0000 7f6a2e000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6a2819aa60 0x7f6a2819edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:11.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:11.985+0000 7f6a2e000700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6a2819f330 0x7f6a2819f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:11.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:11.985+0000 7f6a2e000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2810edb0 con 0x7f6a28074f20 2024-04-24T09:12:11.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:11.985+0000 7f6a2e000700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2810ec30 con 0x7f6a2819f330 2024-04-24T09:12:11.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:11.985+0000 7f6a2e000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2810ef30 con 0x7f6a2819aa60 2024-04-24T09:12:11.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:11.986+0000 7f6a26a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6a2819aa60 0x7f6a2819edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:11.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:11.986+0000 7f6a27400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a28074f20 0x7f6a2819a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:11.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:11.986+0000 7f6a26a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6a2819aa60 0x7f6a2819edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:57436/0 (socket says 172.21.15.90:57436) 2024-04-24T09:12:11.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:11.986+0000 7f6a26a00700 1 -- 172.21.15.90:0/2479325342 learned_addr learned my addr 172.21.15.90:0/2479325342 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:12:11.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:11.986+0000 7f6a27e00700 1 --2- 172.21.15.90:0/2479325342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6a2819f330 0x7f6a2819f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:11.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:11.986+0000 7f6a27400700 1 -- 172.21.15.90:0/2479325342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6a2819aa60 msgr2=0x7f6a2819edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:11.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:11.986+0000 7f6a27400700 1 --2- 172.21.15.90:0/2479325342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6a2819aa60 0x7f6a2819edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:11.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:11.986+0000 7f6a27400700 1 -- 172.21.15.90:0/2479325342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6a2819f330 msgr2=0x7f6a2819f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:11.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:11.986+0000 7f6a27400700 1 --2- 172.21.15.90:0/2479325342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6a2819f330 0x7f6a2819f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:11.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:11.986+0000 7f6a27400700 1 -- 172.21.15.90:0/2479325342 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a281a70a0 con 0x7f6a28074f20 2024-04-24T09:12:11.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:11.987+0000 7f6a27400700 1 --2- 172.21.15.90:0/2479325342 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a28074f20 0x7f6a2819a520 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6a18005950 tx=0x7f6a18004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:11.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:11.987+0000 7f6a1fe00700 1 -- 172.21.15.90:0/2479325342 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a18026070 con 0x7f6a28074f20 2024-04-24T09:12:11.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:11.987+0000 7f6a1fe00700 1 -- 172.21.15.90:0/2479325342 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a1800b750 con 0x7f6a28074f20 2024-04-24T09:12:11.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:11.987+0000 7f6a2e000700 1 -- 172.21.15.90:0/2479325342 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a281a7270 con 0x7f6a28074f20 2024-04-24T09:12:11.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:11.987+0000 7f6a1fe00700 1 -- 172.21.15.90:0/2479325342 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a18003970 con 0x7f6a28074f20 2024-04-24T09:12:11.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:11.987+0000 7f6a2e000700 1 -- 172.21.15.90:0/2479325342 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a281a7750 con 0x7f6a28074f20 2024-04-24T09:12:11.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:11.988+0000 7f6a1fe00700 1 -- 172.21.15.90:0/2479325342 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f6a18029070 con 0x7f6a28074f20 2024-04-24T09:12:11.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:11.988+0000 7f6a2e000700 1 -- 172.21.15.90:0/2479325342 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a28064150 con 0x7f6a28074f20 2024-04-24T09:12:11.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:11.989+0000 7f6a1fe00700 1 --2- 172.21.15.90:0/2479325342 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f6a0c039ae0 0x7f6a0c03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:11.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:11.989+0000 7f6a1fe00700 1 -- 172.21.15.90:0/2479325342 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6a1804d450 con 0x7f6a28074f20 2024-04-24T09:12:11.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:11.989+0000 7f6a26a00700 1 --2- 172.21.15.90:0/2479325342 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f6a0c039ae0 0x7f6a0c03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:11.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:11.990+0000 7f6a26a00700 1 --2- 172.21.15.90:0/2479325342 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f6a0c039ae0 0x7f6a0c03bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6a14009b80 tx=0x7f6a140092c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:11.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:11.995+0000 7f6a1fe00700 1 -- 172.21.15.90:0/2479325342 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6a18013070 con 0x7f6a28074f20 2024-04-24T09:12:12.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:12.223+0000 7f6a2e000700 1 -- 172.21.15.90:0/2479325342 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f6a2819b9a0 con 0x7f6a28074f20 2024-04-24T09:12:12.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:12.224+0000 7f6a1fe00700 1 -- 172.21.15.90:0/2479325342 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f6a18030020 con 0x7f6a28074f20 2024-04-24T09:12:12.224 INFO:teuthology.orchestra.run.smithi090.stdout:# minimal ceph.conf for 30ee9be2-021a-11ef-bc93-c7b262605968 2024-04-24T09:12:12.224 INFO:teuthology.orchestra.run.smithi090.stdout:[global] 2024-04-24T09:12:12.224 INFO:teuthology.orchestra.run.smithi090.stdout: fsid = 30ee9be2-021a-11ef-bc93-c7b262605968 2024-04-24T09:12:12.224 INFO:teuthology.orchestra.run.smithi090.stdout: mon_host = [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] 2024-04-24T09:12:12.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:12.227+0000 7f6a2e000700 1 -- 172.21.15.90:0/2479325342 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f6a0c039ae0 msgr2=0x7f6a0c03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:12.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:12.227+0000 7f6a2e000700 1 --2- 172.21.15.90:0/2479325342 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f6a0c039ae0 0x7f6a0c03bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6a14009b80 tx=0x7f6a140092c0 comp rx=0 tx=0).stop 2024-04-24T09:12:12.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:12.227+0000 7f6a2e000700 1 -- 172.21.15.90:0/2479325342 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a28074f20 msgr2=0x7f6a2819a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:12.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:12.227+0000 7f6a2e000700 1 --2- 172.21.15.90:0/2479325342 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a28074f20 0x7f6a2819a520 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6a18005950 tx=0x7f6a18004f80 comp rx=0 tx=0).stop 2024-04-24T09:12:12.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:12.227+0000 7f6a2e000700 1 -- 172.21.15.90:0/2479325342 shutdown_connections 2024-04-24T09:12:12.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:12.227+0000 7f6a2e000700 1 --2- 172.21.15.90:0/2479325342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6a2819f330 0x7f6a2819f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:12.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:12.227+0000 7f6a2e000700 1 --2- 172.21.15.90:0/2479325342 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f6a0c039ae0 0x7f6a0c03bfa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:12.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:12.227+0000 7f6a2e000700 1 --2- 172.21.15.90:0/2479325342 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a28074f20 0x7f6a2819a520 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:12.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:12.227+0000 7f6a2e000700 1 --2- 172.21.15.90:0/2479325342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6a2819aa60 0x7f6a2819edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:12.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:12.227+0000 7f6a2e000700 1 -- 172.21.15.90:0/2479325342 >> 172.21.15.90:0/2479325342 conn(0x7f6a280fe030 msgr2=0x7f6a281068e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:12.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:12.227+0000 7f6a2e000700 1 -- 172.21.15.90:0/2479325342 shutdown_connections 2024-04-24T09:12:12.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:12.227+0000 7f6a2e000700 1 -- 172.21.15.90:0/2479325342 wait complete. 2024-04-24T09:12:12.547 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:12 smithi090 ceph-mon[27825]: Updating smithi090:/var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T09:12:12.548 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:12 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:12.548 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:12 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:12.548 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:12 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:12.548 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:12 smithi090 ceph-mon[27825]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:12.548 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:12 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:12.548 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:12 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:12.548 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:12 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:12.548 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:12 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:12.548 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:12 smithi090 ceph-mon[27825]: Reconfiguring mon.a (unknown last config time)... 2024-04-24T09:12:12.548 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:12 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T09:12:12.548 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:12 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T09:12:12.548 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:12 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:12.549 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:12 smithi090 ceph-mon[27825]: Reconfiguring daemon mon.a on smithi090 2024-04-24T09:12:12.549 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:12 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/2479325342' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:12.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:12 smithi149 ceph-mon[32592]: Updating smithi090:/var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T09:12:12.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:12 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:12.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:12 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:12.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:12 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:12.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:12 smithi149 ceph-mon[32592]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:12.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:12 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:12.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:12 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:12.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:12 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:12.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:12 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:12.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:12 smithi149 ceph-mon[32592]: Reconfiguring mon.a (unknown last config time)... 2024-04-24T09:12:12.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:12 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T09:12:12.613 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:12 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T09:12:12.613 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:12 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:12.613 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:12 smithi149 ceph-mon[32592]: Reconfiguring daemon mon.a on smithi090 2024-04-24T09:12:12.613 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:12 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/2479325342' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:12.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:12 smithi114 ceph-mon[32501]: Updating smithi090:/var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T09:12:12.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:12 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:12.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:12 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:12.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:12 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:12.677 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:12 smithi114 ceph-mon[32501]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:12.677 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:12 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:12.677 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:12 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:12.677 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:12 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:12.677 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:12 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:12.677 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:12 smithi114 ceph-mon[32501]: Reconfiguring mon.a (unknown last config time)... 2024-04-24T09:12:12.677 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:12 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T09:12:12.677 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:12 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T09:12:12.677 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:12 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:12.677 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:12 smithi114 ceph-mon[32501]: Reconfiguring daemon mon.a on smithi090 2024-04-24T09:12:12.677 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:12 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/2479325342' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:13.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:13 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:13.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:13 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:13.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:13 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:13.961 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-24T09:12:13.961 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-24T09:12:13.962 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/ceph/ceph.conf 2024-04-24T09:12:14.045 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-24T09:12:14.045 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T09:12:14.159 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-24T09:12:14.159 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/ceph/ceph.conf 2024-04-24T09:12:14.191 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-24T09:12:14.191 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T09:12:14.258 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-04-24T09:12:14.258 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/ceph/ceph.conf 2024-04-24T09:12:14.290 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-04-24T09:12:14.291 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T09:12:14.357 INFO:tasks.cephadm:Adding mgr.x on smithi090 2024-04-24T09:12:14.358 INFO:tasks.cephadm:Adding mgr.y on smithi114 2024-04-24T09:12:14.358 INFO:tasks.cephadm:Adding mgr.z on smithi149 2024-04-24T09:12:14.358 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi090=x;smithi114=y;smithi149=z' 2024-04-24T09:12:14.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:14 smithi149 ceph-mon[32592]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:14.629 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T09:12:14.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:14 smithi090 ceph-mon[27825]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:14.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:14 smithi114 ceph-mon[32501]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:15.639 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:15 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:15.639 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:15 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:15.639 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:15 smithi149 ceph-mon[32592]: Reconfiguring mon.b (monmap changed)... 2024-04-24T09:12:15.640 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:15 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T09:12:15.640 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:15 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T09:12:15.640 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:15 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:15.640 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:15 smithi149 ceph-mon[32592]: Reconfiguring daemon mon.b on smithi114 2024-04-24T09:12:15.838 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:15 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:15.838 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:15 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:15.838 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:15 smithi114 ceph-mon[32501]: Reconfiguring mon.b (monmap changed)... 2024-04-24T09:12:15.838 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:15 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T09:12:15.838 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:15 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T09:12:15.838 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:15 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:15.838 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:15 smithi114 ceph-mon[32501]: Reconfiguring daemon mon.b on smithi114 2024-04-24T09:12:15.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:15 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:15.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:15 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:15.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:15 smithi090 ceph-mon[27825]: Reconfiguring mon.b (monmap changed)... 2024-04-24T09:12:15.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:15 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T09:12:15.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:15 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T09:12:15.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:15 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:15.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:15 smithi090 ceph-mon[27825]: Reconfiguring daemon mon.b on smithi114 2024-04-24T09:12:16.113 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.113+0000 7fdeb74e1700 1 -- 172.21.15.149:0/1338501273 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdeb0101590 msgr2=0x7fdeb01019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:16.113 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.113+0000 7fdeb74e1700 1 --2- 172.21.15.149:0/1338501273 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdeb0101590 0x7fdeb01019b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdeac009910 tx=0x7fdeac009c20 comp rx=0 tx=0).stop 2024-04-24T09:12:16.113 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.113+0000 7fdeb74e1700 1 -- 172.21.15.149:0/1338501273 shutdown_connections 2024-04-24T09:12:16.113 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.113+0000 7fdeb74e1700 1 --2- 172.21.15.149:0/1338501273 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdeb0101590 0x7fdeb01019b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:16.113 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.113+0000 7fdeb74e1700 1 -- 172.21.15.149:0/1338501273 >> 172.21.15.149:0/1338501273 conn(0x7fdeb00fcb30 msgr2=0x7fdeb00fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:16.113 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.113+0000 7fdeb74e1700 1 -- 172.21.15.149:0/1338501273 shutdown_connections 2024-04-24T09:12:16.113 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.113+0000 7fdeb74e1700 1 -- 172.21.15.149:0/1338501273 wait complete. 2024-04-24T09:12:16.116 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.114+0000 7fdeb74e1700 1 Processor -- start 2024-04-24T09:12:16.116 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.114+0000 7fdeb74e1700 1 -- start start 2024-04-24T09:12:16.116 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.114+0000 7fdeb74e1700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdeb0101590 0x7fdeb0197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:16.116 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.114+0000 7fdeb74e1700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdeb0198440 0x7fdeb019d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:16.116 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.114+0000 7fdeb74e1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdeb01988c0 0x7fdeb019fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:16.116 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.114+0000 7fdeb74e1700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb01075d0 con 0x7fdeb01988c0 2024-04-24T09:12:16.117 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.114+0000 7fdeb74e1700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb0107450 con 0x7fdeb0198440 2024-04-24T09:12:16.117 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.114+0000 7fdeb74e1700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb0107750 con 0x7fdeb0101590 2024-04-24T09:12:16.117 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.114+0000 7fdeb527d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdeb0101590 0x7fdeb0197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:16.117 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.114+0000 7fdeb527d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdeb0101590 0x7fdeb0197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:57130/0 (socket says 172.21.15.149:57130) 2024-04-24T09:12:16.117 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.114+0000 7fdeb527d700 1 -- 172.21.15.149:0/139914191 learned_addr learned my addr 172.21.15.149:0/139914191 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-04-24T09:12:16.117 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.114+0000 7fdeb4a7c700 1 --2- 172.21.15.149:0/139914191 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdeb0198440 0x7fdeb019d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:16.117 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.114+0000 7fdeb527d700 1 -- 172.21.15.149:0/139914191 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdeb0198440 msgr2=0x7fdeb019d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:16.117 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.114+0000 7fdeb527d700 1 --2- 172.21.15.149:0/139914191 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdeb0198440 0x7fdeb019d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:16.117 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.114+0000 7fdeb527d700 1 -- 172.21.15.149:0/139914191 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdeb01988c0 msgr2=0x7fdeb019fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:16.117 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.114+0000 7fdeb5a7e700 1 --2- 172.21.15.149:0/139914191 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdeb01988c0 0x7fdeb019fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:16.117 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.114+0000 7fdeb527d700 1 --2- 172.21.15.149:0/139914191 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdeb01988c0 0x7fdeb019fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:16.117 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.114+0000 7fdeb527d700 1 -- 172.21.15.149:0/139914191 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeb01a0310 con 0x7fdeb0101590 2024-04-24T09:12:16.117 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.114+0000 7fdeb527d700 1 --2- 172.21.15.149:0/139914191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdeb0101590 0x7fdeb0197f00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fdeac00be40 tx=0x7fdeac00be70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:16.117 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.115+0000 7fdea67fc700 1 -- 172.21.15.149:0/139914191 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeac027070 con 0x7fdeb0101590 2024-04-24T09:12:16.118 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.115+0000 7fdea67fc700 1 -- 172.21.15.149:0/139914191 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdeac005da0 con 0x7fdeb0101590 2024-04-24T09:12:16.118 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.115+0000 7fdea67fc700 1 -- 172.21.15.149:0/139914191 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeac00f460 con 0x7fdeb0101590 2024-04-24T09:12:16.118 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.115+0000 7fdeb74e1700 1 -- 172.21.15.149:0/139914191 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdeb01a05a0 con 0x7fdeb0101590 2024-04-24T09:12:16.118 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.115+0000 7fdeb74e1700 1 -- 172.21.15.149:0/139914191 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdeb01a0ab0 con 0x7fdeb0101590 2024-04-24T09:12:16.118 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.115+0000 7fdea67fc700 1 -- 172.21.15.149:0/139914191 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fdeac02a070 con 0x7fdeb0101590 2024-04-24T09:12:16.118 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.116+0000 7fdeb74e1700 1 -- 172.21.15.149:0/139914191 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdeb0064150 con 0x7fdeb0101590 2024-04-24T09:12:16.118 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.116+0000 7fdea67fc700 1 --2- 172.21.15.149:0/139914191 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fde9c039ae0 0x7fde9c03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:16.118 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.116+0000 7fdea67fc700 1 -- 172.21.15.149:0/139914191 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fdeac04da80 con 0x7fdeb0101590 2024-04-24T09:12:16.118 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.116+0000 7fdeb4a7c700 1 --2- 172.21.15.149:0/139914191 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fde9c039ae0 0x7fde9c03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:16.118 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.117+0000 7fdeb4a7c700 1 --2- 172.21.15.149:0/139914191 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fde9c039ae0 0x7fde9c03bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdea0009ad0 tx=0x7fdea0009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:16.120 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.121+0000 7fdea67fc700 1 -- 172.21.15.149:0/139914191 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdeac013070 con 0x7fdeb0101590 2024-04-24T09:12:16.341 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.341+0000 7fdeb74e1700 1 -- 172.21.15.149:0/139914191 --> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi090=x;smithi114=y;smithi149=z", "target": ["mon-mgr", ""]}) -- 0x7fdeb0063080 con 0x7fde9c039ae0 2024-04-24T09:12:16.355 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.356+0000 7fdea67fc700 1 -- 172.21.15.149:0/139914191 <== mgr.14152 v2:172.21.15.90:6800/4261662956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fdeb0063080 con 0x7fde9c039ae0 2024-04-24T09:12:16.356 INFO:teuthology.orchestra.run.smithi149.stdout:Scheduled mgr update... 2024-04-24T09:12:16.358 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.358+0000 7fdeb74e1700 1 -- 172.21.15.149:0/139914191 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fde9c039ae0 msgr2=0x7fde9c03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:16.358 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.358+0000 7fdeb74e1700 1 --2- 172.21.15.149:0/139914191 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fde9c039ae0 0x7fde9c03bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdea0009ad0 tx=0x7fdea0009210 comp rx=0 tx=0).stop 2024-04-24T09:12:16.358 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.358+0000 7fdeb74e1700 1 -- 172.21.15.149:0/139914191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdeb0101590 msgr2=0x7fdeb0197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:16.358 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.358+0000 7fdeb74e1700 1 --2- 172.21.15.149:0/139914191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdeb0101590 0x7fdeb0197f00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fdeac00be40 tx=0x7fdeac00be70 comp rx=0 tx=0).stop 2024-04-24T09:12:16.358 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.358+0000 7fdeb74e1700 1 -- 172.21.15.149:0/139914191 shutdown_connections 2024-04-24T09:12:16.358 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.358+0000 7fdeb74e1700 1 --2- 172.21.15.149:0/139914191 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fde9c039ae0 0x7fde9c03bfa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:16.358 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.358+0000 7fdeb74e1700 1 --2- 172.21.15.149:0/139914191 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdeb01988c0 0x7fdeb019fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:16.358 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.358+0000 7fdeb74e1700 1 --2- 172.21.15.149:0/139914191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdeb0101590 0x7fdeb0197f00 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:16.358 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.358+0000 7fdeb74e1700 1 --2- 172.21.15.149:0/139914191 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdeb0198440 0x7fdeb019d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:16.358 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.358+0000 7fdeb74e1700 1 -- 172.21.15.149:0/139914191 >> 172.21.15.149:0/139914191 conn(0x7fdeb00fcb30 msgr2=0x7fdeb006b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:16.358 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.358+0000 7fdeb74e1700 1 -- 172.21.15.149:0/139914191 shutdown_connections 2024-04-24T09:12:16.358 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:12:16.358+0000 7fdeb74e1700 1 -- 172.21.15.149:0/139914191 wait complete. 2024-04-24T09:12:16.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:16 smithi090 ceph-mon[27825]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:16.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:16 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:16.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:16 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:16.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:16 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T09:12:16.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:16 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T09:12:16.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:16 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:16.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:16 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:16.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:16 smithi149 ceph-mon[32592]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:16.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:16 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:16.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:16 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:16.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:16 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T09:12:16.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:16 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T09:12:16.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:16 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:16.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:16 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:16.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:16 smithi114 ceph-mon[32501]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:16.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:16 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:16.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:16 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:16.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:16 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T09:12:16.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:16 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T09:12:16.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:16 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:16.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:16 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:16.929 DEBUG:teuthology.orchestra.run.smithi114:mgr.y> sudo journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mgr.y.service 2024-04-24T09:12:16.931 DEBUG:teuthology.orchestra.run.smithi149:mgr.z> sudo journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mgr.z.service 2024-04-24T09:12:16.933 INFO:tasks.cephadm:Deploying OSDs... 2024-04-24T09:12:16.934 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-24T09:12:16.934 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T09:12:16.994 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T09:12:16.994 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_1 2024-04-24T09:12:17.048 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T09:12:17.048 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T09:12:17.048 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-04-24T09:12:17.048 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T09:12:17.048 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T09:12:17.048 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-04-24 09:11:14.157376924 +0000 2024-04-24T09:12:17.048 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-04-24 09:05:26.727393555 +0000 2024-04-24T09:12:17.049 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-04-24 09:05:26.727393555 +0000 2024-04-24T09:12:17.049 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-04-24 09:05:26.727393555 +0000 2024-04-24T09:12:17.049 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T09:12:17.126 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-04-24T09:12:17.126 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-04-24T09:12:17.126 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.00038105 s, 1.3 MB/s 2024-04-24T09:12:17.129 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T09:12:17.183 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_2 2024-04-24T09:12:17.232 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T09:12:17.232 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T09:12:17.232 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 6h/6d Inode: 653 Links: 1 2024-04-24T09:12:17.232 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T09:12:17.232 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T09:12:17.232 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-04-24 09:11:14.255375500 +0000 2024-04-24T09:12:17.232 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-04-24 09:05:27.308384862 +0000 2024-04-24T09:12:17.232 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-04-24 09:05:27.308384862 +0000 2024-04-24T09:12:17.232 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-04-24 09:05:27.308384862 +0000 2024-04-24T09:12:17.233 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T09:12:17.312 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-04-24T09:12:17.312 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-04-24T09:12:17.312 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000347438 s, 1.5 MB/s 2024-04-24T09:12:17.315 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T09:12:17.372 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_3 2024-04-24T09:12:17.422 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T09:12:17.422 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T09:12:17.423 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-04-24T09:12:17.423 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T09:12:17.423 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T09:12:17.423 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-04-24 09:11:14.350374119 +0000 2024-04-24T09:12:17.423 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-04-24 09:05:27.886376213 +0000 2024-04-24T09:12:17.423 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-04-24 09:05:27.886376213 +0000 2024-04-24T09:12:17.423 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-04-24 09:05:27.886376213 +0000 2024-04-24T09:12:17.423 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T09:12:17.497 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-04-24T09:12:17.497 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-04-24T09:12:17.497 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000368151 s, 1.4 MB/s 2024-04-24T09:12:17.500 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T09:12:17.555 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_4 2024-04-24T09:12:17.607 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T09:12:17.607 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T09:12:17.607 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-04-24T09:12:17.607 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T09:12:17.607 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T09:12:17.607 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-04-24 09:11:14.441372797 +0000 2024-04-24T09:12:17.607 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-04-24 09:05:28.499367040 +0000 2024-04-24T09:12:17.607 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-04-24 09:05:28.499367040 +0000 2024-04-24T09:12:17.607 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-04-24 09:05:28.499367040 +0000 2024-04-24T09:12:17.608 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T09:12:17.682 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-04-24T09:12:17.682 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-04-24T09:12:17.682 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000475938 s, 1.1 MB/s 2024-04-24T09:12:17.686 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T09:12:17.743 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-04-24T09:12:17.743 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T09:12:17.761 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T09:12:17.761 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_1 2024-04-24T09:12:17.819 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T09:12:17.819 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T09:12:17.819 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-24T09:12:17.820 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T09:12:17.820 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T09:12:17.820 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-04-24 09:12:02.532191322 +0000 2024-04-24T09:12:17.820 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-04-24 09:04:18.572216611 +0000 2024-04-24T09:12:17.820 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-04-24 09:04:18.572216611 +0000 2024-04-24T09:12:17.820 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: 2024-04-24 09:04:18.572216611 +0000 2024-04-24T09:12:17.820 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T09:12:17.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:17 smithi090 ceph-mon[27825]: Reconfiguring mon.c (monmap changed)... 2024-04-24T09:12:17.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:17 smithi090 ceph-mon[27825]: Reconfiguring daemon mon.c on smithi149 2024-04-24T09:12:17.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:17 smithi090 ceph-mon[27825]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi090=x;smithi114=y;smithi149=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:12:17.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:17 smithi090 ceph-mon[27825]: Saving service mgr spec with placement smithi090=x;smithi114=y;smithi149=z;count:3 2024-04-24T09:12:17.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:17 smithi149 ceph-mon[32592]: Reconfiguring mon.c (monmap changed)... 2024-04-24T09:12:17.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:17 smithi149 ceph-mon[32592]: Reconfiguring daemon mon.c on smithi149 2024-04-24T09:12:17.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:17 smithi149 ceph-mon[32592]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi090=x;smithi114=y;smithi149=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:12:17.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:17 smithi149 ceph-mon[32592]: Saving service mgr spec with placement smithi090=x;smithi114=y;smithi149=z;count:3 2024-04-24T09:12:17.883 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:17 smithi114 ceph-mon[32501]: Reconfiguring mon.c (monmap changed)... 2024-04-24T09:12:17.883 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:17 smithi114 ceph-mon[32501]: Reconfiguring daemon mon.c on smithi149 2024-04-24T09:12:17.883 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:17 smithi114 ceph-mon[32501]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi090=x;smithi114=y;smithi149=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:12:17.883 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:17 smithi114 ceph-mon[32501]: Saving service mgr spec with placement smithi090=x;smithi114=y;smithi149=z;count:3 2024-04-24T09:12:17.885 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-04-24T09:12:17.885 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-04-24T09:12:17.885 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.00020097 s, 2.5 MB/s 2024-04-24T09:12:17.886 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T09:12:17.944 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_2 2024-04-24T09:12:18.001 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T09:12:18.001 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T09:12:18.001 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-24T09:12:18.001 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T09:12:18.001 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T09:12:18.001 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-04-24 09:12:02.532191322 +0000 2024-04-24T09:12:18.001 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-04-24 09:04:18.908209593 +0000 2024-04-24T09:12:18.001 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-04-24 09:04:18.908209593 +0000 2024-04-24T09:12:18.001 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: 2024-04-24 09:04:18.908209593 +0000 2024-04-24T09:12:18.002 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T09:12:18.067 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-04-24T09:12:18.067 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-04-24T09:12:18.067 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000206899 s, 2.5 MB/s 2024-04-24T09:12:18.068 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T09:12:18.125 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_3 2024-04-24T09:12:18.181 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T09:12:18.181 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T09:12:18.181 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-24T09:12:18.181 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T09:12:18.181 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T09:12:18.181 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-04-24 09:12:02.532191322 +0000 2024-04-24T09:12:18.182 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-04-24 09:04:19.244202574 +0000 2024-04-24T09:12:18.182 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-04-24 09:04:19.244202574 +0000 2024-04-24T09:12:18.182 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: 2024-04-24 09:04:19.244202574 +0000 2024-04-24T09:12:18.182 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T09:12:18.245 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-04-24T09:12:18.245 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-04-24T09:12:18.245 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000181452 s, 2.8 MB/s 2024-04-24T09:12:18.247 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T09:12:18.302 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_4 2024-04-24T09:12:18.358 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T09:12:18.358 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T09:12:18.358 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-04-24T09:12:18.358 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T09:12:18.358 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T09:12:18.358 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-04-24 09:12:02.533191301 +0000 2024-04-24T09:12:18.358 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-04-24 09:04:19.558196014 +0000 2024-04-24T09:12:18.358 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-04-24 09:04:19.558196014 +0000 2024-04-24T09:12:18.358 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: 2024-04-24 09:04:19.558196014 +0000 2024-04-24T09:12:18.359 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T09:12:18.421 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-04-24T09:12:18.421 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-04-24T09:12:18.421 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000198066 s, 2.6 MB/s 2024-04-24T09:12:18.422 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T09:12:18.478 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-04-24T09:12:18.478 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T09:12:18.494 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T09:12:18.494 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_1 2024-04-24T09:12:18.551 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T09:12:18.551 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T09:12:18.551 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-24T09:12:18.551 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T09:12:18.551 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T09:12:18.551 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-04-24 09:11:52.529918080 +0000 2024-04-24T09:12:18.551 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-04-24 09:04:41.231837336 +0000 2024-04-24T09:12:18.551 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-04-24 09:04:41.231837336 +0000 2024-04-24T09:12:18.551 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-04-24 09:04:41.231837336 +0000 2024-04-24T09:12:18.551 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T09:12:18.616 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-04-24T09:12:18.616 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-04-24T09:12:18.617 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000304275 s, 1.7 MB/s 2024-04-24T09:12:18.617 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T09:12:18.674 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_2 2024-04-24T09:12:18.730 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T09:12:18.730 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T09:12:18.730 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-24T09:12:18.730 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T09:12:18.730 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T09:12:18.730 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-04-24 09:11:52.530918060 +0000 2024-04-24T09:12:18.730 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-04-24 09:04:41.569830411 +0000 2024-04-24T09:12:18.730 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-04-24 09:04:41.569830411 +0000 2024-04-24T09:12:18.730 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-04-24 09:04:41.569830411 +0000 2024-04-24T09:12:18.731 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T09:12:18.792 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-04-24T09:12:18.792 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-04-24T09:12:18.792 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000224546 s, 2.3 MB/s 2024-04-24T09:12:18.793 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T09:12:18.848 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_3 2024-04-24T09:12:18.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:18 smithi149 ceph-mon[32592]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:18.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:18 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:18.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:18 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:18.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:18 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:12:18.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:18 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:18.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:18 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:12:18.863 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:18 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:18.863 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:18 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T09:12:18.863 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:18 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T09:12:18.863 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:18 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T09:12:18.863 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:18 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:18.878 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T09:12:18.878 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T09:12:18.879 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-24T09:12:18.879 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T09:12:18.879 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T09:12:18.879 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-04-24 09:11:52.530918060 +0000 2024-04-24T09:12:18.879 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-04-24 09:04:41.906823505 +0000 2024-04-24T09:12:18.880 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-04-24 09:04:41.906823505 +0000 2024-04-24T09:12:18.880 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-04-24 09:04:41.906823505 +0000 2024-04-24T09:12:18.880 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T09:12:18.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:18 smithi114 ceph-mon[32501]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:18.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:18 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:18.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:18 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:18.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:18 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:12:18.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:18 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:18.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:18 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:12:18.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:18 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:18.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:18 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T09:12:18.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:18 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T09:12:18.928 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:18 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T09:12:18.928 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:18 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:18.941 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-04-24T09:12:18.941 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-04-24T09:12:18.941 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000213689 s, 2.4 MB/s 2024-04-24T09:12:18.943 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T09:12:19.003 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_4 2024-04-24T09:12:19.058 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T09:12:19.058 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T09:12:19.058 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-24T09:12:19.058 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T09:12:19.058 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T09:12:19.058 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-04-24 09:11:52.530918060 +0000 2024-04-24T09:12:19.058 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-04-24 09:04:42.233816804 +0000 2024-04-24T09:12:19.058 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-04-24 09:04:42.233816804 +0000 2024-04-24T09:12:19.059 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-04-24 09:04:42.233816804 +0000 2024-04-24T09:12:19.059 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T09:12:19.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:18 smithi090 ceph-mon[27825]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:19.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:18 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:19.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:18 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:19.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:18 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:12:19.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:18 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:19.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:18 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:12:19.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:18 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:19.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:18 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T09:12:19.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:18 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T09:12:19.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:18 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T09:12:19.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:18 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:19.120 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-04-24T09:12:19.120 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-04-24T09:12:19.120 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000175034 s, 2.9 MB/s 2024-04-24T09:12:19.121 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T09:12:19.177 INFO:tasks.cephadm:Deploying osd.0 on smithi090 with /dev/vg_nvme/lv_4... 2024-04-24T09:12:19.177 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-24T09:12:19.608 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:12:19.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:19 smithi149 ceph-mon[32592]: Deploying daemon mgr.z on smithi149 2024-04-24T09:12:19.914 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:19 smithi090 ceph-mon[27825]: Deploying daemon mgr.z on smithi149 2024-04-24T09:12:19.925 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:19 smithi114 ceph-mon[32501]: Deploying daemon mgr.z on smithi149 2024-04-24T09:12:20.270 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Apr 24 09:12:20 smithi149 systemd[1]: Starting Ceph mgr.z for 30ee9be2-021a-11ef-bc93-c7b262605968... 2024-04-24T09:12:20.546 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Apr 24 09:12:20 smithi149 podman[34442]: 2024-04-24 09:12:20.270475703 +0000 UTC m=+0.103290424 container create b35998b28eee3bd3be6e9b0c5aaa91998a64ef7de34f1af236eeb3a9f4744a82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-24T09:12:20.547 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Apr 24 09:12:20 smithi149 podman[34442]: 2024-04-24 09:12:20.185275729 +0000 UTC m=+0.018090411 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:12:20.547 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Apr 24 09:12:20 smithi149 podman[34442]: 2024-04-24 09:12:20.472648722 +0000 UTC m=+0.305463395 container init b35998b28eee3bd3be6e9b0c5aaa91998a64ef7de34f1af236eeb3a9f4744a82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-z, org.label-schema.build-date=20240424, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-24T09:12:20.547 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Apr 24 09:12:20 smithi149 podman[34442]: 2024-04-24 09:12:20.475570851 +0000 UTC m=+0.308385524 container start b35998b28eee3bd3be6e9b0c5aaa91998a64ef7de34f1af236eeb3a9f4744a82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-24T09:12:20.547 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Apr 24 09:12:20 smithi149 ceph-mgr[34461]: -- 172.21.15.149:0/2351856962 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563e712685a0 con 0x563e7123cc00 2024-04-24T09:12:20.828 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:20 smithi149 ceph-mon[32592]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:20.829 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Apr 24 09:12:20 smithi149 bash[34442]: b35998b28eee3bd3be6e9b0c5aaa91998a64ef7de34f1af236eeb3a9f4744a82 2024-04-24T09:12:20.829 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Apr 24 09:12:20 smithi149 systemd[1]: Started Ceph mgr.z for 30ee9be2-021a-11ef-bc93-c7b262605968. 2024-04-24T09:12:20.829 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Apr 24 09:12:20 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-z[34457]: 2024-04-24T09:12:20.647+0000 7fb963346200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T09:12:20.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:20 smithi090 ceph-mon[27825]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:20.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:20 smithi114 ceph-mon[32501]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:21.111 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Apr 24 09:12:20 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-z[34457]: 2024-04-24T09:12:20.835+0000 7fb963346200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T09:12:21.111 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Apr 24 09:12:20 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-z[34457]: 2024-04-24T09:12:20.938+0000 7fb963346200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T09:12:21.611 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Apr 24 09:12:21 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-z[34457]: 2024-04-24T09:12:21.246+0000 7fb963346200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T09:12:21.611 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Apr 24 09:12:21 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-z[34457]: 2024-04-24T09:12:21.330+0000 7fb963346200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T09:12:21.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:21 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:21.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:21 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:21.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:21 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:21.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:21 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T09:12:21.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:21 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T09:12:21.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:21 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T09:12:21.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:21 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:21.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:21 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:21.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:21 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:21.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:21 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:21.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:21 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T09:12:21.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:21 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T09:12:21.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:21 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T09:12:21.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:21 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:22.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:21 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:22.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:21 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:22.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:21 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:22.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:21 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T09:12:22.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:21 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T09:12:22.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:21 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T09:12:22.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:21 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:22.442 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-24T09:12:22.443 INFO:teuthology.orchestra.run.smithi090.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-24T09:12:22.443 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10+0 records in 2024-04-24T09:12:22.443 INFO:teuthology.orchestra.run.smithi090.stderr:10+0 records out 2024-04-24T09:12:22.443 INFO:teuthology.orchestra.run.smithi090.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0349493 s, 300 MB/s 2024-04-24T09:12:22.443 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 2024-04-24T09:12:22.443 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping successful for: 2024-04-24T09:12:22.665 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:22 smithi149 ceph-mon[32592]: Deploying daemon mgr.y on smithi114 2024-04-24T09:12:22.665 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:22 smithi149 ceph-mon[32592]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:22.666 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Apr 24 09:12:22 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-z[34457]: 2024-04-24T09:12:22.554+0000 7fb963346200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T09:12:22.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:22 smithi090 ceph-mon[27825]: Deploying daemon mgr.y on smithi114 2024-04-24T09:12:22.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:22 smithi090 ceph-mon[27825]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:22.863 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:22 smithi114 ceph-mon[32501]: Deploying daemon mgr.y on smithi114 2024-04-24T09:12:22.863 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:22 smithi114 ceph-mon[32501]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:22.863 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Apr 24 09:12:22 smithi114 systemd[1]: Starting Ceph mgr.y for 30ee9be2-021a-11ef-bc93-c7b262605968... 2024-04-24T09:12:22.918 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Apr 24 09:12:22 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-z[34457]: 2024-04-24T09:12:22.732+0000 7fb963346200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T09:12:22.918 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Apr 24 09:12:22 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-z[34457]: 2024-04-24T09:12:22.918+0000 7fb963346200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T09:12:23.159 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Apr 24 09:12:22 smithi114 podman[33874]: 2024-04-24 09:12:22.864091178 +0000 UTC m=+0.106275539 container create 5b4d3ed09dc7e77b99b254d6260ce6b0dc14eef6a46dc05c9cbb9ede36bc6b08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-y, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-24T09:12:23.159 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Apr 24 09:12:22 smithi114 podman[33874]: 2024-04-24 09:12:22.775449224 +0000 UTC m=+0.017633597 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:12:23.160 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Apr 24 09:12:23 smithi114 podman[33874]: 2024-04-24 09:12:23.060002006 +0000 UTC m=+0.302186361 container init 5b4d3ed09dc7e77b99b254d6260ce6b0dc14eef6a46dc05c9cbb9ede36bc6b08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T09:12:23.160 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Apr 24 09:12:23 smithi114 podman[33874]: 2024-04-24 09:12:23.062340138 +0000 UTC m=+0.304524493 container start 5b4d3ed09dc7e77b99b254d6260ce6b0dc14eef6a46dc05c9cbb9ede36bc6b08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-y, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T09:12:23.160 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Apr 24 09:12:23 smithi114 ceph-mgr[33893]: -- 172.21.15.114:0/594472966 <== mon.2 v2:172.21.15.114:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55727d3405a0 con 0x55727d314800 2024-04-24T09:12:23.160 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Apr 24 09:12:23 smithi114 bash[33874]: 5b4d3ed09dc7e77b99b254d6260ce6b0dc14eef6a46dc05c9cbb9ede36bc6b08 2024-04-24T09:12:23.270 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi090:vg_nvme/lv_4 2024-04-24T09:12:23.426 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Apr 24 09:12:23 smithi114 systemd[1]: Started Ceph mgr.y for 30ee9be2-021a-11ef-bc93-c7b262605968. 2024-04-24T09:12:23.426 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Apr 24 09:12:23 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-y[33889]: 2024-04-24T09:12:23.212+0000 7f492fcce200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T09:12:23.426 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Apr 24 09:12:23 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-y[33889]: 2024-04-24T09:12:23.398+0000 7f492fcce200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T09:12:23.528 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Apr 24 09:12:23 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-z[34457]: 2024-04-24T09:12:23.217+0000 7fb963346200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T09:12:23.528 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Apr 24 09:12:23 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-z[34457]: 2024-04-24T09:12:23.313+0000 7fb963346200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T09:12:23.528 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Apr 24 09:12:23 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-z[34457]: 2024-04-24T09:12:23.528+0000 7fb963346200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T09:12:23.697 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:12:23.828 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Apr 24 09:12:23 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-y[33889]: 2024-04-24T09:12:23.504+0000 7f492fcce200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T09:12:23.828 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Apr 24 09:12:23 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-y[33889]: 2024-04-24T09:12:23.828+0000 7f492fcce200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T09:12:23.861 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Apr 24 09:12:23 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-z[34457]: 2024-04-24T09:12:23.734+0000 7fb963346200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T09:12:24.176 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Apr 24 09:12:23 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-y[33889]: 2024-04-24T09:12:23.916+0000 7f492fcce200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T09:12:24.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:24 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:24.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:24 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:24.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:24 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:24.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:24 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:24.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:24 smithi090 ceph-mon[27825]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:24.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:24 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:12:24.611 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Apr 24 09:12:24 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-z[34457]: 2024-04-24T09:12:24.381+0000 7fb963346200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T09:12:24.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:24 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:24.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:24 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:24.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:24 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:24.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:24 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:24.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:24 smithi149 ceph-mon[32592]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:24.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:24 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:12:24.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:24 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:24.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:24 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:24.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:24 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:24.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:24 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:24.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:24 smithi114 ceph-mon[32501]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:24.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:24 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:12:25.111 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Apr 24 09:12:24 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-z[34457]: 2024-04-24T09:12:24.823+0000 7fb963346200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T09:12:25.111 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Apr 24 09:12:24 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-z[34457]: 2024-04-24T09:12:24.918+0000 7fb963346200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T09:12:25.426 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Apr 24 09:12:25 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-y[33889]: 2024-04-24T09:12:25.154+0000 7f492fcce200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T09:12:25.426 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Apr 24 09:12:25 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-y[33889]: 2024-04-24T09:12:25.323+0000 7f492fcce200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T09:12:25.611 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Apr 24 09:12:25 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-z[34457]: 2024-04-24T09:12:25.122+0000 7fb963346200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T09:12:25.762 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Apr 24 09:12:25 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-y[33889]: 2024-04-24T09:12:25.492+0000 7f492fcce200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T09:12:25.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:25.866+0000 7f6106400700 1 -- 172.21.15.90:0/114918096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6100109190 msgr2=0x7f61001095b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:25.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:25.866+0000 7f6106400700 1 --2- 172.21.15.90:0/114918096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6100109190 0x7f61001095b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f60f00099c0 tx=0x7f60f0009cd0 comp rx=0 tx=0).stop 2024-04-24T09:12:25.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:25.866+0000 7f6106400700 1 -- 172.21.15.90:0/114918096 shutdown_connections 2024-04-24T09:12:25.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:25.866+0000 7f6106400700 1 --2- 172.21.15.90:0/114918096 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f610010ad30 0x7f61001115e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:25.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:25.866+0000 7f6106400700 1 --2- 172.21.15.90:0/114918096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6100109190 0x7f61001095b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:25.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:25.866+0000 7f6106400700 1 --2- 172.21.15.90:0/114918096 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f610010a390 0x7f610010a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:25.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:25.866+0000 7f6106400700 1 -- 172.21.15.90:0/114918096 >> 172.21.15.90:0/114918096 conn(0x7f6100078160 msgr2=0x7f610007a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:25.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:25.866+0000 7f6106400700 1 -- 172.21.15.90:0/114918096 shutdown_connections 2024-04-24T09:12:25.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:25.866+0000 7f6106400700 1 -- 172.21.15.90:0/114918096 wait complete. 2024-04-24T09:12:25.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:25.867+0000 7f6106400700 1 Processor -- start 2024-04-24T09:12:25.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:25.867+0000 7f6106400700 1 -- start start 2024-04-24T09:12:25.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:25.868+0000 7f6106400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6100109190 0x7f610019f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:25.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:25.868+0000 7f6106400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f610010a390 0x7f610019f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:25.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:25.868+0000 7f6106400700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f610010ad30 0x7f61001a6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:25.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:25.868+0000 7f6106400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6100114dc0 con 0x7f610010a390 2024-04-24T09:12:25.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:25.868+0000 7f6106400700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6100114c40 con 0x7f610010ad30 2024-04-24T09:12:25.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:25.868+0000 7f6106400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6100114f40 con 0x7f6100109190 2024-04-24T09:12:25.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:25.868+0000 7f60ff400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6100109190 0x7f610019f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:25.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:25.868+0000 7f60ffe00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f610010ad30 0x7f61001a6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:25.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:25.868+0000 7f60ff400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6100109190 0x7f610019f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:59690/0 (socket says 172.21.15.90:59690) 2024-04-24T09:12:25.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:25.868+0000 7f60ff400700 1 -- 172.21.15.90:0/806692043 learned_addr learned my addr 172.21.15.90:0/806692043 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:12:25.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:25.868+0000 7f60fea00700 1 --2- 172.21.15.90:0/806692043 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f610010a390 0x7f610019f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:25.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:25.868+0000 7f60ff400700 1 -- 172.21.15.90:0/806692043 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f610010ad30 msgr2=0x7f61001a6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:25.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:25.868+0000 7f60ff400700 1 --2- 172.21.15.90:0/806692043 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f610010ad30 0x7f61001a6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:25.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:25.868+0000 7f60ff400700 1 -- 172.21.15.90:0/806692043 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f610010a390 msgr2=0x7f610019f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:25.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:25.868+0000 7f60ff400700 1 --2- 172.21.15.90:0/806692043 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f610010a390 0x7f610019f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:25.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:25.868+0000 7f60ff400700 1 -- 172.21.15.90:0/806692043 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61001a7000 con 0x7f6100109190 2024-04-24T09:12:25.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:25.869+0000 7f60ff400700 1 --2- 172.21.15.90:0/806692043 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6100109190 0x7f610019f3a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f60f000bee0 tx=0x7f60f000bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:25.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:25.869+0000 7f60f7e00700 1 -- 172.21.15.90:0/806692043 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f0009e60 con 0x7f6100109190 2024-04-24T09:12:25.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:25.869+0000 7f60f7e00700 1 -- 172.21.15.90:0/806692043 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60f0003e30 con 0x7f6100109190 2024-04-24T09:12:25.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:25.869+0000 7f6106400700 1 -- 172.21.15.90:0/806692043 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61001a7290 con 0x7f6100109190 2024-04-24T09:12:25.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:25.869+0000 7f60f7e00700 1 -- 172.21.15.90:0/806692043 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f00173f0 con 0x7f6100109190 2024-04-24T09:12:25.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:25.869+0000 7f6106400700 1 -- 172.21.15.90:0/806692043 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61001a76e0 con 0x7f6100109190 2024-04-24T09:12:25.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:25.870+0000 7f60f7e00700 1 -- 172.21.15.90:0/806692043 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f60f0017590 con 0x7f6100109190 2024-04-24T09:12:25.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:25.870+0000 7f6106400700 1 -- 172.21.15.90:0/806692043 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61000697e0 con 0x7f6100109190 2024-04-24T09:12:25.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:25.870+0000 7f60f7e00700 1 --2- 172.21.15.90:0/806692043 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f60e4039890 0x7f60e403bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:25.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:25.870+0000 7f60f7e00700 1 -- 172.21.15.90:0/806692043 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f60f004d8a0 con 0x7f6100109190 2024-04-24T09:12:25.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:25.871+0000 7f60fea00700 1 --2- 172.21.15.90:0/806692043 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f60e4039890 0x7f60e403bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:25.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:25.871+0000 7f60fea00700 1 --2- 172.21.15.90:0/806692043 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f60e4039890 0x7f60e403bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f60e8009c60 tx=0x7f60e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:25.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:25.876+0000 7f60f7e00700 1 -- 172.21.15.90:0/806692043 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f60f0013070 con 0x7f6100109190 2024-04-24T09:12:25.983 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:25 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:25.983 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:25 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:25.983 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:25 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:25.983 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:25 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:25.983 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:25 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:25.983 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:25 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:12:25.983 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:25 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:25.983 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:25 smithi090 ceph-mon[27825]: Reconfiguring mgr.x (unknown last config time)... 2024-04-24T09:12:25.984 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:25 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T09:12:25.984 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:25 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T09:12:25.984 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:25 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:25.984 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:25 smithi090 ceph-mon[27825]: Reconfiguring daemon mgr.x on smithi090 2024-04-24T09:12:25.984 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:25 smithi090 ceph-mon[27825]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:25.987 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Apr 24 09:12:25 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-z[34457]: 2024-04-24T09:12:25.654+0000 7fb963346200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T09:12:25.987 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Apr 24 09:12:25 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-z[34457]: 2024-04-24T09:12:25.744+0000 7fb963346200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T09:12:25.987 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:25 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:25.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:25 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:25.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:25 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:25.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:25 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:25.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:25 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:25.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:25 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:12:25.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:25 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:25.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:25 smithi149 ceph-mon[32592]: Reconfiguring mgr.x (unknown last config time)... 2024-04-24T09:12:25.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:25 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T09:12:25.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:25 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T09:12:25.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:25 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:25.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:25 smithi149 ceph-mon[32592]: Reconfiguring daemon mgr.x on smithi090 2024-04-24T09:12:25.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:25 smithi149 ceph-mon[32592]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:26.073 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Apr 24 09:12:25 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-y[33889]: 2024-04-24T09:12:25.780+0000 7f492fcce200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T09:12:26.073 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Apr 24 09:12:25 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-y[33889]: 2024-04-24T09:12:25.873+0000 7f492fcce200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T09:12:26.073 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Apr 24 09:12:26 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-y[33889]: 2024-04-24T09:12:26.073+0000 7f492fcce200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T09:12:26.074 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:25 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:26.074 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:25 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:26.074 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:25 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:26.074 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:25 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:26.074 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:25 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:26.074 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:25 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:12:26.074 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:25 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:26.074 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:25 smithi114 ceph-mon[32501]: Reconfiguring mgr.x (unknown last config time)... 2024-04-24T09:12:26.074 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:25 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T09:12:26.074 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:25 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T09:12:26.074 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:25 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:26.074 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:25 smithi114 ceph-mon[32501]: Reconfiguring daemon mgr.x on smithi090 2024-04-24T09:12:26.074 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:25 smithi114 ceph-mon[32501]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:26.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:26.115+0000 7f6106400700 1 -- 172.21.15.90:0/806692043 --> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f6100062f30 con 0x7f60e4039890 2024-04-24T09:12:26.329 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Apr 24 09:12:25 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-z[34457]: 2024-04-24T09:12:25.988+0000 7fb963346200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T09:12:26.329 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Apr 24 09:12:26 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-z[34457]: 2024-04-24T09:12:26.120+0000 7fb963346200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T09:12:26.329 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Apr 24 09:12:26 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-z[34457]: 2024-04-24T09:12:26.215+0000 7fb963346200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T09:12:26.329 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Apr 24 09:12:26 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-z[34457]: 2024-04-24T09:12:26.329+0000 7fb963346200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T09:12:26.426 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Apr 24 09:12:26 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-y[33889]: 2024-04-24T09:12:26.264+0000 7f492fcce200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T09:12:27.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:26 smithi090 ceph-mon[27825]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:12:27.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:26 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:12:27.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:26 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:12:27.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:26 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:27.111 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Apr 24 09:12:26 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-z[34457]: 2024-04-24T09:12:26.639+0000 7fb963346200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T09:12:27.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:26 smithi149 ceph-mon[32592]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:12:27.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:26 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:12:27.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:26 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:12:27.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:26 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:27.176 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Apr 24 09:12:26 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-y[33889]: 2024-04-24T09:12:26.855+0000 7f492fcce200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T09:12:27.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:26 smithi114 ceph-mon[32501]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:12:27.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:26 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:12:27.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:26 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:12:27.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:26 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:27.567 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Apr 24 09:12:27 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-y[33889]: 2024-04-24T09:12:27.271+0000 7f492fcce200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T09:12:27.567 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Apr 24 09:12:27 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-y[33889]: 2024-04-24T09:12:27.366+0000 7f492fcce200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T09:12:27.567 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Apr 24 09:12:27 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-y[33889]: 2024-04-24T09:12:27.567+0000 7f492fcce200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T09:12:27.611 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Apr 24 09:12:27 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-z[34457]: 2024-04-24T09:12:27.237+0000 7fb963346200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T09:12:27.611 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Apr 24 09:12:27 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-z[34457]: 2024-04-24T09:12:27.453+0000 7fb963346200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T09:12:28.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:28.357+0000 7f60f7e00700 1 -- 172.21.15.90:0/806692043 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7f60f002b440 con 0x7f6100109190 2024-04-24T09:12:28.421 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Apr 24 09:12:28 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-y[33889]: 2024-04-24T09:12:28.089+0000 7f492fcce200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T09:12:28.422 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Apr 24 09:12:28 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-y[33889]: 2024-04-24T09:12:28.180+0000 7f492fcce200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T09:12:28.422 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:28 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:28.422 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:28 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:28.422 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:28 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:12:28.422 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:28 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:28.422 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:28 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:12:28.422 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:28 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:28.422 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:28 smithi114 ceph-mon[32501]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:28.422 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:28 smithi114 ceph-mon[32501]: from='mgr.? 172.21.15.149:0/2040469193' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-24T09:12:28.422 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:28 smithi114 ceph-mon[32501]: Standby manager daemon z started 2024-04-24T09:12:28.422 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:28 smithi114 ceph-mon[32501]: from='mgr.? 172.21.15.149:0/2040469193' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T09:12:28.423 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:28 smithi114 ceph-mon[32501]: from='mgr.? 172.21.15.149:0/2040469193' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-24T09:12:28.423 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:28 smithi114 ceph-mon[32501]: from='mgr.? 172.21.15.149:0/2040469193' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T09:12:28.423 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:28 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:28.491 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:28 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:28.492 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:28 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:28.492 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:28 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:12:28.492 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:28 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:28.492 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:28 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:12:28.492 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:28 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:28.492 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:28 smithi090 ceph-mon[27825]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:28.492 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:28 smithi090 ceph-mon[27825]: from='mgr.? 172.21.15.149:0/2040469193' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-24T09:12:28.492 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:28 smithi090 ceph-mon[27825]: Standby manager daemon z started 2024-04-24T09:12:28.492 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:28 smithi090 ceph-mon[27825]: from='mgr.? 172.21.15.149:0/2040469193' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T09:12:28.492 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:28 smithi090 ceph-mon[27825]: from='mgr.? 172.21.15.149:0/2040469193' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-24T09:12:28.492 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:28 smithi090 ceph-mon[27825]: from='mgr.? 172.21.15.149:0/2040469193' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T09:12:28.493 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:28 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:28.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:28 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:28.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:28 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:28.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:28 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:12:28.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:28 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:28.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:28 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:12:28.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:28 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:28.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:28 smithi149 ceph-mon[32592]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:28.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:28 smithi149 ceph-mon[32592]: from='mgr.? 172.21.15.149:0/2040469193' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-24T09:12:28.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:28 smithi149 ceph-mon[32592]: Standby manager daemon z started 2024-04-24T09:12:28.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:28 smithi149 ceph-mon[32592]: from='mgr.? 172.21.15.149:0/2040469193' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T09:12:28.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:28 smithi149 ceph-mon[32592]: from='mgr.? 172.21.15.149:0/2040469193' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-24T09:12:28.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:28 smithi149 ceph-mon[32592]: from='mgr.? 172.21.15.149:0/2040469193' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T09:12:28.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:28 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:28.676 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Apr 24 09:12:28 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-y[33889]: 2024-04-24T09:12:28.421+0000 7f492fcce200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T09:12:28.676 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Apr 24 09:12:28 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-y[33889]: 2024-04-24T09:12:28.553+0000 7f492fcce200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T09:12:28.676 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Apr 24 09:12:28 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-y[33889]: 2024-04-24T09:12:28.646+0000 7f492fcce200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T09:12:29.069 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Apr 24 09:12:28 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-y[33889]: 2024-04-24T09:12:28.761+0000 7f492fcce200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T09:12:29.069 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Apr 24 09:12:29 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-y[33889]: 2024-04-24T09:12:29.069+0000 7f492fcce200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T09:12:29.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:29 smithi090 ceph-mon[27825]: mgrmap e14: x(active, since 90s), standbys: z 2024-04-24T09:12:29.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:29 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-24T09:12:29.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:29 smithi149 ceph-mon[32592]: mgrmap e14: x(active, since 90s), standbys: z 2024-04-24T09:12:29.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:29 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-24T09:12:29.675 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:29 smithi114 ceph-mon[32501]: mgrmap e14: x(active, since 90s), standbys: z 2024-04-24T09:12:29.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:29 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-24T09:12:30.176 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Apr 24 09:12:29 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-y[33889]: 2024-04-24T09:12:29.697+0000 7f492fcce200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T09:12:30.176 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Apr 24 09:12:29 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-y[33889]: 2024-04-24T09:12:29.920+0000 7f492fcce200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T09:12:30.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:30.376+0000 7f60f7e00700 1 -- 172.21.15.90:0/806692043 <== mon.1 v2:172.21.15.149:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f60f0017c50 con 0x7f6100109190 2024-04-24T09:12:30.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:30 smithi114 ceph-mon[32501]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:30.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:30 smithi114 ceph-mon[32501]: from='mgr.? 172.21.15.114:0/976275735' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-24T09:12:30.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:30 smithi114 ceph-mon[32501]: Standby manager daemon y started 2024-04-24T09:12:30.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:30 smithi114 ceph-mon[32501]: from='mgr.? 172.21.15.114:0/976275735' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T09:12:30.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:30 smithi114 ceph-mon[32501]: from='mgr.? 172.21.15.114:0/976275735' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-24T09:12:30.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:30 smithi114 ceph-mon[32501]: from='mgr.? 172.21.15.114:0/976275735' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T09:12:30.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:30 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/798069974' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13a59b9f-b46e-4ec4-90af-bb7a3d5c753b"}]: dispatch 2024-04-24T09:12:30.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:30 smithi114 ceph-mon[32501]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13a59b9f-b46e-4ec4-90af-bb7a3d5c753b"}]: dispatch 2024-04-24T09:12:30.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:30 smithi114 ceph-mon[32501]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "13a59b9f-b46e-4ec4-90af-bb7a3d5c753b"}]': finished 2024-04-24T09:12:30.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:30 smithi114 ceph-mon[32501]: osdmap e5: 1 total, 0 up, 1 in 2024-04-24T09:12:30.677 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:30 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T09:12:30.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:30 smithi090 ceph-mon[27825]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:30.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:30 smithi090 ceph-mon[27825]: from='mgr.? 172.21.15.114:0/976275735' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-24T09:12:30.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:30 smithi090 ceph-mon[27825]: Standby manager daemon y started 2024-04-24T09:12:30.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:30 smithi090 ceph-mon[27825]: from='mgr.? 172.21.15.114:0/976275735' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T09:12:30.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:30 smithi090 ceph-mon[27825]: from='mgr.? 172.21.15.114:0/976275735' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-24T09:12:30.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:30 smithi090 ceph-mon[27825]: from='mgr.? 172.21.15.114:0/976275735' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T09:12:30.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:30 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/798069974' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13a59b9f-b46e-4ec4-90af-bb7a3d5c753b"}]: dispatch 2024-04-24T09:12:30.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:30 smithi090 ceph-mon[27825]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13a59b9f-b46e-4ec4-90af-bb7a3d5c753b"}]: dispatch 2024-04-24T09:12:30.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:30 smithi090 ceph-mon[27825]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "13a59b9f-b46e-4ec4-90af-bb7a3d5c753b"}]': finished 2024-04-24T09:12:30.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:30 smithi090 ceph-mon[27825]: osdmap e5: 1 total, 0 up, 1 in 2024-04-24T09:12:30.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:30 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T09:12:30.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:30 smithi149 ceph-mon[32592]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:30.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:30 smithi149 ceph-mon[32592]: from='mgr.? 172.21.15.114:0/976275735' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-24T09:12:30.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:30 smithi149 ceph-mon[32592]: Standby manager daemon y started 2024-04-24T09:12:30.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:30 smithi149 ceph-mon[32592]: from='mgr.? 172.21.15.114:0/976275735' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T09:12:30.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:30 smithi149 ceph-mon[32592]: from='mgr.? 172.21.15.114:0/976275735' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-24T09:12:30.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:30 smithi149 ceph-mon[32592]: from='mgr.? 172.21.15.114:0/976275735' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T09:12:30.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:30 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/798069974' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13a59b9f-b46e-4ec4-90af-bb7a3d5c753b"}]: dispatch 2024-04-24T09:12:30.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:30 smithi149 ceph-mon[32592]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13a59b9f-b46e-4ec4-90af-bb7a3d5c753b"}]: dispatch 2024-04-24T09:12:30.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:30 smithi149 ceph-mon[32592]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "13a59b9f-b46e-4ec4-90af-bb7a3d5c753b"}]': finished 2024-04-24T09:12:30.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:30 smithi149 ceph-mon[32592]: osdmap e5: 1 total, 0 up, 1 in 2024-04-24T09:12:30.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:30 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T09:12:31.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:31 smithi114 ceph-mon[32501]: mgrmap e15: x(active, since 92s), standbys: z, y 2024-04-24T09:12:31.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:31 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-24T09:12:31.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:31 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/1458347026' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:12:31.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:31 smithi090 ceph-mon[27825]: mgrmap e15: x(active, since 92s), standbys: z, y 2024-04-24T09:12:31.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:31 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-24T09:12:31.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:31 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/1458347026' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:12:31.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:31 smithi149 ceph-mon[32592]: mgrmap e15: x(active, since 92s), standbys: z, y 2024-04-24T09:12:31.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:31 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-24T09:12:31.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:31 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/1458347026' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:12:32.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:32 smithi114 ceph-mon[32501]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:32.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:32 smithi090 ceph-mon[27825]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:32.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:32 smithi149 ceph-mon[32592]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:34.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:34 smithi090 ceph-mon[27825]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:34.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:34 smithi149 ceph-mon[32592]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:34.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:34 smithi114 ceph-mon[32501]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:36.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:36 smithi090 ceph-mon[27825]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:36.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:36 smithi149 ceph-mon[32592]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:36.925 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:36 smithi114 ceph-mon[32501]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:38.794 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:38 smithi090 ceph-mon[27825]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:38.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:38 smithi149 ceph-mon[32592]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:38.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:38 smithi114 ceph-mon[32501]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:39.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:39 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-24T09:12:39.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:39 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:39.771 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:39 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-24T09:12:39.771 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:39 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:39.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:39 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-24T09:12:39.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:39 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:40.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:40 smithi090 ceph-mon[27825]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:40.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:40 smithi090 ceph-mon[27825]: Deploying daemon osd.0 on smithi090 2024-04-24T09:12:40.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:40 smithi114 ceph-mon[32501]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:40.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:40 smithi114 ceph-mon[32501]: Deploying daemon osd.0 on smithi090 2024-04-24T09:12:40.763 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:40 smithi149 ceph-mon[32592]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:40.763 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:40 smithi149 ceph-mon[32592]: Deploying daemon osd.0 on smithi090 2024-04-24T09:12:42.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:42 smithi090 ceph-mon[27825]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:42.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:42 smithi149 ceph-mon[32592]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:42.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:42 smithi114 ceph-mon[32501]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:44.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:44 smithi090 ceph-mon[27825]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:44.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:44 smithi149 ceph-mon[32592]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:44.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:44 smithi114 ceph-mon[32501]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:46.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:46 smithi090 ceph-mon[27825]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:46.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:46 smithi149 ceph-mon[32592]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:46.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:46 smithi114 ceph-mon[32501]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:48.851 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:48 smithi090 ceph-mon[27825]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:48.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:48 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:48.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:48 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:48.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:48 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:12:48.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:48 smithi149 ceph-mon[32592]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:48.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:48 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:48.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:48 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:48.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:48 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:12:48.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:48 smithi114 ceph-mon[32501]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:48.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:48 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:48.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:48 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:48.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:48 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:12:50.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:50 smithi090 ceph-mon[27825]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:50.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:50 smithi149 ceph-mon[32592]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:50.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:50 smithi114 ceph-mon[32501]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:52.830 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:52 smithi090 ceph-mon[27825]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:52.830 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:52 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:52.830 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:52 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:52.831 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:52 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:52.831 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:52 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:12:52.831 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:52 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:52.831 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:52 smithi090 ceph-mon[27825]: from='osd.0 [v2:172.21.15.90:6802/2864288066,v1:172.21.15.90:6803/2864288066]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-24T09:12:52.831 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:52 smithi090 ceph-mon[27825]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-24T09:12:52.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:52 smithi149 ceph-mon[32592]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:52.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:52 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:52.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:52 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:52.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:52 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:52.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:52 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:12:52.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:52 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:52.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:52 smithi149 ceph-mon[32592]: from='osd.0 [v2:172.21.15.90:6802/2864288066,v1:172.21.15.90:6803/2864288066]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-24T09:12:52.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:52 smithi149 ceph-mon[32592]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-24T09:12:52.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:52 smithi114 ceph-mon[32501]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:52.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:52 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:52.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:52 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:52.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:52 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:52.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:52 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:12:52.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:52 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:52.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:52 smithi114 ceph-mon[32501]: from='osd.0 [v2:172.21.15.90:6802/2864288066,v1:172.21.15.90:6803/2864288066]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-24T09:12:52.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:52 smithi114 ceph-mon[32501]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-24T09:12:52.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:52.969+0000 7f60f7e00700 1 -- 172.21.15.90:0/806692043 <== mgr.14152 v2:172.21.15.90:6800/4261662956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6100062f30 con 0x7f60e4039890 2024-04-24T09:12:52.970 INFO:teuthology.orchestra.run.smithi090.stdout:Created osd(s) 0 on host 'smithi090' 2024-04-24T09:12:52.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:52.972+0000 7f6106400700 1 -- 172.21.15.90:0/806692043 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f60e4039890 msgr2=0x7f60e403bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:52.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:52.972+0000 7f6106400700 1 --2- 172.21.15.90:0/806692043 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f60e4039890 0x7f60e403bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f60e8009c60 tx=0x7f60e8009260 comp rx=0 tx=0).stop 2024-04-24T09:12:52.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:52.972+0000 7f6106400700 1 -- 172.21.15.90:0/806692043 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6100109190 msgr2=0x7f610019f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:52.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:52.972+0000 7f6106400700 1 --2- 172.21.15.90:0/806692043 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6100109190 0x7f610019f3a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f60f000bee0 tx=0x7f60f000bf10 comp rx=0 tx=0).stop 2024-04-24T09:12:52.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:52.972+0000 7f6106400700 1 -- 172.21.15.90:0/806692043 shutdown_connections 2024-04-24T09:12:52.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:52.972+0000 7f6106400700 1 --2- 172.21.15.90:0/806692043 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f610010ad30 0x7f61001a6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:52.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:52.973+0000 7f6106400700 1 --2- 172.21.15.90:0/806692043 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6100109190 0x7f610019f3a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:52.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:52.973+0000 7f6106400700 1 --2- 172.21.15.90:0/806692043 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f60e4039890 0x7f60e403bd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:52.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:52.973+0000 7f6106400700 1 --2- 172.21.15.90:0/806692043 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f610010a390 0x7f610019f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:52.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:52.973+0000 7f6106400700 1 -- 172.21.15.90:0/806692043 >> 172.21.15.90:0/806692043 conn(0x7f6100078160 msgr2=0x7f610010d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:52.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:52.973+0000 7f6106400700 1 -- 172.21.15.90:0/806692043 shutdown_connections 2024-04-24T09:12:52.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:12:52.973+0000 7f6106400700 1 -- 172.21.15.90:0/806692043 wait complete. 2024-04-24T09:12:53.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:53 smithi090 ceph-mon[27825]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-24T09:12:53.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:53 smithi090 ceph-mon[27825]: from='osd.0 [v2:172.21.15.90:6802/2864288066,v1:172.21.15.90:6803/2864288066]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-24T09:12:53.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:53 smithi090 ceph-mon[27825]: osdmap e6: 1 total, 0 up, 1 in 2024-04-24T09:12:53.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:53 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T09:12:53.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:53 smithi090 ceph-mon[27825]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-24T09:12:53.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:53 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:53.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:53 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:53.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:53 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:12:53.757 DEBUG:teuthology.orchestra.run.smithi090:osd.0> sudo journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@osd.0.service 2024-04-24T09:12:53.764 INFO:tasks.cephadm:Deploying osd.1 on smithi114 with /dev/vg_nvme/lv_4... 2024-04-24T09:12:53.764 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-24T09:12:53.791 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:53 smithi114 ceph-mon[32501]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-24T09:12:53.791 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:53 smithi114 ceph-mon[32501]: from='osd.0 [v2:172.21.15.90:6802/2864288066,v1:172.21.15.90:6803/2864288066]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-24T09:12:53.791 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:53 smithi114 ceph-mon[32501]: osdmap e6: 1 total, 0 up, 1 in 2024-04-24T09:12:53.791 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:53 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T09:12:53.791 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:53 smithi114 ceph-mon[32501]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-24T09:12:53.791 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:53 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:53.791 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:53 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:53.791 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:53 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:12:53.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:53 smithi149 ceph-mon[32592]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-24T09:12:53.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:53 smithi149 ceph-mon[32592]: from='osd.0 [v2:172.21.15.90:6802/2864288066,v1:172.21.15.90:6803/2864288066]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-24T09:12:53.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:53 smithi149 ceph-mon[32592]: osdmap e6: 1 total, 0 up, 1 in 2024-04-24T09:12:53.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:53 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T09:12:53.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:53 smithi149 ceph-mon[32592]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-24T09:12:53.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:53 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:53.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:53 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:53.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:53 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:12:54.007 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T09:12:54.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:54 smithi090 ceph-mon[27825]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:54.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:54 smithi090 ceph-mon[27825]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-04-24T09:12:54.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:54 smithi090 ceph-mon[27825]: osdmap e7: 1 total, 0 up, 1 in 2024-04-24T09:12:54.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:54 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T09:12:54.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:54 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T09:12:54.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:54 smithi149 ceph-mon[32592]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:54.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:54 smithi149 ceph-mon[32592]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-04-24T09:12:54.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:54 smithi149 ceph-mon[32592]: osdmap e7: 1 total, 0 up, 1 in 2024-04-24T09:12:54.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:54 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T09:12:54.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:54 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T09:12:54.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:54 smithi114 ceph-mon[32501]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:54.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:54 smithi114 ceph-mon[32501]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-04-24T09:12:54.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:54 smithi114 ceph-mon[32501]: osdmap e7: 1 total, 0 up, 1 in 2024-04-24T09:12:54.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:54 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T09:12:54.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:54 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T09:12:55.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:55 smithi114 ceph-mon[32501]: purged_snaps scrub starts 2024-04-24T09:12:55.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:55 smithi114 ceph-mon[32501]: purged_snaps scrub ok 2024-04-24T09:12:55.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:55 smithi114 ceph-mon[32501]: osd.0 [v2:172.21.15.90:6802/2864288066,v1:172.21.15.90:6803/2864288066] boot 2024-04-24T09:12:55.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:55 smithi114 ceph-mon[32501]: osdmap e8: 1 total, 1 up, 1 in 2024-04-24T09:12:55.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:55 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T09:12:55.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:55 smithi090 ceph-mon[27825]: purged_snaps scrub starts 2024-04-24T09:12:55.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:55 smithi090 ceph-mon[27825]: purged_snaps scrub ok 2024-04-24T09:12:55.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:55 smithi090 ceph-mon[27825]: osd.0 [v2:172.21.15.90:6802/2864288066,v1:172.21.15.90:6803/2864288066] boot 2024-04-24T09:12:55.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:55 smithi090 ceph-mon[27825]: osdmap e8: 1 total, 1 up, 1 in 2024-04-24T09:12:55.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:55 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T09:12:55.859 INFO:teuthology.orchestra.run.smithi114.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-24T09:12:55.859 INFO:teuthology.orchestra.run.smithi114.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-24T09:12:55.859 INFO:teuthology.orchestra.run.smithi114.stderr: stderr: 10+0 records in 2024-04-24T09:12:55.859 INFO:teuthology.orchestra.run.smithi114.stderr:10+0 records out 2024-04-24T09:12:55.859 INFO:teuthology.orchestra.run.smithi114.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115823 s, 905 MB/s 2024-04-24T09:12:55.860 INFO:teuthology.orchestra.run.smithi114.stderr:--> Zapping successful for: 2024-04-24T09:12:55.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:55 smithi149 ceph-mon[32592]: purged_snaps scrub starts 2024-04-24T09:12:55.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:55 smithi149 ceph-mon[32592]: purged_snaps scrub ok 2024-04-24T09:12:55.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:55 smithi149 ceph-mon[32592]: osd.0 [v2:172.21.15.90:6802/2864288066,v1:172.21.15.90:6803/2864288066] boot 2024-04-24T09:12:55.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:55 smithi149 ceph-mon[32592]: osdmap e8: 1 total, 1 up, 1 in 2024-04-24T09:12:55.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:55 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T09:12:56.474 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi114:vg_nvme/lv_4 2024-04-24T09:12:56.752 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T09:12:56.755 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:56 smithi114 ceph-mon[32501]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:56.755 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:56 smithi114 ceph-mon[32501]: osdmap e9: 1 total, 1 up, 1 in 2024-04-24T09:12:56.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:56 smithi090 ceph-mon[27825]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:56.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:56 smithi090 ceph-mon[27825]: osdmap e9: 1 total, 1 up, 1 in 2024-04-24T09:12:56.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:56 smithi149 ceph-mon[32592]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T09:12:56.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:56 smithi149 ceph-mon[32592]: osdmap e9: 1 total, 1 up, 1 in 2024-04-24T09:12:58.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:12:58.142+0000 7ff14de1b700 1 -- 172.21.15.114:0/3303829222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1481046f0 msgr2=0x7ff14810afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:58.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:12:58.142+0000 7ff14de1b700 1 --2- 172.21.15.114:0/3303829222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1481046f0 0x7ff14810afa0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff1400099c0 tx=0x7ff140009cd0 comp rx=0 tx=0).stop 2024-04-24T09:12:58.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:12:58.142+0000 7ff14de1b700 1 -- 172.21.15.114:0/3303829222 shutdown_connections 2024-04-24T09:12:58.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:12:58.142+0000 7ff14de1b700 1 --2- 172.21.15.114:0/3303829222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff1481046f0 0x7ff14810afa0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:58.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:12:58.142+0000 7ff14de1b700 1 --2- 172.21.15.114:0/3303829222 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff148102b50 0x7ff148102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:58.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:12:58.142+0000 7ff14de1b700 1 --2- 172.21.15.114:0/3303829222 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff148103d50 0x7ff1481041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:58.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:12:58.142+0000 7ff14de1b700 1 -- 172.21.15.114:0/3303829222 >> 172.21.15.114:0/3303829222 conn(0x7ff1480fe0f0 msgr2=0x7ff148100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:58.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:12:58.142+0000 7ff14de1b700 1 -- 172.21.15.114:0/3303829222 shutdown_connections 2024-04-24T09:12:58.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:12:58.142+0000 7ff14de1b700 1 -- 172.21.15.114:0/3303829222 wait complete. 2024-04-24T09:12:58.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:12:58.143+0000 7ff14de1b700 1 Processor -- start 2024-04-24T09:12:58.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:12:58.143+0000 7ff14de1b700 1 -- start start 2024-04-24T09:12:58.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:12:58.143+0000 7ff14de1b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff148102b50 0x7ff148198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:58.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:12:58.143+0000 7ff14de1b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff148103d50 0x7ff1481992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:58.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:12:58.143+0000 7ff14de1b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1481046f0 0x7ff1481a0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:58.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:12:58.143+0000 7ff14de1b700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff14810e800 con 0x7ff1481046f0 2024-04-24T09:12:58.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:12:58.143+0000 7ff14de1b700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff14810e680 con 0x7ff148103d50 2024-04-24T09:12:58.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:12:58.143+0000 7ff14de1b700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff14810e980 con 0x7ff148102b50 2024-04-24T09:12:58.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:12:58.143+0000 7ff146ffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff148103d50 0x7ff1481992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:58.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:12:58.143+0000 7ff146ffd700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff148103d50 0x7ff1481992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49414/0 (socket says 172.21.15.114:49414) 2024-04-24T09:12:58.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:12:58.143+0000 7ff146ffd700 1 -- 172.21.15.114:0/2599792545 learned_addr learned my addr 172.21.15.114:0/2599792545 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-24T09:12:58.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:12:58.143+0000 7ff1477fe700 1 --2- 172.21.15.114:0/2599792545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff148102b50 0x7ff148198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:58.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:12:58.144+0000 7ff146ffd700 1 -- 172.21.15.114:0/2599792545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff148102b50 msgr2=0x7ff148198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:58.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:12:58.144+0000 7ff146ffd700 1 --2- 172.21.15.114:0/2599792545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff148102b50 0x7ff148198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:58.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:12:58.144+0000 7ff146ffd700 1 -- 172.21.15.114:0/2599792545 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1481046f0 msgr2=0x7ff1481a0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:58.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:12:58.144+0000 7ff146ffd700 1 --2- 172.21.15.114:0/2599792545 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1481046f0 0x7ff1481a0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:58.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:12:58.144+0000 7ff146ffd700 1 -- 172.21.15.114:0/2599792545 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1481a0a10 con 0x7ff148103d50 2024-04-24T09:12:58.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:12:58.144+0000 7ff146ffd700 1 --2- 172.21.15.114:0/2599792545 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff148103d50 0x7ff1481992f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff13c00b530 tx=0x7ff13c00b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:58.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:12:58.144+0000 7ff144ff9700 1 -- 172.21.15.114:0/2599792545 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff13c015020 con 0x7ff148103d50 2024-04-24T09:12:58.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:12:58.144+0000 7ff144ff9700 1 -- 172.21.15.114:0/2599792545 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff13c011d10 con 0x7ff148103d50 2024-04-24T09:12:58.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:12:58.144+0000 7ff14de1b700 1 -- 172.21.15.114:0/2599792545 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1481a0d00 con 0x7ff148103d50 2024-04-24T09:12:58.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:12:58.144+0000 7ff144ff9700 1 -- 172.21.15.114:0/2599792545 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff13c00f2e0 con 0x7ff148103d50 2024-04-24T09:12:58.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:12:58.144+0000 7ff14de1b700 1 -- 172.21.15.114:0/2599792545 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1481a1260 con 0x7ff148103d50 2024-04-24T09:12:58.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:12:58.145+0000 7ff14de1b700 1 -- 172.21.15.114:0/2599792545 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff14802dfe0 con 0x7ff148103d50 2024-04-24T09:12:58.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:12:58.146+0000 7ff144ff9700 1 -- 172.21.15.114:0/2599792545 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7ff13c018020 con 0x7ff148103d50 2024-04-24T09:12:58.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:12:58.147+0000 7ff144ff9700 1 --2- 172.21.15.114:0/2599792545 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7ff1300a3bb0 0x7ff1300a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:58.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:12:58.147+0000 7ff144ff9700 1 -- 172.21.15.114:0/2599792545 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7ff13c0ce630 con 0x7ff148103d50 2024-04-24T09:12:58.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:12:58.147+0000 7ff1477fe700 1 --2- 172.21.15.114:0/2599792545 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7ff1300a3bb0 0x7ff1300a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:58.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:12:58.148+0000 7ff1477fe700 1 --2- 172.21.15.114:0/2599792545 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7ff1300a3bb0 0x7ff1300a6070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff1380060b0 tx=0x7ff138007b80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:58.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:12:58.151+0000 7ff144ff9700 1 -- 172.21.15.114:0/2599792545 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff13c01f020 con 0x7ff148103d50 2024-04-24T09:12:58.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:12:58.355+0000 7ff14de1b700 1 -- 172.21.15.114:0/2599792545 --> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7ff148062f30 con 0x7ff1300a3bb0 2024-04-24T09:12:58.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:58 smithi114 ceph-mon[32501]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T09:12:58.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:58 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:12:58.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:58 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:12:58.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:58 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:58.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:58 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:58.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:58 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:58.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:58 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:12:58.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:58 smithi090 ceph-mon[27825]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T09:12:58.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:58 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:12:58.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:58 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:12:58.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:58 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:58.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:58 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:58.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:58 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:58.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:58 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:12:58.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:58 smithi149 ceph-mon[32592]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T09:12:58.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:58 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:12:58.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:58 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:12:58.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:58 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:58.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:58 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:58.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:58 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:58.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:58 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:12:59.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:59 smithi090 ceph-mon[27825]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:12:59.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:59 smithi090 ceph-mon[27825]: Detected new or changed devices on smithi090 2024-04-24T09:12:59.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:59 smithi090 ceph-mon[27825]: Adjusting osd_memory_target on smithi090 to 17089M 2024-04-24T09:12:59.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:59 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:59.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:59 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:59.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:59 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:12:59.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:12:59 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:59.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:59 smithi149 ceph-mon[32592]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:12:59.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:59 smithi149 ceph-mon[32592]: Detected new or changed devices on smithi090 2024-04-24T09:12:59.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:59 smithi149 ceph-mon[32592]: Adjusting osd_memory_target on smithi090 to 17089M 2024-04-24T09:12:59.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:59 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:59.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:59 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:59.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:59 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:12:59.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:12:59 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:59.876 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:59 smithi114 ceph-mon[32501]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:12:59.877 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:59 smithi114 ceph-mon[32501]: Detected new or changed devices on smithi090 2024-04-24T09:12:59.877 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:59 smithi114 ceph-mon[32501]: Adjusting osd_memory_target on smithi090 to 17089M 2024-04-24T09:12:59.877 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:59 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:12:59.877 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:59 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:59.877 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:59 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:12:59.877 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:12:59 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:00.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:00 smithi090 ceph-mon[27825]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T09:13:00.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:00 smithi149 ceph-mon[32592]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T09:13:00.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:00 smithi114 ceph-mon[32501]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T09:13:01.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:01 smithi090 ceph-mon[27825]: from='client.? 172.21.15.114:0/3462518985' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1567ce8f-24e6-42cf-bf64-f6f0c534a9bd"}]: dispatch 2024-04-24T09:13:01.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:01 smithi090 ceph-mon[27825]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1567ce8f-24e6-42cf-bf64-f6f0c534a9bd"}]: dispatch 2024-04-24T09:13:01.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:01 smithi090 ceph-mon[27825]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1567ce8f-24e6-42cf-bf64-f6f0c534a9bd"}]': finished 2024-04-24T09:13:01.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:01 smithi090 ceph-mon[27825]: osdmap e10: 2 total, 1 up, 2 in 2024-04-24T09:13:01.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:01 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T09:13:01.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:01 smithi149 ceph-mon[32592]: from='client.? 172.21.15.114:0/3462518985' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1567ce8f-24e6-42cf-bf64-f6f0c534a9bd"}]: dispatch 2024-04-24T09:13:01.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:01 smithi149 ceph-mon[32592]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1567ce8f-24e6-42cf-bf64-f6f0c534a9bd"}]: dispatch 2024-04-24T09:13:01.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:01 smithi149 ceph-mon[32592]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1567ce8f-24e6-42cf-bf64-f6f0c534a9bd"}]': finished 2024-04-24T09:13:01.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:01 smithi149 ceph-mon[32592]: osdmap e10: 2 total, 1 up, 2 in 2024-04-24T09:13:01.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:01 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T09:13:01.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:01 smithi114 ceph-mon[32501]: from='client.? 172.21.15.114:0/3462518985' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1567ce8f-24e6-42cf-bf64-f6f0c534a9bd"}]: dispatch 2024-04-24T09:13:01.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:01 smithi114 ceph-mon[32501]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1567ce8f-24e6-42cf-bf64-f6f0c534a9bd"}]: dispatch 2024-04-24T09:13:01.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:01 smithi114 ceph-mon[32501]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1567ce8f-24e6-42cf-bf64-f6f0c534a9bd"}]': finished 2024-04-24T09:13:01.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:01 smithi114 ceph-mon[32501]: osdmap e10: 2 total, 1 up, 2 in 2024-04-24T09:13:01.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:01 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T09:13:02.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:02 smithi090 ceph-mon[27825]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T09:13:02.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:02 smithi090 ceph-mon[27825]: from='client.? 172.21.15.114:0/4157432688' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:13:02.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:02 smithi149 ceph-mon[32592]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T09:13:02.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:02 smithi149 ceph-mon[32592]: from='client.? 172.21.15.114:0/4157432688' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:13:02.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:02 smithi114 ceph-mon[32501]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T09:13:02.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:02 smithi114 ceph-mon[32501]: from='client.? 172.21.15.114:0/4157432688' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:13:04.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:04 smithi090 ceph-mon[27825]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T09:13:04.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:04 smithi149 ceph-mon[32592]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T09:13:04.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:04 smithi114 ceph-mon[32501]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T09:13:06.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:06 smithi090 ceph-mon[27825]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T09:13:06.857 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:06 smithi114 ceph-mon[32501]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T09:13:06.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:06 smithi149 ceph-mon[32592]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T09:13:08.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:08 smithi149 ceph-mon[32592]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T09:13:08.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:08 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-24T09:13:08.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:08 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:13:08.891 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:08 smithi114 ceph-mon[32501]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T09:13:08.891 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:08 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-24T09:13:08.891 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:08 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:13:09.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:08 smithi090 ceph-mon[27825]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T09:13:09.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:08 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-24T09:13:09.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:08 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:13:09.743 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:09 smithi114 ceph-mon[32501]: Deploying daemon osd.1 on smithi114 2024-04-24T09:13:10.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:09 smithi090 ceph-mon[27825]: Deploying daemon osd.1 on smithi114 2024-04-24T09:13:10.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:09 smithi149 ceph-mon[32592]: Deploying daemon osd.1 on smithi114 2024-04-24T09:13:10.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:10 smithi114 ceph-mon[32501]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T09:13:11.012 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:10 smithi149 ceph-mon[32592]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T09:13:11.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:10 smithi090 ceph-mon[27825]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T09:13:12.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:12 smithi114 ceph-mon[32501]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T09:13:13.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:12 smithi090 ceph-mon[27825]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T09:13:13.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:12 smithi149 ceph-mon[32592]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T09:13:14.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:14 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:14.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:14 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:14.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:14 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:13:14.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:14 smithi114 ceph-mon[32501]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T09:13:14.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:14 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:14.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:14 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:14.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:14 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:13:14.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:14 smithi090 ceph-mon[27825]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T09:13:14.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:14 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:14.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:14 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:14.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:14 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:13:14.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:14 smithi149 ceph-mon[32592]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T09:13:16.552 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:16 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:16.552 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:16 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:16.552 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:16 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:13:16.553 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:16 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:13:16.553 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:16 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:16.553 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:16 smithi114 ceph-mon[32501]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T09:13:16.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:16 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:16.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:16 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:16.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:16 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:13:16.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:16 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:13:16.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:16 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:16.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:16 smithi090 ceph-mon[27825]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T09:13:16.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:16 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:16.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:16 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:16.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:16 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:13:16.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:16 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:13:16.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:16 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:16.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:16 smithi149 ceph-mon[32592]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T09:13:16.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:13:16.641+0000 7ff144ff9700 1 -- 172.21.15.114:0/2599792545 <== mgr.14152 v2:172.21.15.90:6800/4261662956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff148062f30 con 0x7ff1300a3bb0 2024-04-24T09:13:16.641 INFO:teuthology.orchestra.run.smithi114.stdout:Created osd(s) 1 on host 'smithi114' 2024-04-24T09:13:16.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:13:16.643+0000 7ff14de1b700 1 -- 172.21.15.114:0/2599792545 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7ff1300a3bb0 msgr2=0x7ff1300a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:16.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:13:16.643+0000 7ff14de1b700 1 --2- 172.21.15.114:0/2599792545 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7ff1300a3bb0 0x7ff1300a6070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff1380060b0 tx=0x7ff138007b80 comp rx=0 tx=0).stop 2024-04-24T09:13:16.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:13:16.643+0000 7ff14de1b700 1 -- 172.21.15.114:0/2599792545 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff148103d50 msgr2=0x7ff1481992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:16.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:13:16.643+0000 7ff14de1b700 1 --2- 172.21.15.114:0/2599792545 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff148103d50 0x7ff1481992f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff13c00b530 tx=0x7ff13c00b8f0 comp rx=0 tx=0).stop 2024-04-24T09:13:16.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:13:16.644+0000 7ff14de1b700 1 -- 172.21.15.114:0/2599792545 shutdown_connections 2024-04-24T09:13:16.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:13:16.644+0000 7ff14de1b700 1 --2- 172.21.15.114:0/2599792545 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7ff1300a3bb0 0x7ff1300a6070 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:16.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:13:16.644+0000 7ff14de1b700 1 --2- 172.21.15.114:0/2599792545 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1481046f0 0x7ff1481a0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:16.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:13:16.644+0000 7ff14de1b700 1 --2- 172.21.15.114:0/2599792545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff148102b50 0x7ff148198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:16.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:13:16.644+0000 7ff14de1b700 1 --2- 172.21.15.114:0/2599792545 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff148103d50 0x7ff1481992f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:16.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:13:16.644+0000 7ff14de1b700 1 -- 172.21.15.114:0/2599792545 >> 172.21.15.114:0/2599792545 conn(0x7ff1480fe0f0 msgr2=0x7ff148107050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:16.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:13:16.644+0000 7ff14de1b700 1 -- 172.21.15.114:0/2599792545 shutdown_connections 2024-04-24T09:13:16.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:13:16.644+0000 7ff14de1b700 1 -- 172.21.15.114:0/2599792545 wait complete. 2024-04-24T09:13:17.281 DEBUG:teuthology.orchestra.run.smithi114:osd.1> sudo journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@osd.1.service 2024-04-24T09:13:17.283 INFO:tasks.cephadm:Deploying osd.2 on smithi149 with /dev/vg_nvme/lv_4... 2024-04-24T09:13:17.284 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-24T09:13:17.528 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T09:13:17.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:17 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:17.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:17 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:17.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:17 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:13:17.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:17 smithi090 ceph-mon[27825]: from='osd.1 [v2:172.21.15.114:6800/344222414,v1:172.21.15.114:6801/344222414]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T09:13:17.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:17 smithi090 ceph-mon[27825]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T09:13:17.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:17 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:17.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:17 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:17.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:17 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:13:17.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:17 smithi149 ceph-mon[32592]: from='osd.1 [v2:172.21.15.114:6800/344222414,v1:172.21.15.114:6801/344222414]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T09:13:17.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:17 smithi149 ceph-mon[32592]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T09:13:17.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:17 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:17.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:17 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:17.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:17 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:13:17.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:17 smithi114 ceph-mon[32501]: from='osd.1 [v2:172.21.15.114:6800/344222414,v1:172.21.15.114:6801/344222414]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T09:13:17.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:17 smithi114 ceph-mon[32501]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T09:13:18.665 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:18 smithi114 ceph-mon[32501]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T09:13:18.665 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:18 smithi114 ceph-mon[32501]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-24T09:13:18.665 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:18 smithi114 ceph-mon[32501]: from='osd.1 [v2:172.21.15.114:6800/344222414,v1:172.21.15.114:6801/344222414]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-04-24T09:13:18.665 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:18 smithi114 ceph-mon[32501]: osdmap e11: 2 total, 1 up, 2 in 2024-04-24T09:13:18.665 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:18 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T09:13:18.665 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:18 smithi114 ceph-mon[32501]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-04-24T09:13:18.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:18 smithi149 ceph-mon[32592]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T09:13:18.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:18 smithi149 ceph-mon[32592]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-24T09:13:18.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:18 smithi149 ceph-mon[32592]: from='osd.1 [v2:172.21.15.114:6800/344222414,v1:172.21.15.114:6801/344222414]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-04-24T09:13:18.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:18 smithi149 ceph-mon[32592]: osdmap e11: 2 total, 1 up, 2 in 2024-04-24T09:13:18.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:18 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T09:13:18.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:18 smithi149 ceph-mon[32592]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-04-24T09:13:19.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:18 smithi090 ceph-mon[27825]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T09:13:19.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:18 smithi090 ceph-mon[27825]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-24T09:13:19.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:18 smithi090 ceph-mon[27825]: from='osd.1 [v2:172.21.15.114:6800/344222414,v1:172.21.15.114:6801/344222414]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-04-24T09:13:19.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:18 smithi090 ceph-mon[27825]: osdmap e11: 2 total, 1 up, 2 in 2024-04-24T09:13:19.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:18 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T09:13:19.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:18 smithi090 ceph-mon[27825]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-04-24T09:13:19.426 INFO:journalctl@ceph.osd.1.smithi114.stdout:Apr 24 09:13:19 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-1[37080]: 2024-04-24T09:13:19.046+0000 7f3a51f26700 -1 osd.1 0 waiting for initial osdmap 2024-04-24T09:13:19.426 INFO:journalctl@ceph.osd.1.smithi114.stdout:Apr 24 09:13:19 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-1[37080]: 2024-04-24T09:13:19.050+0000 7f3a49d11700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T09:13:19.460 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-24T09:13:19.460 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-24T09:13:19.460 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2024-04-24T09:13:19.460 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2024-04-24T09:13:19.461 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115217 s, 910 MB/s 2024-04-24T09:13:19.461 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2024-04-24T09:13:19.852 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:19 smithi114 ceph-mon[32501]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-04-24T09:13:19.853 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:19 smithi114 ceph-mon[32501]: osdmap e12: 2 total, 1 up, 2 in 2024-04-24T09:13:19.853 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:19 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T09:13:19.853 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:19 smithi114 ceph-mon[32501]: from='osd.1 ' entity='osd.1' 2024-04-24T09:13:19.853 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:19 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T09:13:19.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:19 smithi090 ceph-mon[27825]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-04-24T09:13:19.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:19 smithi090 ceph-mon[27825]: osdmap e12: 2 total, 1 up, 2 in 2024-04-24T09:13:19.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:19 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T09:13:19.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:19 smithi090 ceph-mon[27825]: from='osd.1 ' entity='osd.1' 2024-04-24T09:13:19.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:19 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T09:13:20.016 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:19 smithi149 ceph-mon[32592]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-04-24T09:13:20.017 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:19 smithi149 ceph-mon[32592]: osdmap e12: 2 total, 1 up, 2 in 2024-04-24T09:13:20.017 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:19 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T09:13:20.017 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:19 smithi149 ceph-mon[32592]: from='osd.1 ' entity='osd.1' 2024-04-24T09:13:20.017 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:19 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T09:13:20.045 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi149:vg_nvme/lv_4 2024-04-24T09:13:20.295 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T09:13:20.678 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:20 smithi149 ceph-mon[32592]: purged_snaps scrub starts 2024-04-24T09:13:20.678 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:20 smithi149 ceph-mon[32592]: purged_snaps scrub ok 2024-04-24T09:13:20.678 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:20 smithi149 ceph-mon[32592]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T09:13:20.678 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:20 smithi149 ceph-mon[32592]: osd.1 [v2:172.21.15.114:6800/344222414,v1:172.21.15.114:6801/344222414] boot 2024-04-24T09:13:20.678 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:20 smithi149 ceph-mon[32592]: osdmap e13: 2 total, 2 up, 2 in 2024-04-24T09:13:20.678 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:20 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T09:13:20.678 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:20 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:20.679 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:20 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:20.679 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:20 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:13:20.679 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:20 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:20.679 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:20 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:13:20.679 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:20 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:13:20.679 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:20 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:20.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:20 smithi114 ceph-mon[32501]: purged_snaps scrub starts 2024-04-24T09:13:20.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:20 smithi114 ceph-mon[32501]: purged_snaps scrub ok 2024-04-24T09:13:20.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:20 smithi114 ceph-mon[32501]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T09:13:20.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:20 smithi114 ceph-mon[32501]: osd.1 [v2:172.21.15.114:6800/344222414,v1:172.21.15.114:6801/344222414] boot 2024-04-24T09:13:20.929 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:20 smithi114 ceph-mon[32501]: osdmap e13: 2 total, 2 up, 2 in 2024-04-24T09:13:20.929 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:20 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T09:13:20.929 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:20 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:20.929 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:20 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:20.929 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:20 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:13:20.929 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:20 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:20.930 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:20 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:13:20.930 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:20 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:13:20.930 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:20 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:21.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:20 smithi090 ceph-mon[27825]: purged_snaps scrub starts 2024-04-24T09:13:21.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:20 smithi090 ceph-mon[27825]: purged_snaps scrub ok 2024-04-24T09:13:21.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:20 smithi090 ceph-mon[27825]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-24T09:13:21.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:20 smithi090 ceph-mon[27825]: osd.1 [v2:172.21.15.114:6800/344222414,v1:172.21.15.114:6801/344222414] boot 2024-04-24T09:13:21.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:20 smithi090 ceph-mon[27825]: osdmap e13: 2 total, 2 up, 2 in 2024-04-24T09:13:21.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:20 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T09:13:21.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:20 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:21.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:20 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:21.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:20 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:13:21.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:20 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:21.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:20 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:13:21.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:20 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:13:21.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:20 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:21.656 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:21 smithi149 ceph-mon[32592]: Detected new or changed devices on smithi114 2024-04-24T09:13:21.657 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:21 smithi149 ceph-mon[32592]: Adjusting osd_memory_target on smithi114 to 17088M 2024-04-24T09:13:21.919 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:21.919+0000 7f62c1497700 1 -- 172.21.15.149:0/2724765597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62bc103600 msgr2=0x7f62bc109eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:21.919 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:21.919+0000 7f62c1497700 1 --2- 172.21.15.149:0/2724765597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62bc103600 0x7f62bc109eb0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f62b400b0a0 tx=0x7f62b400b3b0 comp rx=0 tx=0).stop 2024-04-24T09:13:21.919 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:21.919+0000 7f62c1497700 1 -- 172.21.15.149:0/2724765597 shutdown_connections 2024-04-24T09:13:21.919 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:21.919+0000 7f62c1497700 1 --2- 172.21.15.149:0/2724765597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62bc103600 0x7f62bc109eb0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:21.919 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:21.919+0000 7f62c1497700 1 --2- 172.21.15.149:0/2724765597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62bc101a60 0x7f62bc101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:21.919 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:21.919+0000 7f62c1497700 1 --2- 172.21.15.149:0/2724765597 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f62bc102c60 0x7f62bc1030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:21.919 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:21.919+0000 7f62c1497700 1 -- 172.21.15.149:0/2724765597 >> 172.21.15.149:0/2724765597 conn(0x7f62bc0fd020 msgr2=0x7f62bc0ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:21.919 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:21.919+0000 7f62c1497700 1 -- 172.21.15.149:0/2724765597 shutdown_connections 2024-04-24T09:13:21.919 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:21.919+0000 7f62c1497700 1 -- 172.21.15.149:0/2724765597 wait complete. 2024-04-24T09:13:21.919 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:21.919+0000 7f62c1497700 1 Processor -- start 2024-04-24T09:13:21.919 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:21.920+0000 7f62c1497700 1 -- start start 2024-04-24T09:13:21.921 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:21.920+0000 7f62c1497700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62bc101a60 0x7f62bc19f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:21.921 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:21.920+0000 7f62c1497700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f62bc102c60 0x7f62bc19fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:21.921 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:21.920+0000 7f62c1497700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62bc103600 0x7f62bc1a6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:21.921 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:21.920+0000 7f62c1497700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f62bc10c600 con 0x7f62bc103600 2024-04-24T09:13:21.921 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:21.920+0000 7f62c1497700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f62bc10c480 con 0x7f62bc102c60 2024-04-24T09:13:21.922 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:21.920+0000 7f62c1497700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f62bc10c780 con 0x7f62bc101a60 2024-04-24T09:13:21.922 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:21.920+0000 7f62baffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62bc101a60 0x7f62bc19f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:21.922 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:21.920+0000 7f62baffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62bc101a60 0x7f62bc19f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:56238/0 (socket says 172.21.15.149:56238) 2024-04-24T09:13:21.922 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:21.920+0000 7f62baffd700 1 -- 172.21.15.149:0/1728092490 learned_addr learned my addr 172.21.15.149:0/1728092490 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-04-24T09:13:21.922 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:21.920+0000 7f62ba7fc700 1 --2- 172.21.15.149:0/1728092490 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f62bc102c60 0x7f62bc19fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:21.922 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:21.920+0000 7f62baffd700 1 -- 172.21.15.149:0/1728092490 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f62bc102c60 msgr2=0x7f62bc19fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:21.922 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:21.920+0000 7f62baffd700 1 --2- 172.21.15.149:0/1728092490 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f62bc102c60 0x7f62bc19fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:21.922 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:21.920+0000 7f62baffd700 1 -- 172.21.15.149:0/1728092490 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62bc103600 msgr2=0x7f62bc1a6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:21.922 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:21.920+0000 7f62baffd700 1 --2- 172.21.15.149:0/1728092490 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62bc103600 0x7f62bc1a6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:21.922 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:21.920+0000 7f62baffd700 1 -- 172.21.15.149:0/1728092490 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62bc1a7060 con 0x7f62bc101a60 2024-04-24T09:13:21.922 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:21.921+0000 7f62baffd700 1 --2- 172.21.15.149:0/1728092490 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62bc101a60 0x7f62bc19f980 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f62ac00b500 tx=0x7f62ac00b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:21.922 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:21.921+0000 7f62a3fff700 1 -- 172.21.15.149:0/1728092490 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ac015020 con 0x7f62bc101a60 2024-04-24T09:13:21.922 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:21.921+0000 7f62a3fff700 1 -- 172.21.15.149:0/1728092490 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62ac011d10 con 0x7f62bc101a60 2024-04-24T09:13:21.923 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:21.921+0000 7f62c1497700 1 -- 172.21.15.149:0/1728092490 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62bc1a7350 con 0x7f62bc101a60 2024-04-24T09:13:21.923 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:21.921+0000 7f62a3fff700 1 -- 172.21.15.149:0/1728092490 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ac010e50 con 0x7f62bc101a60 2024-04-24T09:13:21.923 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:21.921+0000 7f62c1497700 1 -- 172.21.15.149:0/1728092490 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62bc1a7910 con 0x7f62bc101a60 2024-04-24T09:13:21.923 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:21.922+0000 7f62c1497700 1 -- 172.21.15.149:0/1728092490 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62bc111f70 con 0x7f62bc101a60 2024-04-24T09:13:21.924 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:21.923+0000 7f62a3fff700 1 -- 172.21.15.149:0/1728092490 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f62ac018020 con 0x7f62bc101a60 2024-04-24T09:13:21.924 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:21.923+0000 7f62a3fff700 1 --2- 172.21.15.149:0/1728092490 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f62a40a3d50 0x7f62a40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:21.924 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:21.923+0000 7f62a3fff700 1 -- 172.21.15.149:0/1728092490 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f62ac0ce2f0 con 0x7f62bc101a60 2024-04-24T09:13:21.924 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:21.924+0000 7f62ba7fc700 1 --2- 172.21.15.149:0/1728092490 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f62a40a3d50 0x7f62a40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:21.924 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:21.925+0000 7f62ba7fc700 1 --2- 172.21.15.149:0/1728092490 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f62a40a3d50 0x7f62a40a6210 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f62b0005950 tx=0x7f62b000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:21.925 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:21 smithi114 ceph-mon[32501]: Detected new or changed devices on smithi114 2024-04-24T09:13:21.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:21 smithi114 ceph-mon[32501]: Adjusting osd_memory_target on smithi114 to 17088M 2024-04-24T09:13:21.928 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:21.928+0000 7f62a3fff700 1 -- 172.21.15.149:0/1728092490 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f62ac01f020 con 0x7f62bc101a60 2024-04-24T09:13:22.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:21 smithi090 ceph-mon[27825]: Detected new or changed devices on smithi114 2024-04-24T09:13:22.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:21 smithi090 ceph-mon[27825]: Adjusting osd_memory_target on smithi114 to 17088M 2024-04-24T09:13:22.138 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:22.139+0000 7f62c1497700 1 -- 172.21.15.149:0/1728092490 --> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f62bc062f30 con 0x7f62a40a3d50 2024-04-24T09:13:22.904 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:22 smithi149 ceph-mon[32592]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T09:13:22.904 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:22 smithi149 ceph-mon[32592]: osdmap e14: 2 total, 2 up, 2 in 2024-04-24T09:13:22.904 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:22 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:13:22.904 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:22 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:13:22.904 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:22 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:13:22.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:22 smithi114 ceph-mon[32501]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T09:13:22.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:22 smithi114 ceph-mon[32501]: osdmap e14: 2 total, 2 up, 2 in 2024-04-24T09:13:22.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:22 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:13:22.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:22 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:13:22.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:22 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:13:23.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:22 smithi090 ceph-mon[27825]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T09:13:23.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:22 smithi090 ceph-mon[27825]: osdmap e14: 2 total, 2 up, 2 in 2024-04-24T09:13:23.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:22 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:13:23.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:22 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:13:23.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:22 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:13:23.908 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:23 smithi149 ceph-mon[32592]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:13:23.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:23 smithi114 ceph-mon[32501]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:13:24.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:23 smithi090 ceph-mon[27825]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:13:24.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:24 smithi114 ceph-mon[32501]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T09:13:25.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:24 smithi090 ceph-mon[27825]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T09:13:25.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:24 smithi149 ceph-mon[32592]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T09:13:25.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:25 smithi114 ceph-mon[32501]: from='client.? 172.21.15.149:0/3162139723' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f861b304-9c7d-4629-8522-37f33a747945"}]: dispatch 2024-04-24T09:13:25.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:25 smithi114 ceph-mon[32501]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f861b304-9c7d-4629-8522-37f33a747945"}]: dispatch 2024-04-24T09:13:25.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:25 smithi114 ceph-mon[32501]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f861b304-9c7d-4629-8522-37f33a747945"}]': finished 2024-04-24T09:13:25.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:25 smithi114 ceph-mon[32501]: osdmap e15: 3 total, 2 up, 3 in 2024-04-24T09:13:25.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:25 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T09:13:25.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:25 smithi114 ceph-mon[32501]: from='client.? 172.21.15.149:0/2765463911' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:13:26.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:25 smithi090 ceph-mon[27825]: from='client.? 172.21.15.149:0/3162139723' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f861b304-9c7d-4629-8522-37f33a747945"}]: dispatch 2024-04-24T09:13:26.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:25 smithi090 ceph-mon[27825]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f861b304-9c7d-4629-8522-37f33a747945"}]: dispatch 2024-04-24T09:13:26.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:25 smithi090 ceph-mon[27825]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f861b304-9c7d-4629-8522-37f33a747945"}]': finished 2024-04-24T09:13:26.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:25 smithi090 ceph-mon[27825]: osdmap e15: 3 total, 2 up, 3 in 2024-04-24T09:13:26.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:25 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T09:13:26.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:25 smithi090 ceph-mon[27825]: from='client.? 172.21.15.149:0/2765463911' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:13:26.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:25 smithi149 ceph-mon[32592]: from='client.? 172.21.15.149:0/3162139723' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f861b304-9c7d-4629-8522-37f33a747945"}]: dispatch 2024-04-24T09:13:26.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:25 smithi149 ceph-mon[32592]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f861b304-9c7d-4629-8522-37f33a747945"}]: dispatch 2024-04-24T09:13:26.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:25 smithi149 ceph-mon[32592]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f861b304-9c7d-4629-8522-37f33a747945"}]': finished 2024-04-24T09:13:26.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:25 smithi149 ceph-mon[32592]: osdmap e15: 3 total, 2 up, 3 in 2024-04-24T09:13:26.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:25 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T09:13:26.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:25 smithi149 ceph-mon[32592]: from='client.? 172.21.15.149:0/2765463911' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:13:26.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:26 smithi114 ceph-mon[32501]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T09:13:27.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:26 smithi090 ceph-mon[27825]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T09:13:27.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:26 smithi149 ceph-mon[32592]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T09:13:27.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:27 smithi114 ceph-mon[32501]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T09:13:28.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:27 smithi090 ceph-mon[27825]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T09:13:28.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:27 smithi149 ceph-mon[32592]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T09:13:30.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:30 smithi149 ceph-mon[32592]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T09:13:30.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:30 smithi090 ceph-mon[27825]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T09:13:30.925 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:30 smithi114 ceph-mon[32501]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T09:13:32.781 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:32 smithi149 ceph-mon[32592]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T09:13:32.781 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:32 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-24T09:13:32.781 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:32 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:13:32.824 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:32 smithi114 ceph-mon[32501]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T09:13:32.824 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:32 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-24T09:13:32.824 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:32 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:13:32.851 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:32 smithi090 ceph-mon[27825]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T09:13:32.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:32 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-24T09:13:32.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:32 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:13:33.830 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:33 smithi149 ceph-mon[32592]: Deploying daemon osd.2 on smithi149 2024-04-24T09:13:33.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:33 smithi090 ceph-mon[27825]: Deploying daemon osd.2 on smithi149 2024-04-24T09:13:33.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:33 smithi114 ceph-mon[32501]: Deploying daemon osd.2 on smithi149 2024-04-24T09:13:34.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:34 smithi149 ceph-mon[32592]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T09:13:34.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:34 smithi090 ceph-mon[27825]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T09:13:34.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:34 smithi114 ceph-mon[32501]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T09:13:36.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:36 smithi149 ceph-mon[32592]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T09:13:36.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:36 smithi090 ceph-mon[27825]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T09:13:36.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:36 smithi114 ceph-mon[32501]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T09:13:38.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:38 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:38.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:38 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:38.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:38 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:13:38.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:38 smithi090 ceph-mon[27825]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T09:13:38.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:38 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:38.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:38 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:38.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:38 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:13:38.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:38 smithi149 ceph-mon[32592]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T09:13:38.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:38 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:38.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:38 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:38.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:38 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:13:38.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:38 smithi114 ceph-mon[32501]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T09:13:39.972 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:39.971+0000 7f62a3fff700 1 -- 172.21.15.149:0/1728092490 <== mgr.14152 v2:172.21.15.90:6800/4261662956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f62bc062f30 con 0x7f62a40a3d50 2024-04-24T09:13:39.972 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 2 on host 'smithi149' 2024-04-24T09:13:39.974 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:39.973+0000 7f62c1497700 1 -- 172.21.15.149:0/1728092490 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f62a40a3d50 msgr2=0x7f62a40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:39.974 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:39.973+0000 7f62c1497700 1 --2- 172.21.15.149:0/1728092490 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f62a40a3d50 0x7f62a40a6210 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f62b0005950 tx=0x7f62b000a300 comp rx=0 tx=0).stop 2024-04-24T09:13:39.974 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:39.973+0000 7f62c1497700 1 -- 172.21.15.149:0/1728092490 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62bc101a60 msgr2=0x7f62bc19f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:39.974 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:39.973+0000 7f62c1497700 1 --2- 172.21.15.149:0/1728092490 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62bc101a60 0x7f62bc19f980 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f62ac00b500 tx=0x7f62ac00b810 comp rx=0 tx=0).stop 2024-04-24T09:13:39.974 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:39.973+0000 7f62c1497700 1 -- 172.21.15.149:0/1728092490 shutdown_connections 2024-04-24T09:13:39.974 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:39.973+0000 7f62c1497700 1 --2- 172.21.15.149:0/1728092490 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f62a40a3d50 0x7f62a40a6210 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:39.974 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:39.973+0000 7f62c1497700 1 --2- 172.21.15.149:0/1728092490 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62bc103600 0x7f62bc1a6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:39.974 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:39.973+0000 7f62c1497700 1 --2- 172.21.15.149:0/1728092490 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62bc101a60 0x7f62bc19f980 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:39.974 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:39.973+0000 7f62c1497700 1 --2- 172.21.15.149:0/1728092490 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f62bc102c60 0x7f62bc19fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:39.974 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:39.973+0000 7f62c1497700 1 -- 172.21.15.149:0/1728092490 >> 172.21.15.149:0/1728092490 conn(0x7f62bc0fd020 msgr2=0x7f62bc10f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:39.975 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:39.973+0000 7f62c1497700 1 -- 172.21.15.149:0/1728092490 shutdown_connections 2024-04-24T09:13:39.975 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:13:39.973+0000 7f62c1497700 1 -- 172.21.15.149:0/1728092490 wait complete. 2024-04-24T09:13:40.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:40 smithi149 ceph-mon[32592]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T09:13:40.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:40 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:40.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:40 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:40.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:40 smithi090 ceph-mon[27825]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T09:13:40.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:40 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:40.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:40 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:40.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:40 smithi114 ceph-mon[32501]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T09:13:40.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:40 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:40.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:40 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:40.981 DEBUG:teuthology.orchestra.run.smithi149:osd.2> sudo journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@osd.2.service 2024-04-24T09:13:40.983 INFO:tasks.cephadm:Deploying osd.3 on smithi090 with /dev/vg_nvme/lv_3... 2024-04-24T09:13:40.983 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-24T09:13:41.411 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:13:41.536 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:41 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:41.536 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:41 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:41.536 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:41 smithi149 ceph-mon[32592]: from='osd.2 [v2:172.21.15.149:6800/3257067419,v1:172.21.15.149:6801/3257067419]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T09:13:41.536 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:41 smithi149 ceph-mon[32592]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T09:13:41.792 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:41 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:41.792 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:41 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:41.792 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:41 smithi090 ceph-mon[27825]: from='osd.2 [v2:172.21.15.149:6800/3257067419,v1:172.21.15.149:6801/3257067419]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T09:13:41.792 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:41 smithi090 ceph-mon[27825]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T09:13:41.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:41 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:41.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:41 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:41.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:41 smithi114 ceph-mon[32501]: from='osd.2 [v2:172.21.15.149:6800/3257067419,v1:172.21.15.149:6801/3257067419]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T09:13:41.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:41 smithi114 ceph-mon[32501]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T09:13:42.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:42 smithi114 ceph-mon[32501]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T09:13:42.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:42 smithi114 ceph-mon[32501]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-24T09:13:42.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:42 smithi114 ceph-mon[32501]: osdmap e16: 3 total, 2 up, 3 in 2024-04-24T09:13:42.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:42 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T09:13:42.677 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:42 smithi114 ceph-mon[32501]: from='osd.2 [v2:172.21.15.149:6800/3257067419,v1:172.21.15.149:6801/3257067419]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-04-24T09:13:42.677 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:42 smithi114 ceph-mon[32501]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-04-24T09:13:42.771 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:42 smithi090 ceph-mon[27825]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T09:13:42.771 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:42 smithi090 ceph-mon[27825]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-24T09:13:42.772 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:42 smithi090 ceph-mon[27825]: osdmap e16: 3 total, 2 up, 3 in 2024-04-24T09:13:42.772 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:42 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T09:13:42.772 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:42 smithi090 ceph-mon[27825]: from='osd.2 [v2:172.21.15.149:6800/3257067419,v1:172.21.15.149:6801/3257067419]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-04-24T09:13:42.772 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:42 smithi090 ceph-mon[27825]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-04-24T09:13:42.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:42 smithi149 ceph-mon[32592]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T09:13:42.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:42 smithi149 ceph-mon[32592]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-24T09:13:42.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:42 smithi149 ceph-mon[32592]: osdmap e16: 3 total, 2 up, 3 in 2024-04-24T09:13:42.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:42 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T09:13:42.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:42 smithi149 ceph-mon[32592]: from='osd.2 [v2:172.21.15.149:6800/3257067419,v1:172.21.15.149:6801/3257067419]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-04-24T09:13:42.863 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:42 smithi149 ceph-mon[32592]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-04-24T09:13:42.863 INFO:journalctl@ceph.osd.2.smithi149.stdout:Apr 24 09:13:42 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-2[37755]: 2024-04-24T09:13:42.529+0000 7fb327822700 -1 osd.2 0 waiting for initial osdmap 2024-04-24T09:13:42.863 INFO:journalctl@ceph.osd.2.smithi149.stdout:Apr 24 09:13:42 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-2[37755]: 2024-04-24T09:13:42.533+0000 7fb323615700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T09:13:43.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:43 smithi090 ceph-mon[27825]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-04-24T09:13:43.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:43 smithi090 ceph-mon[27825]: osdmap e17: 3 total, 2 up, 3 in 2024-04-24T09:13:43.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:43 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T09:13:43.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:43 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T09:13:43.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:43 smithi149 ceph-mon[32592]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-04-24T09:13:43.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:43 smithi149 ceph-mon[32592]: osdmap e17: 3 total, 2 up, 3 in 2024-04-24T09:13:43.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:43 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T09:13:43.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:43 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T09:13:43.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:43 smithi114 ceph-mon[32501]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-04-24T09:13:43.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:43 smithi114 ceph-mon[32501]: osdmap e17: 3 total, 2 up, 3 in 2024-04-24T09:13:43.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:43 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T09:13:43.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:43 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T09:13:44.283 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-24T09:13:44.283 INFO:teuthology.orchestra.run.smithi090.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-24T09:13:44.283 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10+0 records in 2024-04-24T09:13:44.283 INFO:teuthology.orchestra.run.smithi090.stderr:10+0 records out 2024-04-24T09:13:44.283 INFO:teuthology.orchestra.run.smithi090.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0334078 s, 314 MB/s 2024-04-24T09:13:44.283 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping successful for: 2024-04-24T09:13:44.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:44 smithi090 ceph-mon[27825]: purged_snaps scrub starts 2024-04-24T09:13:44.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:44 smithi090 ceph-mon[27825]: purged_snaps scrub ok 2024-04-24T09:13:44.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:44 smithi090 ceph-mon[27825]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T09:13:44.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:44 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T09:13:44.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:44 smithi090 ceph-mon[27825]: osd.2 [v2:172.21.15.149:6800/3257067419,v1:172.21.15.149:6801/3257067419] boot 2024-04-24T09:13:44.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:44 smithi090 ceph-mon[27825]: osdmap e18: 3 total, 3 up, 3 in 2024-04-24T09:13:44.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:44 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T09:13:44.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:44 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:44.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:44 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:44.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:44 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:13:44.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:44 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:44.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:44 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:13:44.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:44 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:13:44.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:44 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:44.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:44 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:13:44.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:44 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:13:44.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:44 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:13:44.854 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:44 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:44.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:44 smithi149 ceph-mon[32592]: purged_snaps scrub starts 2024-04-24T09:13:44.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:44 smithi149 ceph-mon[32592]: purged_snaps scrub ok 2024-04-24T09:13:44.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:44 smithi149 ceph-mon[32592]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T09:13:44.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:44 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T09:13:44.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:44 smithi149 ceph-mon[32592]: osd.2 [v2:172.21.15.149:6800/3257067419,v1:172.21.15.149:6801/3257067419] boot 2024-04-24T09:13:44.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:44 smithi149 ceph-mon[32592]: osdmap e18: 3 total, 3 up, 3 in 2024-04-24T09:13:44.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:44 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T09:13:44.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:44 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:44.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:44 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:44.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:44 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:13:44.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:44 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:44.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:44 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:13:44.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:44 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:13:44.863 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:44 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:44.863 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:44 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:13:44.863 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:44 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:13:44.863 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:44 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:13:44.863 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:44 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:44.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:44 smithi114 ceph-mon[32501]: purged_snaps scrub starts 2024-04-24T09:13:44.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:44 smithi114 ceph-mon[32501]: purged_snaps scrub ok 2024-04-24T09:13:44.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:44 smithi114 ceph-mon[32501]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-24T09:13:44.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:44 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T09:13:44.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:44 smithi114 ceph-mon[32501]: osd.2 [v2:172.21.15.149:6800/3257067419,v1:172.21.15.149:6801/3257067419] boot 2024-04-24T09:13:44.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:44 smithi114 ceph-mon[32501]: osdmap e18: 3 total, 3 up, 3 in 2024-04-24T09:13:44.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:44 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T09:13:44.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:44 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:44.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:44 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:44.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:44 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:13:44.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:44 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:44.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:44 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:13:44.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:44 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:13:44.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:44 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:44.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:44 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:13:44.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:44 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:13:44.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:44 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:13:44.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:44 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:45.146 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi090:vg_nvme/lv_3 2024-04-24T09:13:45.587 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:13:45.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:45 smithi090 ceph-mon[27825]: Detected new or changed devices on smithi149 2024-04-24T09:13:45.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:45 smithi090 ceph-mon[27825]: Adjusting osd_memory_target on smithi149 to 5816M 2024-04-24T09:13:45.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:45 smithi090 ceph-mon[27825]: osdmap e19: 3 total, 3 up, 3 in 2024-04-24T09:13:45.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:45 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-24T09:13:45.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:45 smithi149 ceph-mon[32592]: Detected new or changed devices on smithi149 2024-04-24T09:13:45.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:45 smithi149 ceph-mon[32592]: Adjusting osd_memory_target on smithi149 to 5816M 2024-04-24T09:13:45.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:45 smithi149 ceph-mon[32592]: osdmap e19: 3 total, 3 up, 3 in 2024-04-24T09:13:45.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:45 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-24T09:13:45.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:45 smithi114 ceph-mon[32501]: Detected new or changed devices on smithi149 2024-04-24T09:13:45.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:45 smithi114 ceph-mon[32501]: Adjusting osd_memory_target on smithi149 to 5816M 2024-04-24T09:13:45.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:45 smithi114 ceph-mon[32501]: osdmap e19: 3 total, 3 up, 3 in 2024-04-24T09:13:45.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:45 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-24T09:13:46.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:46 smithi090 ceph-mon[27825]: pgmap v93: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T09:13:46.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:46 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-24T09:13:46.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:46 smithi090 ceph-mon[27825]: osdmap e20: 3 total, 3 up, 3 in 2024-04-24T09:13:46.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:46 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T09:13:46.853 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 09:13:46 smithi090 sudo[46756]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T09:13:46.853 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 09:13:46 smithi090 sudo[46756]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T09:13:46.853 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 09:13:46 smithi090 sudo[46756]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T09:13:46.853 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 09:13:46 smithi090 sudo[46756]: pam_unix(sudo:session): session closed for user root 2024-04-24T09:13:46.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:46 smithi149 ceph-mon[32592]: pgmap v93: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T09:13:46.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:46 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-24T09:13:46.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:46 smithi149 ceph-mon[32592]: osdmap e20: 3 total, 3 up, 3 in 2024-04-24T09:13:46.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:46 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T09:13:46.924 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:46 smithi114 ceph-mon[32501]: pgmap v93: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T09:13:46.924 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:46 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-24T09:13:46.924 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:46 smithi114 ceph-mon[32501]: osdmap e20: 3 total, 3 up, 3 in 2024-04-24T09:13:46.924 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:46 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T09:13:47.112 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 09:13:46 smithi090 sudo[46759]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T09:13:47.112 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 09:13:46 smithi090 sudo[46759]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T09:13:47.112 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 09:13:46 smithi090 sudo[46759]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T09:13:47.112 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 09:13:46 smithi090 sudo[46759]: pam_unix(sudo:session): session closed for user root 2024-04-24T09:13:47.176 INFO:journalctl@ceph.osd.1.smithi114.stdout:Apr 24 09:13:46 smithi114 sudo[41186]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T09:13:47.176 INFO:journalctl@ceph.osd.1.smithi114.stdout:Apr 24 09:13:46 smithi114 sudo[41186]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T09:13:47.176 INFO:journalctl@ceph.osd.1.smithi114.stdout:Apr 24 09:13:46 smithi114 sudo[41186]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T09:13:47.176 INFO:journalctl@ceph.osd.1.smithi114.stdout:Apr 24 09:13:46 smithi114 sudo[41186]: pam_unix(sudo:session): session closed for user root 2024-04-24T09:13:47.176 INFO:journalctl@ceph.osd.1.smithi114.stdout:Apr 24 09:13:46 smithi114 sudo[41189]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T09:13:47.176 INFO:journalctl@ceph.osd.1.smithi114.stdout:Apr 24 09:13:46 smithi114 sudo[41189]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T09:13:47.176 INFO:journalctl@ceph.osd.1.smithi114.stdout:Apr 24 09:13:46 smithi114 sudo[41189]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T09:13:47.176 INFO:journalctl@ceph.osd.1.smithi114.stdout:Apr 24 09:13:46 smithi114 sudo[41189]: pam_unix(sudo:session): session closed for user root 2024-04-24T09:13:47.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:47 smithi149 sudo[41818]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T09:13:47.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:47 smithi149 sudo[41818]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T09:13:47.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:47 smithi149 sudo[41818]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T09:13:47.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:47 smithi149 sudo[41818]: pam_unix(sudo:session): session closed for user root 2024-04-24T09:13:47.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:47 smithi149 sudo[41821]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T09:13:47.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:47 smithi149 sudo[41821]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T09:13:47.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:47 smithi149 sudo[41821]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T09:13:47.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:47 smithi149 sudo[41821]: pam_unix(sudo:session): session closed for user root 2024-04-24T09:13:47.362 INFO:journalctl@ceph.osd.2.smithi149.stdout:Apr 24 09:13:47 smithi149 sudo[41812]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T09:13:47.362 INFO:journalctl@ceph.osd.2.smithi149.stdout:Apr 24 09:13:47 smithi149 sudo[41812]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T09:13:47.363 INFO:journalctl@ceph.osd.2.smithi149.stdout:Apr 24 09:13:47 smithi149 sudo[41812]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T09:13:47.363 INFO:journalctl@ceph.osd.2.smithi149.stdout:Apr 24 09:13:47 smithi149 sudo[41812]: pam_unix(sudo:session): session closed for user root 2024-04-24T09:13:47.363 INFO:journalctl@ceph.osd.2.smithi149.stdout:Apr 24 09:13:47 smithi149 sudo[41815]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T09:13:47.363 INFO:journalctl@ceph.osd.2.smithi149.stdout:Apr 24 09:13:47 smithi149 sudo[41815]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T09:13:47.363 INFO:journalctl@ceph.osd.2.smithi149.stdout:Apr 24 09:13:47 smithi149 sudo[41815]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T09:13:47.363 INFO:journalctl@ceph.osd.2.smithi149.stdout:Apr 24 09:13:47 smithi149 sudo[41815]: pam_unix(sudo:session): session closed for user root 2024-04-24T09:13:47.393 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:47 smithi090 sudo[46762]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T09:13:47.393 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:47 smithi090 sudo[46762]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T09:13:47.393 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:47 smithi090 sudo[46762]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T09:13:47.393 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:47 smithi090 sudo[46762]: pam_unix(sudo:session): session closed for user root 2024-04-24T09:13:47.393 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:47 smithi090 sudo[46767]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T09:13:47.393 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:47 smithi090 sudo[46767]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T09:13:47.393 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:47 smithi090 sudo[46767]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T09:13:47.393 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:47 smithi090 sudo[46767]: pam_unix(sudo:session): session closed for user root 2024-04-24T09:13:47.587 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:47 smithi114 sudo[41192]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T09:13:47.587 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:47 smithi114 sudo[41192]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T09:13:47.587 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:47 smithi114 sudo[41192]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T09:13:47.587 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:47 smithi114 sudo[41192]: pam_unix(sudo:session): session closed for user root 2024-04-24T09:13:47.587 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:47 smithi114 sudo[41195]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T09:13:47.587 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:47 smithi114 sudo[41195]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T09:13:47.587 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:47 smithi114 sudo[41195]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T09:13:47.587 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:47 smithi114 sudo[41195]: pam_unix(sudo:session): session closed for user root 2024-04-24T09:13:47.587 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:47 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-24T09:13:47.587 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:47 smithi114 ceph-mon[32501]: osdmap e21: 3 total, 3 up, 3 in 2024-04-24T09:13:47.587 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:47 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:47.588 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:47 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:47.588 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:47 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:47.588 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:47 smithi114 ceph-mon[32501]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T09:13:47.588 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:47 smithi114 ceph-mon[32501]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T09:13:47.588 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:47 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T09:13:47.588 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:47 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T09:13:47.588 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:47 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T09:13:47.588 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:47 smithi114 ceph-mon[32501]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T09:13:47.588 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:47 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T09:13:47.588 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:47 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T09:13:47.588 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:47 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T09:13:47.588 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:47 smithi114 ceph-mon[32501]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T09:13:47.588 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:47 smithi114 ceph-mon[32501]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T09:13:47.588 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:47 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T09:13:47.588 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:47 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T09:13:47.589 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:47 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T09:13:47.589 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:47 smithi114 ceph-mon[32501]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T09:13:47.740 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:47 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-24T09:13:47.741 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:47 smithi090 ceph-mon[27825]: osdmap e21: 3 total, 3 up, 3 in 2024-04-24T09:13:47.741 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:47 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:47.741 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:47 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:47.741 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:47 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:47.741 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:47 smithi090 ceph-mon[27825]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T09:13:47.741 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:47 smithi090 ceph-mon[27825]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T09:13:47.741 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:47 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T09:13:47.741 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:47 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T09:13:47.741 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:47 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T09:13:47.741 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:47 smithi090 ceph-mon[27825]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T09:13:47.741 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:47 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T09:13:47.741 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:47 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T09:13:47.741 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:47 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T09:13:47.741 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:47 smithi090 ceph-mon[27825]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T09:13:47.742 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:47 smithi090 ceph-mon[27825]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T09:13:47.742 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:47 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T09:13:47.742 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:47 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T09:13:47.742 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:47 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T09:13:47.742 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:47 smithi090 ceph-mon[27825]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T09:13:47.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:47 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-24T09:13:47.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:47 smithi149 ceph-mon[32592]: osdmap e21: 3 total, 3 up, 3 in 2024-04-24T09:13:47.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:47 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:47.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:47 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:47.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:47 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:13:47.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:47 smithi149 ceph-mon[32592]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T09:13:47.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:47 smithi149 ceph-mon[32592]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T09:13:47.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:47 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T09:13:47.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:47 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T09:13:47.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:47 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T09:13:47.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:47 smithi149 ceph-mon[32592]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T09:13:47.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:47 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T09:13:47.863 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:47 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T09:13:47.863 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:47 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T09:13:47.863 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:47 smithi149 ceph-mon[32592]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T09:13:47.863 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:47 smithi149 ceph-mon[32592]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T09:13:47.863 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:47 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T09:13:47.863 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:47 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T09:13:47.863 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:47 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T09:13:47.863 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:47 smithi149 ceph-mon[32592]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T09:13:47.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:13:47.908+0000 7f9ec9200700 1 -- 172.21.15.90:0/3302622728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ec410a950 msgr2=0x7f9ec410cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:47.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:13:47.908+0000 7f9ec9200700 1 --2- 172.21.15.90:0/3302622728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ec410a950 0x7f9ec410cde0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc00b0a0 tx=0x7f9ebc00b3b0 comp rx=0 tx=0).stop 2024-04-24T09:13:47.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:13:47.908+0000 7f9ec9200700 1 -- 172.21.15.90:0/3302622728 shutdown_connections 2024-04-24T09:13:47.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:13:47.908+0000 7f9ec9200700 1 --2- 172.21.15.90:0/3302622728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ec410a950 0x7f9ec410cde0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:47.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:13:47.908+0000 7f9ec9200700 1 --2- 172.21.15.90:0/3302622728 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ec4076ac0 0x7f9ec4074f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:47.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:13:47.908+0000 7f9ec9200700 1 --2- 172.21.15.90:0/3302622728 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ec40754f0 0x7f9ec4075970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:47.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:13:47.908+0000 7f9ec9200700 1 -- 172.21.15.90:0/3302622728 >> 172.21.15.90:0/3302622728 conn(0x7f9ec40fe030 msgr2=0x7f9ec4100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:47.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:13:47.908+0000 7f9ec9200700 1 -- 172.21.15.90:0/3302622728 shutdown_connections 2024-04-24T09:13:47.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:13:47.908+0000 7f9ec9200700 1 -- 172.21.15.90:0/3302622728 wait complete. 2024-04-24T09:13:47.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:13:47.909+0000 7f9ec9200700 1 Processor -- start 2024-04-24T09:13:47.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:13:47.910+0000 7f9ec9200700 1 -- start start 2024-04-24T09:13:47.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:13:47.910+0000 7f9ec9200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ec40754f0 0x7f9ec419ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:47.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:13:47.910+0000 7f9ec9200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ec4076ac0 0x7f9ec419b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:47.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:13:47.910+0000 7f9ec9200700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ec410a950 0x7f9ec41a2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:47.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:13:47.910+0000 7f9ec9200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec4110810 con 0x7f9ec40754f0 2024-04-24T09:13:47.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:13:47.910+0000 7f9ec9200700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec4110690 con 0x7f9ec410a950 2024-04-24T09:13:47.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:13:47.910+0000 7f9ec9200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec4110990 con 0x7f9ec4076ac0 2024-04-24T09:13:47.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:13:47.910+0000 7f9ec3400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ec40754f0 0x7f9ec419ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:47.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:13:47.910+0000 7f9ec3400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ec40754f0 0x7f9ec419ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53548/0 (socket says 172.21.15.90:53548) 2024-04-24T09:13:47.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:13:47.910+0000 7f9ec3400700 1 -- 172.21.15.90:0/1934034680 learned_addr learned my addr 172.21.15.90:0/1934034680 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:13:47.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:13:47.910+0000 7f9ec2a00700 1 --2- 172.21.15.90:0/1934034680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ec4076ac0 0x7f9ec419b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:47.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:13:47.910+0000 7f9ec3e00700 1 --2- 172.21.15.90:0/1934034680 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ec410a950 0x7f9ec41a2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:47.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:13:47.911+0000 7f9ec3400700 1 -- 172.21.15.90:0/1934034680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ec4076ac0 msgr2=0x7f9ec419b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:47.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:13:47.911+0000 7f9ec3400700 1 --2- 172.21.15.90:0/1934034680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ec4076ac0 0x7f9ec419b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:47.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:13:47.911+0000 7f9ec3400700 1 -- 172.21.15.90:0/1934034680 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ec410a950 msgr2=0x7f9ec41a2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:47.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:13:47.911+0000 7f9ec3400700 1 --2- 172.21.15.90:0/1934034680 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ec410a950 0x7f9ec41a2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:47.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:13:47.911+0000 7f9ec3400700 1 -- 172.21.15.90:0/1934034680 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ec41a29a0 con 0x7f9ec40754f0 2024-04-24T09:13:47.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:13:47.911+0000 7f9ec3400700 1 --2- 172.21.15.90:0/1934034680 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ec40754f0 0x7f9ec419ad40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9eb400eb60 tx=0x7f9eb400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:47.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:13:47.911+0000 7f9ebbe00700 1 -- 172.21.15.90:0/1934034680 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb400cbd0 con 0x7f9ec40754f0 2024-04-24T09:13:47.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:13:47.911+0000 7f9ebbe00700 1 -- 172.21.15.90:0/1934034680 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9eb4004500 con 0x7f9ec40754f0 2024-04-24T09:13:47.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:13:47.911+0000 7f9ec9200700 1 -- 172.21.15.90:0/1934034680 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ec41a2c90 con 0x7f9ec40754f0 2024-04-24T09:13:47.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:13:47.911+0000 7f9ebbe00700 1 -- 172.21.15.90:0/1934034680 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb4010430 con 0x7f9ec40754f0 2024-04-24T09:13:47.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:13:47.912+0000 7f9ec9200700 1 -- 172.21.15.90:0/1934034680 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ec41a3180 con 0x7f9ec40754f0 2024-04-24T09:13:47.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:13:47.913+0000 7f9ec9200700 1 -- 172.21.15.90:0/1934034680 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ec40697e0 con 0x7f9ec40754f0 2024-04-24T09:13:47.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:13:47.915+0000 7f9ebbe00700 1 -- 172.21.15.90:0/1934034680 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f9eb4003680 con 0x7f9ec40754f0 2024-04-24T09:13:47.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:13:47.916+0000 7f9ebbe00700 1 --2- 172.21.15.90:0/1934034680 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f9ea80a3bb0 0x7f9ea80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:47.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:13:47.917+0000 7f9ebbe00700 1 -- 172.21.15.90:0/1934034680 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 3237+0+0 (secure 0 0 0) 0x7f9eb40cf230 con 0x7f9ec40754f0 2024-04-24T09:13:47.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:13:47.917+0000 7f9ec2a00700 1 --2- 172.21.15.90:0/1934034680 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f9ea80a3bb0 0x7f9ea80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:47.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:13:47.917+0000 7f9ec2a00700 1 --2- 172.21.15.90:0/1934034680 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f9ea80a3bb0 0x7f9ea80a6070 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0009c60 tx=0x7f9eb0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:47.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:13:47.920+0000 7f9ebbe00700 1 -- 172.21.15.90:0/1934034680 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9eb400cd70 con 0x7f9ec40754f0 2024-04-24T09:13:48.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:13:48.104+0000 7f9ebbe00700 1 -- 172.21.15.90:0/1934034680 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9eb4099020 con 0x7f9ec40754f0 2024-04-24T09:13:48.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:13:48.166+0000 7f9ec9200700 1 -- 172.21.15.90:0/1934034680 --> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f9ec4062f30 con 0x7f9ea80a3bb0 2024-04-24T09:13:48.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:48 smithi090 ceph-mon[27825]: pgmap v96: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T09:13:48.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:48 smithi090 ceph-mon[27825]: osdmap e22: 3 total, 3 up, 3 in 2024-04-24T09:13:48.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:48 smithi090 ceph-mon[27825]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-24T09:13:48.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:48 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:13:48.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:48 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:13:48.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:48 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:13:48.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:48 smithi149 ceph-mon[32592]: pgmap v96: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T09:13:48.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:48 smithi149 ceph-mon[32592]: osdmap e22: 3 total, 3 up, 3 in 2024-04-24T09:13:48.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:48 smithi149 ceph-mon[32592]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-24T09:13:48.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:48 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:13:48.863 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:48 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:13:48.863 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:48 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:13:48.930 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:48 smithi114 ceph-mon[32501]: pgmap v96: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T09:13:48.930 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:48 smithi114 ceph-mon[32501]: osdmap e22: 3 total, 3 up, 3 in 2024-04-24T09:13:48.930 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:48 smithi114 ceph-mon[32501]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-24T09:13:48.930 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:48 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:13:48.930 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:48 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:13:48.931 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:48 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:13:49.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:49 smithi090 ceph-mon[27825]: from='client.14250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:13:49.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:49 smithi149 ceph-mon[32592]: from='client.14250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:13:49.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:49 smithi114 ceph-mon[32501]: from='client.14250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:13:50.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:50 smithi114 ceph-mon[32501]: pgmap v98: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T09:13:51.036 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:50 smithi090 ceph-mon[27825]: pgmap v98: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T09:13:51.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:50 smithi149 ceph-mon[32592]: pgmap v98: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T09:13:52.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:52 smithi149 ceph-mon[32592]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T09:13:52.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:52 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/739342148' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9377ab6e-5996-45fb-93a9-475a2e5f17a7"}]: dispatch 2024-04-24T09:13:52.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:52 smithi149 ceph-mon[32592]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9377ab6e-5996-45fb-93a9-475a2e5f17a7"}]: dispatch 2024-04-24T09:13:52.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:52 smithi149 ceph-mon[32592]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9377ab6e-5996-45fb-93a9-475a2e5f17a7"}]': finished 2024-04-24T09:13:52.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:52 smithi149 ceph-mon[32592]: osdmap e23: 4 total, 3 up, 4 in 2024-04-24T09:13:52.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:52 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T09:13:52.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:52 smithi114 ceph-mon[32501]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T09:13:52.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:52 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/739342148' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9377ab6e-5996-45fb-93a9-475a2e5f17a7"}]: dispatch 2024-04-24T09:13:52.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:52 smithi114 ceph-mon[32501]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9377ab6e-5996-45fb-93a9-475a2e5f17a7"}]: dispatch 2024-04-24T09:13:52.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:52 smithi114 ceph-mon[32501]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9377ab6e-5996-45fb-93a9-475a2e5f17a7"}]': finished 2024-04-24T09:13:52.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:52 smithi114 ceph-mon[32501]: osdmap e23: 4 total, 3 up, 4 in 2024-04-24T09:13:52.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:52 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T09:13:53.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:52 smithi090 ceph-mon[27825]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T09:13:53.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:52 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/739342148' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9377ab6e-5996-45fb-93a9-475a2e5f17a7"}]: dispatch 2024-04-24T09:13:53.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:52 smithi090 ceph-mon[27825]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9377ab6e-5996-45fb-93a9-475a2e5f17a7"}]: dispatch 2024-04-24T09:13:53.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:52 smithi090 ceph-mon[27825]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9377ab6e-5996-45fb-93a9-475a2e5f17a7"}]': finished 2024-04-24T09:13:53.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:52 smithi090 ceph-mon[27825]: osdmap e23: 4 total, 3 up, 4 in 2024-04-24T09:13:53.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:52 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T09:13:53.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:53 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/3076084621' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:13:54.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:53 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/3076084621' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:13:54.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:53 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/3076084621' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:13:54.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:54 smithi114 ceph-mon[32501]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T09:13:55.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:54 smithi090 ceph-mon[27825]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T09:13:55.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:54 smithi149 ceph-mon[32592]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T09:13:56.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:56 smithi114 ceph-mon[32501]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T09:13:57.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:56 smithi090 ceph-mon[27825]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T09:13:57.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:56 smithi149 ceph-mon[32592]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T09:13:58.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:13:58 smithi090 ceph-mon[27825]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T09:13:58.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:13:58 smithi114 ceph-mon[32501]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T09:13:59.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:13:58 smithi149 ceph-mon[32592]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T09:14:00.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:00 smithi114 ceph-mon[32501]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T09:14:01.029 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:00 smithi090 ceph-mon[27825]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T09:14:01.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:00 smithi149 ceph-mon[32592]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T09:14:01.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:01 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-24T09:14:01.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:01 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:14:02.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:01 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-24T09:14:02.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:01 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:14:02.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:01 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-24T09:14:02.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:01 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:14:02.698 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:02 smithi090 ceph-mon[27825]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T09:14:02.699 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:02 smithi090 ceph-mon[27825]: Deploying daemon osd.3 on smithi090 2024-04-24T09:14:02.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:02 smithi114 ceph-mon[32501]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T09:14:02.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:02 smithi114 ceph-mon[32501]: Deploying daemon osd.3 on smithi090 2024-04-24T09:14:03.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:02 smithi149 ceph-mon[32592]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T09:14:03.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:02 smithi149 ceph-mon[32592]: Deploying daemon osd.3 on smithi090 2024-04-24T09:14:04.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:04 smithi090 ceph-mon[27825]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T09:14:04.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:04 smithi114 ceph-mon[32501]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T09:14:05.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:04 smithi149 ceph-mon[32592]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T09:14:06.835 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:06 smithi090 ceph-mon[27825]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T09:14:06.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:06 smithi114 ceph-mon[32501]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T09:14:07.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:06 smithi149 ceph-mon[32592]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T09:14:08.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:07 smithi090 ceph-mon[27825]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T09:14:08.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:07 smithi149 ceph-mon[32592]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T09:14:08.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:07 smithi114 ceph-mon[32501]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T09:14:10.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:10 smithi090 ceph-mon[27825]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T09:14:10.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:10 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:10.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:10 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:10.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:10 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:14:10.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:10 smithi149 ceph-mon[32592]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T09:14:10.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:10 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:10.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:10 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:10.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:10 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:14:10.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:10 smithi114 ceph-mon[32501]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T09:14:10.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:10 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:10.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:10 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:10.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:10 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:14:12.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:12 smithi090 ceph-mon[27825]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T09:14:12.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:12 smithi149 ceph-mon[32592]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T09:14:12.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:12 smithi114 ceph-mon[32501]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T09:14:14.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:14 smithi090 ceph-mon[27825]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T09:14:14.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:14 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:14.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:14 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:14.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:14 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:14:14.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:14 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:14:14.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:14 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:14.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:14 smithi090 ceph-mon[27825]: from='osd.3 [v2:172.21.15.90:6810/3068524970,v1:172.21.15.90:6811/3068524970]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T09:14:14.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:14 smithi090 ceph-mon[27825]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T09:14:14.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:14 smithi149 ceph-mon[32592]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T09:14:14.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:14 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:14.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:14 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:14.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:14 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:14:14.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:14 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:14:14.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:14 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:14.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:14 smithi149 ceph-mon[32592]: from='osd.3 [v2:172.21.15.90:6810/3068524970,v1:172.21.15.90:6811/3068524970]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T09:14:14.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:14 smithi149 ceph-mon[32592]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T09:14:14.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:14 smithi114 ceph-mon[32501]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T09:14:14.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:14 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:14.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:14 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:14.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:14 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:14:14.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:14 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:14:14.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:14 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:14.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:14 smithi114 ceph-mon[32501]: from='osd.3 [v2:172.21.15.90:6810/3068524970,v1:172.21.15.90:6811/3068524970]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T09:14:14.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:14 smithi114 ceph-mon[32501]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T09:14:15.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:14:15.916+0000 7f9ebbe00700 1 -- 172.21.15.90:0/1934034680 <== mgr.14152 v2:172.21.15.90:6800/4261662956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9ec4062f30 con 0x7f9ea80a3bb0 2024-04-24T09:14:15.917 INFO:teuthology.orchestra.run.smithi090.stdout:Created osd(s) 3 on host 'smithi090' 2024-04-24T09:14:15.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:14:15.921+0000 7f9ec9200700 1 -- 172.21.15.90:0/1934034680 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f9ea80a3bb0 msgr2=0x7f9ea80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:15.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:14:15.921+0000 7f9ec9200700 1 --2- 172.21.15.90:0/1934034680 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f9ea80a3bb0 0x7f9ea80a6070 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0009c60 tx=0x7f9eb0009260 comp rx=0 tx=0).stop 2024-04-24T09:14:15.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:14:15.922+0000 7f9ec9200700 1 -- 172.21.15.90:0/1934034680 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ec40754f0 msgr2=0x7f9ec419ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:15.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:14:15.922+0000 7f9ec9200700 1 --2- 172.21.15.90:0/1934034680 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ec40754f0 0x7f9ec419ad40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9eb400eb60 tx=0x7f9eb400ee70 comp rx=0 tx=0).stop 2024-04-24T09:14:15.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:14:15.922+0000 7f9ec9200700 1 -- 172.21.15.90:0/1934034680 shutdown_connections 2024-04-24T09:14:15.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:14:15.922+0000 7f9ec9200700 1 --2- 172.21.15.90:0/1934034680 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ec410a950 0x7f9ec41a2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:15.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:14:15.922+0000 7f9ec9200700 1 --2- 172.21.15.90:0/1934034680 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f9ea80a3bb0 0x7f9ea80a6070 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:15.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:14:15.922+0000 7f9ec9200700 1 --2- 172.21.15.90:0/1934034680 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ec40754f0 0x7f9ec419ad40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:15.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:14:15.922+0000 7f9ec9200700 1 --2- 172.21.15.90:0/1934034680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ec4076ac0 0x7f9ec419b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:15.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:14:15.922+0000 7f9ec9200700 1 -- 172.21.15.90:0/1934034680 >> 172.21.15.90:0/1934034680 conn(0x7f9ec40fe030 msgr2=0x7f9ec41068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:15.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:14:15.922+0000 7f9ec9200700 1 -- 172.21.15.90:0/1934034680 shutdown_connections 2024-04-24T09:14:15.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:14:15.922+0000 7f9ec9200700 1 -- 172.21.15.90:0/1934034680 wait complete. 2024-04-24T09:14:16.069 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:16 smithi090 ceph-mon[27825]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-24T09:14:16.069 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:16 smithi090 ceph-mon[27825]: from='osd.3 [v2:172.21.15.90:6810/3068524970,v1:172.21.15.90:6811/3068524970]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-24T09:14:16.069 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:16 smithi090 ceph-mon[27825]: osdmap e24: 4 total, 3 up, 4 in 2024-04-24T09:14:16.070 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:16 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T09:14:16.070 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:16 smithi090 ceph-mon[27825]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-24T09:14:16.070 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:16 smithi090 ceph-mon[27825]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T09:14:16.070 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:16 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:16.070 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:16 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:16.070 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:16 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:14:16.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:16 smithi149 ceph-mon[32592]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-24T09:14:16.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:16 smithi149 ceph-mon[32592]: from='osd.3 [v2:172.21.15.90:6810/3068524970,v1:172.21.15.90:6811/3068524970]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-24T09:14:16.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:16 smithi149 ceph-mon[32592]: osdmap e24: 4 total, 3 up, 4 in 2024-04-24T09:14:16.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:16 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T09:14:16.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:16 smithi149 ceph-mon[32592]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-24T09:14:16.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:16 smithi149 ceph-mon[32592]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T09:14:16.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:16 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:16.363 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:16 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:16.363 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:16 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:14:16.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:16 smithi114 ceph-mon[32501]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-24T09:14:16.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:16 smithi114 ceph-mon[32501]: from='osd.3 [v2:172.21.15.90:6810/3068524970,v1:172.21.15.90:6811/3068524970]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-24T09:14:16.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:16 smithi114 ceph-mon[32501]: osdmap e24: 4 total, 3 up, 4 in 2024-04-24T09:14:16.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:16 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T09:14:16.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:16 smithi114 ceph-mon[32501]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-24T09:14:16.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:16 smithi114 ceph-mon[32501]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-24T09:14:16.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:16 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:16.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:16 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:16.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:16 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:14:17.132 DEBUG:teuthology.orchestra.run.smithi090:osd.3> sudo journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@osd.3.service 2024-04-24T09:14:17.139 INFO:tasks.cephadm:Deploying osd.4 on smithi114 with /dev/vg_nvme/lv_3... 2024-04-24T09:14:17.140 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-24T09:14:17.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:17 smithi090 ceph-mon[27825]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-04-24T09:14:17.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:17 smithi090 ceph-mon[27825]: osdmap e25: 4 total, 3 up, 4 in 2024-04-24T09:14:17.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:17 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T09:14:17.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:17 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T09:14:17.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:17 smithi090 ceph-mon[27825]: osd.3 [v2:172.21.15.90:6810/3068524970,v1:172.21.15.90:6811/3068524970] boot 2024-04-24T09:14:17.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:17 smithi090 ceph-mon[27825]: osdmap e26: 4 total, 4 up, 4 in 2024-04-24T09:14:17.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:17 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T09:14:17.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:17 smithi149 ceph-mon[32592]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-04-24T09:14:17.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:17 smithi149 ceph-mon[32592]: osdmap e25: 4 total, 3 up, 4 in 2024-04-24T09:14:17.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:17 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T09:14:17.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:17 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T09:14:17.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:17 smithi149 ceph-mon[32592]: osd.3 [v2:172.21.15.90:6810/3068524970,v1:172.21.15.90:6811/3068524970] boot 2024-04-24T09:14:17.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:17 smithi149 ceph-mon[32592]: osdmap e26: 4 total, 4 up, 4 in 2024-04-24T09:14:17.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:17 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T09:14:17.382 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T09:14:17.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:17 smithi114 ceph-mon[32501]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-04-24T09:14:17.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:17 smithi114 ceph-mon[32501]: osdmap e25: 4 total, 3 up, 4 in 2024-04-24T09:14:17.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:17 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T09:14:17.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:17 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T09:14:17.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:17 smithi114 ceph-mon[32501]: osd.3 [v2:172.21.15.90:6810/3068524970,v1:172.21.15.90:6811/3068524970] boot 2024-04-24T09:14:17.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:17 smithi114 ceph-mon[32501]: osdmap e26: 4 total, 4 up, 4 in 2024-04-24T09:14:17.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:17 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T09:14:18.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:18 smithi090 ceph-mon[27825]: purged_snaps scrub starts 2024-04-24T09:14:18.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:18 smithi090 ceph-mon[27825]: purged_snaps scrub ok 2024-04-24T09:14:18.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:18 smithi090 ceph-mon[27825]: osdmap e27: 4 total, 4 up, 4 in 2024-04-24T09:14:18.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:18 smithi090 ceph-mon[27825]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-24T09:14:18.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:18 smithi149 ceph-mon[32592]: purged_snaps scrub starts 2024-04-24T09:14:18.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:18 smithi149 ceph-mon[32592]: purged_snaps scrub ok 2024-04-24T09:14:18.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:18 smithi149 ceph-mon[32592]: osdmap e27: 4 total, 4 up, 4 in 2024-04-24T09:14:18.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:18 smithi149 ceph-mon[32592]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-24T09:14:18.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:18 smithi114 ceph-mon[32501]: purged_snaps scrub starts 2024-04-24T09:14:18.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:18 smithi114 ceph-mon[32501]: purged_snaps scrub ok 2024-04-24T09:14:18.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:18 smithi114 ceph-mon[32501]: osdmap e27: 4 total, 4 up, 4 in 2024-04-24T09:14:18.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:18 smithi114 ceph-mon[32501]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-24T09:14:19.231 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:19 smithi090 ceph-mon[27825]: osdmap e28: 4 total, 4 up, 4 in 2024-04-24T09:14:19.284 INFO:teuthology.orchestra.run.smithi114.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-24T09:14:19.284 INFO:teuthology.orchestra.run.smithi114.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-24T09:14:19.284 INFO:teuthology.orchestra.run.smithi114.stderr: stderr: 10+0 records in 2024-04-24T09:14:19.284 INFO:teuthology.orchestra.run.smithi114.stderr:10+0 records out 2024-04-24T09:14:19.284 INFO:teuthology.orchestra.run.smithi114.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114837 s, 913 MB/s 2024-04-24T09:14:19.285 INFO:teuthology.orchestra.run.smithi114.stderr:--> Zapping successful for: 2024-04-24T09:14:19.415 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:19 smithi114 ceph-mon[32501]: osdmap e28: 4 total, 4 up, 4 in 2024-04-24T09:14:19.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:19 smithi149 ceph-mon[32592]: osdmap e28: 4 total, 4 up, 4 in 2024-04-24T09:14:19.949 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi114:vg_nvme/lv_3 2024-04-24T09:14:20.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:20 smithi114 ceph-mon[32501]: osdmap e29: 4 total, 4 up, 4 in 2024-04-24T09:14:20.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:20 smithi114 ceph-mon[32501]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-24T09:14:20.194 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T09:14:20.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:20 smithi090 ceph-mon[27825]: osdmap e29: 4 total, 4 up, 4 in 2024-04-24T09:14:20.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:20 smithi090 ceph-mon[27825]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-24T09:14:20.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:20 smithi149 ceph-mon[32592]: osdmap e29: 4 total, 4 up, 4 in 2024-04-24T09:14:20.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:20 smithi149 ceph-mon[32592]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-24T09:14:21.288 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:21 smithi114 ceph-mon[32501]: osdmap e30: 4 total, 4 up, 4 in 2024-04-24T09:14:21.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:21 smithi090 ceph-mon[27825]: osdmap e30: 4 total, 4 up, 4 in 2024-04-24T09:14:21.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:21 smithi149 ceph-mon[32592]: osdmap e30: 4 total, 4 up, 4 in 2024-04-24T09:14:21.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:21.802+0000 7fc11b276700 1 -- 172.21.15.114:0/3002112341 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc114103d60 msgr2=0x7fc1141041c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:21.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:21.802+0000 7fc11b276700 1 --2- 172.21.15.114:0/3002112341 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc114103d60 0x7fc1141041c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc1040099c0 tx=0x7fc104009cd0 comp rx=0 tx=0).stop 2024-04-24T09:14:21.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:21.802+0000 7fc11b276700 1 -- 172.21.15.114:0/3002112341 shutdown_connections 2024-04-24T09:14:21.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:21.802+0000 7fc11b276700 1 --2- 172.21.15.114:0/3002112341 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc114104700 0x7fc11410afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:21.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:21.802+0000 7fc11b276700 1 --2- 172.21.15.114:0/3002112341 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc114102b60 0x7fc114102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:21.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:21.802+0000 7fc11b276700 1 --2- 172.21.15.114:0/3002112341 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc114103d60 0x7fc1141041c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:21.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:21.802+0000 7fc11b276700 1 -- 172.21.15.114:0/3002112341 >> 172.21.15.114:0/3002112341 conn(0x7fc1140fe0c0 msgr2=0x7fc114100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:21.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:21.802+0000 7fc11b276700 1 -- 172.21.15.114:0/3002112341 shutdown_connections 2024-04-24T09:14:21.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:21.802+0000 7fc11b276700 1 -- 172.21.15.114:0/3002112341 wait complete. 2024-04-24T09:14:21.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:21.803+0000 7fc11b276700 1 Processor -- start 2024-04-24T09:14:21.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:21.803+0000 7fc11b276700 1 -- start start 2024-04-24T09:14:21.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:21.803+0000 7fc11b276700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc114102b60 0x7fc114198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:21.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:21.803+0000 7fc11b276700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc114103d60 0x7fc114199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:21.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:21.803+0000 7fc11b276700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc114104700 0x7fc1141a0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:21.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:21.803+0000 7fc11b276700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc11410e7c0 con 0x7fc114104700 2024-04-24T09:14:21.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:21.803+0000 7fc11b276700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc11410e640 con 0x7fc114102b60 2024-04-24T09:14:21.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:21.803+0000 7fc11b276700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc11410e940 con 0x7fc114103d60 2024-04-24T09:14:21.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:21.803+0000 7fc119012700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc114102b60 0x7fc114198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:21.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:21.803+0000 7fc119012700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc114102b60 0x7fc114198dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58952/0 (socket says 172.21.15.114:58952) 2024-04-24T09:14:21.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:21.803+0000 7fc119012700 1 -- 172.21.15.114:0/854168512 learned_addr learned my addr 172.21.15.114:0/854168512 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-24T09:14:21.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:21.803+0000 7fc119012700 1 -- 172.21.15.114:0/854168512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc114103d60 msgr2=0x7fc114199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:21.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:21.803+0000 7fc119012700 1 --2- 172.21.15.114:0/854168512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc114103d60 0x7fc114199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:21.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:21.803+0000 7fc119012700 1 -- 172.21.15.114:0/854168512 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc114104700 msgr2=0x7fc1141a0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:21.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:21.803+0000 7fc119012700 1 --2- 172.21.15.114:0/854168512 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc114104700 0x7fc1141a0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:21.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:21.803+0000 7fc119012700 1 -- 172.21.15.114:0/854168512 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1141a0a30 con 0x7fc114102b60 2024-04-24T09:14:21.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:21.803+0000 7fc119012700 1 --2- 172.21.15.114:0/854168512 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc114102b60 0x7fc114198dd0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc11000b9c0 tx=0x7fc11000bd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:21.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:21.803+0000 7fc10a7fc700 1 -- 172.21.15.114:0/854168512 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc110019070 con 0x7fc114102b60 2024-04-24T09:14:21.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:21.803+0000 7fc10a7fc700 1 -- 172.21.15.114:0/854168512 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc110014440 con 0x7fc114102b60 2024-04-24T09:14:21.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:21.803+0000 7fc11b276700 1 -- 172.21.15.114:0/854168512 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1141a0d20 con 0x7fc114102b60 2024-04-24T09:14:21.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:21.804+0000 7fc10a7fc700 1 -- 172.21.15.114:0/854168512 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1100125a0 con 0x7fc114102b60 2024-04-24T09:14:21.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:21.804+0000 7fc11b276700 1 -- 172.21.15.114:0/854168512 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1141a1270 con 0x7fc114102b60 2024-04-24T09:14:21.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:21.804+0000 7fc11b276700 1 -- 172.21.15.114:0/854168512 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc11402dfe0 con 0x7fc114102b60 2024-04-24T09:14:21.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:21.805+0000 7fc10a7fc700 1 -- 172.21.15.114:0/854168512 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc11001c070 con 0x7fc114102b60 2024-04-24T09:14:21.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:21.806+0000 7fc10a7fc700 1 --2- 172.21.15.114:0/854168512 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fc1000a3cc0 0x7fc1000a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:21.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:21.806+0000 7fc10a7fc700 1 -- 172.21.15.114:0/854168512 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 3669+0+0 (secure 0 0 0) 0x7fc1100ce580 con 0x7fc114102b60 2024-04-24T09:14:21.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:21.806+0000 7fc118811700 1 --2- 172.21.15.114:0/854168512 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fc1000a3cc0 0x7fc1000a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:21.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:21.807+0000 7fc118811700 1 --2- 172.21.15.114:0/854168512 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fc1000a3cc0 0x7fc1000a6180 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc104003890 tx=0x7fc104000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:21.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:21.811+0000 7fc10a7fc700 1 -- 172.21.15.114:0/854168512 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc11009c180 con 0x7fc114102b60 2024-04-24T09:14:22.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:22.018+0000 7fc11b276700 1 -- 172.21.15.114:0/854168512 --> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fc114062f30 con 0x7fc1000a3cc0 2024-04-24T09:14:22.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:22 smithi114 ceph-mon[32501]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 134 KiB/s, 0 objects/s recovering 2024-04-24T09:14:22.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:22 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:14:22.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:22 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:14:22.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:22 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:14:22.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:22 smithi090 ceph-mon[27825]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 134 KiB/s, 0 objects/s recovering 2024-04-24T09:14:22.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:22 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:14:22.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:22 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:14:22.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:22 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:14:22.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:22 smithi149 ceph-mon[32592]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 134 KiB/s, 0 objects/s recovering 2024-04-24T09:14:22.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:22 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:14:22.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:22 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:14:22.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:22 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:14:23.546 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:23 smithi090 ceph-mon[27825]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:14:23.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:23 smithi149 ceph-mon[32592]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:14:23.644 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:23 smithi114 ceph-mon[32501]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:14:24.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:24 smithi114 ceph-mon[32501]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T09:14:24.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:24 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:24.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:24 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:24.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:24 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:14:24.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:24 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:14:24.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:24 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:24.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:24 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:14:24.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:24 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:14:24.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:24 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:24.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:24 smithi090 ceph-mon[27825]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T09:14:24.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:24 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:24.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:24 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:24.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:24 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:14:24.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:24 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:14:24.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:24 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:24.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:24 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:14:24.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:24 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:14:24.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:24 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:24.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:24 smithi149 ceph-mon[32592]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T09:14:24.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:24 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:24.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:24 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:24.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:24 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:14:24.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:24 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:14:24.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:24 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:24.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:24 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:14:24.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:24 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:14:24.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:24 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:25.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:25 smithi114 ceph-mon[32501]: Detected new or changed devices on smithi090 2024-04-24T09:14:25.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:25 smithi114 ceph-mon[32501]: Adjusting osd_memory_target on smithi090 to 8544M 2024-04-24T09:14:25.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:25 smithi114 ceph-mon[32501]: from='client.? 172.21.15.114:0/3121882291' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2ddd19e-e641-4d61-b342-02b5f12b1c4b"}]: dispatch 2024-04-24T09:14:25.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:25 smithi114 ceph-mon[32501]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2ddd19e-e641-4d61-b342-02b5f12b1c4b"}]: dispatch 2024-04-24T09:14:25.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:25 smithi114 ceph-mon[32501]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e2ddd19e-e641-4d61-b342-02b5f12b1c4b"}]': finished 2024-04-24T09:14:25.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:25 smithi114 ceph-mon[32501]: osdmap e31: 5 total, 4 up, 5 in 2024-04-24T09:14:25.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:25 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T09:14:25.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:25 smithi090 ceph-mon[27825]: Detected new or changed devices on smithi090 2024-04-24T09:14:25.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:25 smithi090 ceph-mon[27825]: Adjusting osd_memory_target on smithi090 to 8544M 2024-04-24T09:14:25.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:25 smithi090 ceph-mon[27825]: from='client.? 172.21.15.114:0/3121882291' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2ddd19e-e641-4d61-b342-02b5f12b1c4b"}]: dispatch 2024-04-24T09:14:25.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:25 smithi090 ceph-mon[27825]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2ddd19e-e641-4d61-b342-02b5f12b1c4b"}]: dispatch 2024-04-24T09:14:25.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:25 smithi090 ceph-mon[27825]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e2ddd19e-e641-4d61-b342-02b5f12b1c4b"}]': finished 2024-04-24T09:14:25.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:25 smithi090 ceph-mon[27825]: osdmap e31: 5 total, 4 up, 5 in 2024-04-24T09:14:25.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:25 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T09:14:25.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:25 smithi149 ceph-mon[32592]: Detected new or changed devices on smithi090 2024-04-24T09:14:25.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:25 smithi149 ceph-mon[32592]: Adjusting osd_memory_target on smithi090 to 8544M 2024-04-24T09:14:25.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:25 smithi149 ceph-mon[32592]: from='client.? 172.21.15.114:0/3121882291' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2ddd19e-e641-4d61-b342-02b5f12b1c4b"}]: dispatch 2024-04-24T09:14:25.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:25 smithi149 ceph-mon[32592]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2ddd19e-e641-4d61-b342-02b5f12b1c4b"}]: dispatch 2024-04-24T09:14:25.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:25 smithi149 ceph-mon[32592]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e2ddd19e-e641-4d61-b342-02b5f12b1c4b"}]': finished 2024-04-24T09:14:25.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:25 smithi149 ceph-mon[32592]: osdmap e31: 5 total, 4 up, 5 in 2024-04-24T09:14:25.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:25 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T09:14:26.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:26 smithi090 ceph-mon[27825]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-04-24T09:14:26.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:26 smithi090 ceph-mon[27825]: from='client.? 172.21.15.114:0/789235244' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:14:26.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:26 smithi149 ceph-mon[32592]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-04-24T09:14:26.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:26 smithi149 ceph-mon[32592]: from='client.? 172.21.15.114:0/789235244' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:14:26.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:26 smithi114 ceph-mon[32501]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-04-24T09:14:26.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:26 smithi114 ceph-mon[32501]: from='client.? 172.21.15.114:0/789235244' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:14:28.806 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:28 smithi114 ceph-mon[32501]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T09:14:28.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:28 smithi090 ceph-mon[27825]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T09:14:28.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:28 smithi149 ceph-mon[32592]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T09:14:30.747 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:30 smithi114 ceph-mon[32501]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-04-24T09:14:30.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:30 smithi090 ceph-mon[27825]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-04-24T09:14:30.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:30 smithi149 ceph-mon[32592]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-04-24T09:14:32.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:32 smithi114 ceph-mon[32501]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-24T09:14:32.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:32 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-24T09:14:32.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:32 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:14:32.854 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:32 smithi090 ceph-mon[27825]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-24T09:14:32.855 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:32 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-24T09:14:32.855 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:32 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:14:32.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:32 smithi149 ceph-mon[32592]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-24T09:14:32.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:32 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-24T09:14:32.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:32 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:14:33.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:33 smithi114 ceph-mon[32501]: Deploying daemon osd.4 on smithi114 2024-04-24T09:14:33.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:33 smithi090 ceph-mon[27825]: Deploying daemon osd.4 on smithi114 2024-04-24T09:14:33.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:33 smithi149 ceph-mon[32592]: Deploying daemon osd.4 on smithi114 2024-04-24T09:14:34.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:34 smithi114 ceph-mon[32501]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-24T09:14:34.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:34 smithi090 ceph-mon[27825]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-24T09:14:34.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:34 smithi149 ceph-mon[32592]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-24T09:14:36.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:36 smithi114 ceph-mon[32501]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-24T09:14:36.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:36 smithi090 ceph-mon[27825]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-24T09:14:36.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:36 smithi149 ceph-mon[32592]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-24T09:14:38.884 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:38 smithi114 ceph-mon[32501]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-24T09:14:38.884 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:38 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:38.884 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:38 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:14:38.884 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:38 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:39.029 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:38 smithi090 ceph-mon[27825]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-24T09:14:39.029 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:38 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:39.029 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:38 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:14:39.029 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:38 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:39.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:38 smithi149 ceph-mon[32592]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-24T09:14:39.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:38 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:39.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:38 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:14:39.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:38 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:40.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:40 smithi114 ceph-mon[32501]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-24T09:14:40.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:40 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:40.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:40 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:40.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:40 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:14:40.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:40 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:14:40.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:40 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:41.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:40 smithi090 ceph-mon[27825]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-24T09:14:41.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:40 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:41.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:40 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:41.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:40 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:14:41.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:40 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:14:41.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:40 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:41.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:40 smithi149 ceph-mon[32592]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-24T09:14:41.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:40 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:41.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:40 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:41.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:40 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:14:41.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:40 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:14:41.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:40 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:41.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:41.548+0000 7fc10a7fc700 1 -- 172.21.15.114:0/854168512 <== mgr.14152 v2:172.21.15.90:6800/4261662956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc114062f30 con 0x7fc1000a3cc0 2024-04-24T09:14:41.549 INFO:teuthology.orchestra.run.smithi114.stdout:Created osd(s) 4 on host 'smithi114' 2024-04-24T09:14:41.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:41.550+0000 7fc11b276700 1 -- 172.21.15.114:0/854168512 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fc1000a3cc0 msgr2=0x7fc1000a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:41.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:41.550+0000 7fc11b276700 1 --2- 172.21.15.114:0/854168512 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fc1000a3cc0 0x7fc1000a6180 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc104003890 tx=0x7fc104000bc0 comp rx=0 tx=0).stop 2024-04-24T09:14:41.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:41.550+0000 7fc11b276700 1 -- 172.21.15.114:0/854168512 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc114102b60 msgr2=0x7fc114198dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:41.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:41.550+0000 7fc11b276700 1 --2- 172.21.15.114:0/854168512 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc114102b60 0x7fc114198dd0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc11000b9c0 tx=0x7fc11000bd80 comp rx=0 tx=0).stop 2024-04-24T09:14:41.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:41.550+0000 7fc11b276700 1 -- 172.21.15.114:0/854168512 shutdown_connections 2024-04-24T09:14:41.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:41.550+0000 7fc11b276700 1 --2- 172.21.15.114:0/854168512 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fc1000a3cc0 0x7fc1000a6180 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:41.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:41.550+0000 7fc11b276700 1 --2- 172.21.15.114:0/854168512 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc114104700 0x7fc1141a0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:41.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:41.550+0000 7fc11b276700 1 --2- 172.21.15.114:0/854168512 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc114102b60 0x7fc114198dd0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:41.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:41.550+0000 7fc11b276700 1 --2- 172.21.15.114:0/854168512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc114103d60 0x7fc114199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:41.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:41.550+0000 7fc11b276700 1 -- 172.21.15.114:0/854168512 >> 172.21.15.114:0/854168512 conn(0x7fc1140fe0c0 msgr2=0x7fc114107060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:41.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:41.550+0000 7fc11b276700 1 -- 172.21.15.114:0/854168512 shutdown_connections 2024-04-24T09:14:41.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:14:41.550+0000 7fc11b276700 1 -- 172.21.15.114:0/854168512 wait complete. 2024-04-24T09:14:42.244 DEBUG:teuthology.orchestra.run.smithi114:osd.4> sudo journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@osd.4.service 2024-04-24T09:14:42.247 INFO:tasks.cephadm:Deploying osd.5 on smithi149 with /dev/vg_nvme/lv_3... 2024-04-24T09:14:42.247 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-24T09:14:42.496 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T09:14:42.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:42 smithi149 ceph-mon[32592]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-24T09:14:42.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:42 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:42.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:42 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:42.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:42 smithi149 ceph-mon[32592]: from='osd.4 [v2:172.21.15.114:6808/2782015522,v1:172.21.15.114:6809/2782015522]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T09:14:42.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:42 smithi149 ceph-mon[32592]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T09:14:42.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:42 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:14:42.771 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:42 smithi114 ceph-mon[32501]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-24T09:14:42.771 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:42 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:42.771 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:42 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:42.771 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:42 smithi114 ceph-mon[32501]: from='osd.4 [v2:172.21.15.114:6808/2782015522,v1:172.21.15.114:6809/2782015522]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T09:14:42.771 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:42 smithi114 ceph-mon[32501]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T09:14:42.771 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:42 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:14:42.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:42 smithi090 ceph-mon[27825]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-24T09:14:42.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:42 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:42.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:42 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:42.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:42 smithi090 ceph-mon[27825]: from='osd.4 [v2:172.21.15.114:6808/2782015522,v1:172.21.15.114:6809/2782015522]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T09:14:42.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:42 smithi090 ceph-mon[27825]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T09:14:42.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:42 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:14:43.701 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:43 smithi114 ceph-mon[32501]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-24T09:14:43.702 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:43 smithi114 ceph-mon[32501]: from='osd.4 [v2:172.21.15.114:6808/2782015522,v1:172.21.15.114:6809/2782015522]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-04-24T09:14:43.702 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:43 smithi114 ceph-mon[32501]: osdmap e32: 5 total, 4 up, 5 in 2024-04-24T09:14:43.702 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:43 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T09:14:43.702 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:43 smithi114 ceph-mon[32501]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-04-24T09:14:43.703 INFO:journalctl@ceph.osd.4.smithi114.stdout:Apr 24 09:14:43 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-4[44018]: 2024-04-24T09:14:43.515+0000 7f7443674700 -1 osd.4 0 waiting for initial osdmap 2024-04-24T09:14:43.703 INFO:journalctl@ceph.osd.4.smithi114.stdout:Apr 24 09:14:43 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-4[44018]: 2024-04-24T09:14:43.522+0000 7f743f467700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T09:14:43.815 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:43 smithi149 ceph-mon[32592]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-24T09:14:43.815 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:43 smithi149 ceph-mon[32592]: from='osd.4 [v2:172.21.15.114:6808/2782015522,v1:172.21.15.114:6809/2782015522]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-04-24T09:14:43.815 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:43 smithi149 ceph-mon[32592]: osdmap e32: 5 total, 4 up, 5 in 2024-04-24T09:14:43.816 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:43 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T09:14:43.816 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:43 smithi149 ceph-mon[32592]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-04-24T09:14:43.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:43 smithi090 ceph-mon[27825]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-24T09:14:43.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:43 smithi090 ceph-mon[27825]: from='osd.4 [v2:172.21.15.114:6808/2782015522,v1:172.21.15.114:6809/2782015522]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-04-24T09:14:43.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:43 smithi090 ceph-mon[27825]: osdmap e32: 5 total, 4 up, 5 in 2024-04-24T09:14:43.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:43 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T09:14:43.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:43 smithi090 ceph-mon[27825]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-04-24T09:14:44.517 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-24T09:14:44.517 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-24T09:14:44.517 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2024-04-24T09:14:44.517 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2024-04-24T09:14:44.517 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0129504 s, 810 MB/s 2024-04-24T09:14:44.518 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2024-04-24T09:14:44.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:44 smithi090 ceph-mon[27825]: purged_snaps scrub starts 2024-04-24T09:14:44.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:44 smithi090 ceph-mon[27825]: purged_snaps scrub ok 2024-04-24T09:14:44.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:44 smithi090 ceph-mon[27825]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-24T09:14:44.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:44 smithi090 ceph-mon[27825]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-04-24T09:14:44.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:44 smithi090 ceph-mon[27825]: osdmap e33: 5 total, 4 up, 5 in 2024-04-24T09:14:44.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:44 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T09:14:44.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:44 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T09:14:44.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:44 smithi149 ceph-mon[32592]: purged_snaps scrub starts 2024-04-24T09:14:44.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:44 smithi149 ceph-mon[32592]: purged_snaps scrub ok 2024-04-24T09:14:44.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:44 smithi149 ceph-mon[32592]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-24T09:14:44.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:44 smithi149 ceph-mon[32592]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-04-24T09:14:44.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:44 smithi149 ceph-mon[32592]: osdmap e33: 5 total, 4 up, 5 in 2024-04-24T09:14:44.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:44 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T09:14:44.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:44 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T09:14:44.872 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:44 smithi114 ceph-mon[32501]: purged_snaps scrub starts 2024-04-24T09:14:44.872 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:44 smithi114 ceph-mon[32501]: purged_snaps scrub ok 2024-04-24T09:14:44.872 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:44 smithi114 ceph-mon[32501]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-24T09:14:44.872 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:44 smithi114 ceph-mon[32501]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-04-24T09:14:44.872 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:44 smithi114 ceph-mon[32501]: osdmap e33: 5 total, 4 up, 5 in 2024-04-24T09:14:44.872 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:44 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T09:14:44.872 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:44 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T09:14:45.221 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi149:vg_nvme/lv_3 2024-04-24T09:14:45.473 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T09:14:45.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:45 smithi090 ceph-mon[27825]: osd.4 [v2:172.21.15.114:6808/2782015522,v1:172.21.15.114:6809/2782015522] boot 2024-04-24T09:14:45.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:45 smithi090 ceph-mon[27825]: osdmap e34: 5 total, 5 up, 5 in 2024-04-24T09:14:45.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:45 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T09:14:45.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:45 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:45.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:45 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:45.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:45 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:14:45.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:45 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:14:45.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:45 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:45.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:45 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:14:45.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:45 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:14:45.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:45 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:45.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:45 smithi149 ceph-mon[32592]: osd.4 [v2:172.21.15.114:6808/2782015522,v1:172.21.15.114:6809/2782015522] boot 2024-04-24T09:14:45.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:45 smithi149 ceph-mon[32592]: osdmap e34: 5 total, 5 up, 5 in 2024-04-24T09:14:45.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:45 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T09:14:45.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:45 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:45.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:45 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:45.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:45 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:14:45.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:45 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:14:45.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:45 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:45.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:45 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:14:45.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:45 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:14:45.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:45 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:45.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:45 smithi114 ceph-mon[32501]: osd.4 [v2:172.21.15.114:6808/2782015522,v1:172.21.15.114:6809/2782015522] boot 2024-04-24T09:14:45.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:45 smithi114 ceph-mon[32501]: osdmap e34: 5 total, 5 up, 5 in 2024-04-24T09:14:45.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:45 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T09:14:45.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:45 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:45.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:45 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:45.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:45 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:14:45.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:45 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:14:45.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:45 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:45.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:45 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:14:45.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:45 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:14:45.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:45 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:14:46.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:46 smithi149 ceph-mon[32592]: Detected new or changed devices on smithi114 2024-04-24T09:14:46.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:46 smithi149 ceph-mon[32592]: Adjusting osd_memory_target on smithi114 to 8544M 2024-04-24T09:14:46.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:46 smithi149 ceph-mon[32592]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-24T09:14:46.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:46 smithi149 ceph-mon[32592]: osdmap e35: 5 total, 5 up, 5 in 2024-04-24T09:14:46.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:46 smithi090 ceph-mon[27825]: Detected new or changed devices on smithi114 2024-04-24T09:14:46.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:46 smithi090 ceph-mon[27825]: Adjusting osd_memory_target on smithi114 to 8544M 2024-04-24T09:14:46.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:46 smithi090 ceph-mon[27825]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-24T09:14:46.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:46 smithi090 ceph-mon[27825]: osdmap e35: 5 total, 5 up, 5 in 2024-04-24T09:14:46.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:46 smithi114 ceph-mon[32501]: Detected new or changed devices on smithi114 2024-04-24T09:14:46.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:46 smithi114 ceph-mon[32501]: Adjusting osd_memory_target on smithi114 to 8544M 2024-04-24T09:14:46.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:46 smithi114 ceph-mon[32501]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-24T09:14:46.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:46 smithi114 ceph-mon[32501]: osdmap e35: 5 total, 5 up, 5 in 2024-04-24T09:14:47.139 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:14:47.137+0000 7fb3fa758700 1 -- 172.21.15.149:0/1414409690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb3f4102c70 msgr2=0x7fb3f41030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:47.139 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:14:47.137+0000 7fb3fa758700 1 --2- 172.21.15.149:0/1414409690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb3f4102c70 0x7fb3f41030d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8009990 tx=0x7fb3e8009ca0 comp rx=0 tx=0).stop 2024-04-24T09:14:47.140 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:14:47.137+0000 7fb3fa758700 1 -- 172.21.15.149:0/1414409690 shutdown_connections 2024-04-24T09:14:47.140 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:14:47.137+0000 7fb3fa758700 1 --2- 172.21.15.149:0/1414409690 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb3f4103610 0x7fb3f4109ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:47.140 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:14:47.137+0000 7fb3fa758700 1 --2- 172.21.15.149:0/1414409690 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3f4101a70 0x7fb3f4101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:47.140 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:14:47.137+0000 7fb3fa758700 1 --2- 172.21.15.149:0/1414409690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb3f4102c70 0x7fb3f41030d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:47.140 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:14:47.137+0000 7fb3fa758700 1 -- 172.21.15.149:0/1414409690 >> 172.21.15.149:0/1414409690 conn(0x7fb3f40fd010 msgr2=0x7fb3f40ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:47.140 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:14:47.138+0000 7fb3fa758700 1 -- 172.21.15.149:0/1414409690 shutdown_connections 2024-04-24T09:14:47.140 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:14:47.138+0000 7fb3fa758700 1 -- 172.21.15.149:0/1414409690 wait complete. 2024-04-24T09:14:47.140 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:14:47.138+0000 7fb3fa758700 1 Processor -- start 2024-04-24T09:14:47.140 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:14:47.138+0000 7fb3fa758700 1 -- start start 2024-04-24T09:14:47.140 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:14:47.138+0000 7fb3fa758700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3f4101a70 0x7fb3f406bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:47.140 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:14:47.138+0000 7fb3fa758700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb3f4102c70 0x7fb3f419dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:47.140 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:14:47.138+0000 7fb3fa758700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb3f4103610 0x7fb3f41a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:47.140 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:14:47.138+0000 7fb3fa758700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f410d750 con 0x7fb3f4102c70 2024-04-24T09:14:47.141 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:14:47.138+0000 7fb3fa758700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f410d5d0 con 0x7fb3f4101a70 2024-04-24T09:14:47.141 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:14:47.138+0000 7fb3fa758700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f410d8d0 con 0x7fb3f4103610 2024-04-24T09:14:47.141 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:14:47.139+0000 7fb3f8cf5700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb3f4103610 0x7fb3f41a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:47.141 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:14:47.139+0000 7fb3f8cf5700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb3f4103610 0x7fb3f41a0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:40290/0 (socket says 172.21.15.149:40290) 2024-04-24T09:14:47.141 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:14:47.139+0000 7fb3f8cf5700 1 -- 172.21.15.149:0/818045267 learned_addr learned my addr 172.21.15.149:0/818045267 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-04-24T09:14:47.141 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:14:47.139+0000 7fb3f3fff700 1 --2- 172.21.15.149:0/818045267 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3f4101a70 0x7fb3f406bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:47.141 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:14:47.139+0000 7fb3f8cf5700 1 -- 172.21.15.149:0/818045267 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3f4101a70 msgr2=0x7fb3f406bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:47.141 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:14:47.139+0000 7fb3f8cf5700 1 --2- 172.21.15.149:0/818045267 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3f4101a70 0x7fb3f406bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:47.141 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:14:47.139+0000 7fb3f8cf5700 1 -- 172.21.15.149:0/818045267 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb3f4102c70 msgr2=0x7fb3f419dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:47.141 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:14:47.139+0000 7fb3f8cf5700 1 --2- 172.21.15.149:0/818045267 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb3f4102c70 0x7fb3f419dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:47.141 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:14:47.139+0000 7fb3f8cf5700 1 -- 172.21.15.149:0/818045267 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3f41a0b30 con 0x7fb3f4103610 2024-04-24T09:14:47.141 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:14:47.139+0000 7fb3f8cf5700 1 --2- 172.21.15.149:0/818045267 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb3f4103610 0x7fb3f41a0260 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec00b970 tx=0x7fb3ec00bd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:47.142 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:14:47.139+0000 7fb3f17fa700 1 -- 172.21.15.149:0/818045267 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec019070 con 0x7fb3f4103610 2024-04-24T09:14:47.142 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:14:47.139+0000 7fb3f17fa700 1 -- 172.21.15.149:0/818045267 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3ec014440 con 0x7fb3f4103610 2024-04-24T09:14:47.142 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:14:47.139+0000 7fb3fa758700 1 -- 172.21.15.149:0/818045267 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3f41a0e20 con 0x7fb3f4103610 2024-04-24T09:14:47.142 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:14:47.139+0000 7fb3f17fa700 1 -- 172.21.15.149:0/818045267 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec0075d0 con 0x7fb3f4103610 2024-04-24T09:14:47.142 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:14:47.139+0000 7fb3fa758700 1 -- 172.21.15.149:0/818045267 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3f41a1360 con 0x7fb3f4103610 2024-04-24T09:14:47.143 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:14:47.140+0000 7fb3fa758700 1 -- 172.21.15.149:0/818045267 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3f40697e0 con 0x7fb3f4103610 2024-04-24T09:14:47.143 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:14:47.141+0000 7fb3f17fa700 1 -- 172.21.15.149:0/818045267 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb3ec01c070 con 0x7fb3f4103610 2024-04-24T09:14:47.143 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:14:47.142+0000 7fb3f17fa700 1 --2- 172.21.15.149:0/818045267 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fb3dc0a3c40 0x7fb3dc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:47.143 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:14:47.142+0000 7fb3f17fa700 1 -- 172.21.15.149:0/818045267 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4101+0+0 (secure 0 0 0) 0x7fb3ec0cec80 con 0x7fb3f4103610 2024-04-24T09:14:47.143 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:14:47.142+0000 7fb3f3fff700 1 --2- 172.21.15.149:0/818045267 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fb3dc0a3c40 0x7fb3dc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:47.143 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:14:47.143+0000 7fb3f3fff700 1 --2- 172.21.15.149:0/818045267 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fb3dc0a3c40 0x7fb3dc0a6100 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb3e4006fd0 tx=0x7fb3e4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:47.145 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:14:47.146+0000 7fb3f17fa700 1 -- 172.21.15.149:0/818045267 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb3ec09c6d0 con 0x7fb3f4103610 2024-04-24T09:14:47.350 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:14:47.349+0000 7fb3fa758700 1 -- 172.21.15.149:0/818045267 --> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fb3f4062f30 con 0x7fb3dc0a3c40 2024-04-24T09:14:47.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:47 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:14:47.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:47 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:14:47.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:47 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:14:47.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:47 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:14:47.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:47 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:14:47.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:47 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:14:47.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:47 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:14:47.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:47 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:14:47.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:47 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:14:48.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:48 smithi090 ceph-mon[27825]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:14:48.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:48 smithi090 ceph-mon[27825]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-24T09:14:48.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:48 smithi149 ceph-mon[32592]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:14:48.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:48 smithi149 ceph-mon[32592]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-24T09:14:48.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:48 smithi114 ceph-mon[32501]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:14:48.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:48 smithi114 ceph-mon[32501]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-24T09:14:50.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:50 smithi090 ceph-mon[27825]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-24T09:14:50.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:50 smithi090 ceph-mon[27825]: from='client.? 172.21.15.149:0/3851629157' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6ff9d10-5210-4b46-a192-6dcbf9d1f156"}]: dispatch 2024-04-24T09:14:50.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:50 smithi090 ceph-mon[27825]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6ff9d10-5210-4b46-a192-6dcbf9d1f156"}]: dispatch 2024-04-24T09:14:50.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:50 smithi090 ceph-mon[27825]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b6ff9d10-5210-4b46-a192-6dcbf9d1f156"}]': finished 2024-04-24T09:14:50.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:50 smithi090 ceph-mon[27825]: osdmap e36: 6 total, 5 up, 6 in 2024-04-24T09:14:50.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:50 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T09:14:50.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:50 smithi149 ceph-mon[32592]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-24T09:14:50.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:50 smithi149 ceph-mon[32592]: from='client.? 172.21.15.149:0/3851629157' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6ff9d10-5210-4b46-a192-6dcbf9d1f156"}]: dispatch 2024-04-24T09:14:50.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:50 smithi149 ceph-mon[32592]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6ff9d10-5210-4b46-a192-6dcbf9d1f156"}]: dispatch 2024-04-24T09:14:50.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:50 smithi149 ceph-mon[32592]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b6ff9d10-5210-4b46-a192-6dcbf9d1f156"}]': finished 2024-04-24T09:14:50.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:50 smithi149 ceph-mon[32592]: osdmap e36: 6 total, 5 up, 6 in 2024-04-24T09:14:50.863 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:50 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T09:14:50.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:50 smithi114 ceph-mon[32501]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-24T09:14:50.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:50 smithi114 ceph-mon[32501]: from='client.? 172.21.15.149:0/3851629157' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6ff9d10-5210-4b46-a192-6dcbf9d1f156"}]: dispatch 2024-04-24T09:14:50.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:50 smithi114 ceph-mon[32501]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6ff9d10-5210-4b46-a192-6dcbf9d1f156"}]: dispatch 2024-04-24T09:14:50.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:50 smithi114 ceph-mon[32501]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b6ff9d10-5210-4b46-a192-6dcbf9d1f156"}]': finished 2024-04-24T09:14:50.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:50 smithi114 ceph-mon[32501]: osdmap e36: 6 total, 5 up, 6 in 2024-04-24T09:14:50.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:50 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T09:14:51.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:51 smithi090 ceph-mon[27825]: from='client.? 172.21.15.149:0/318359692' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:14:51.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:51 smithi149 ceph-mon[32592]: from='client.? 172.21.15.149:0/318359692' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:14:51.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:51 smithi114 ceph-mon[32501]: from='client.? 172.21.15.149:0/318359692' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:14:52.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:52 smithi090 ceph-mon[27825]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-24T09:14:52.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:52 smithi149 ceph-mon[32592]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-24T09:14:52.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:52 smithi114 ceph-mon[32501]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-24T09:14:54.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:54 smithi090 ceph-mon[27825]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-24T09:14:54.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:54 smithi149 ceph-mon[32592]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-24T09:14:54.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:54 smithi114 ceph-mon[32501]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-24T09:14:56.851 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:56 smithi149 ceph-mon[32592]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-24T09:14:56.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:56 smithi090 ceph-mon[27825]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-24T09:14:56.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:56 smithi114 ceph-mon[32501]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-24T09:14:58.741 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:58 smithi149 ceph-mon[32592]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-24T09:14:58.741 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:58 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-24T09:14:58.741 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:58 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:14:58.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:58 smithi090 ceph-mon[27825]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-24T09:14:58.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:58 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-24T09:14:58.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:58 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:14:58.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:58 smithi114 ceph-mon[32501]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-24T09:14:58.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:58 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-24T09:14:58.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:58 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:14:59.727 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:14:59 smithi149 ceph-mon[32592]: Deploying daemon osd.5 on smithi149 2024-04-24T09:14:59.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:14:59 smithi114 ceph-mon[32501]: Deploying daemon osd.5 on smithi149 2024-04-24T09:15:00.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:14:59 smithi090 ceph-mon[27825]: Deploying daemon osd.5 on smithi149 2024-04-24T09:15:00.726 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:00 smithi149 ceph-mon[32592]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-24T09:15:00.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:00 smithi114 ceph-mon[32501]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-24T09:15:01.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:00 smithi090 ceph-mon[27825]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-24T09:15:02.718 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:02 smithi149 ceph-mon[32592]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-24T09:15:02.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:02 smithi114 ceph-mon[32501]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-24T09:15:03.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:02 smithi090 ceph-mon[27825]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-24T09:15:04.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:04 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:04.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:04 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:15:04.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:04 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:04.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:04 smithi090 ceph-mon[27825]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-24T09:15:04.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:04 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:04.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:04 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:15:04.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:04 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:04.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:04 smithi149 ceph-mon[32592]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-24T09:15:04.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:04 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:04.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:04 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:15:04.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:04 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:04.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:04 smithi114 ceph-mon[32501]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-24T09:15:06.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:06 smithi149 ceph-mon[32592]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-24T09:15:06.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:06 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:06.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:06 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:06.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:06 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:15:06.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:06 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:15:06.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:06 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:06.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:06 smithi090 ceph-mon[27825]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-24T09:15:06.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:06 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:06.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:06 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:06.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:06 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:15:06.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:06 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:15:06.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:06 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:06.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:06 smithi114 ceph-mon[32501]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-24T09:15:06.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:06 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:06.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:06 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:06.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:06 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:15:06.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:06 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:15:06.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:06 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:07.087 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:15:07.087+0000 7fb3f17fa700 1 -- 172.21.15.149:0/818045267 <== mgr.14152 v2:172.21.15.90:6800/4261662956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb3f4062f30 con 0x7fb3dc0a3c40 2024-04-24T09:15:07.087 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 5 on host 'smithi149' 2024-04-24T09:15:07.090 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:15:07.089+0000 7fb3fa758700 1 -- 172.21.15.149:0/818045267 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fb3dc0a3c40 msgr2=0x7fb3dc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:07.090 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:15:07.089+0000 7fb3fa758700 1 --2- 172.21.15.149:0/818045267 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fb3dc0a3c40 0x7fb3dc0a6100 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb3e4006fd0 tx=0x7fb3e4008040 comp rx=0 tx=0).stop 2024-04-24T09:15:07.090 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:15:07.089+0000 7fb3fa758700 1 -- 172.21.15.149:0/818045267 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb3f4103610 msgr2=0x7fb3f41a0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:07.090 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:15:07.089+0000 7fb3fa758700 1 --2- 172.21.15.149:0/818045267 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb3f4103610 0x7fb3f41a0260 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec00b970 tx=0x7fb3ec00bd30 comp rx=0 tx=0).stop 2024-04-24T09:15:07.090 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:15:07.089+0000 7fb3fa758700 1 -- 172.21.15.149:0/818045267 shutdown_connections 2024-04-24T09:15:07.090 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:15:07.089+0000 7fb3fa758700 1 --2- 172.21.15.149:0/818045267 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb3f4103610 0x7fb3f41a0260 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.090 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:15:07.089+0000 7fb3fa758700 1 --2- 172.21.15.149:0/818045267 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb3f4101a70 0x7fb3f406bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.090 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:15:07.089+0000 7fb3fa758700 1 --2- 172.21.15.149:0/818045267 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fb3dc0a3c40 0x7fb3dc0a6100 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.090 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:15:07.089+0000 7fb3fa758700 1 --2- 172.21.15.149:0/818045267 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb3f4102c70 0x7fb3f419dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.090 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:15:07.089+0000 7fb3fa758700 1 -- 172.21.15.149:0/818045267 >> 172.21.15.149:0/818045267 conn(0x7fb3f40fd010 msgr2=0x7fb3f4105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:07.091 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:15:07.089+0000 7fb3fa758700 1 -- 172.21.15.149:0/818045267 shutdown_connections 2024-04-24T09:15:07.091 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:15:07.089+0000 7fb3fa758700 1 -- 172.21.15.149:0/818045267 wait complete. 2024-04-24T09:15:07.734 DEBUG:teuthology.orchestra.run.smithi149:osd.5> sudo journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@osd.5.service 2024-04-24T09:15:07.736 INFO:tasks.cephadm:Deploying osd.6 on smithi090 with /dev/vg_nvme/lv_2... 2024-04-24T09:15:07.737 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-24T09:15:08.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:08 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:08.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:08 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:08.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:08 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:15:08.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:08 smithi090 ceph-mon[27825]: from='osd.5 [v2:172.21.15.149:6808/4266460479,v1:172.21.15.149:6809/4266460479]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T09:15:08.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:08 smithi090 ceph-mon[27825]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T09:15:08.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:08 smithi090 ceph-mon[27825]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-24T09:15:08.179 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:15:08.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:08 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:08.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:08 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:08.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:08 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:15:08.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:08 smithi149 ceph-mon[32592]: from='osd.5 [v2:172.21.15.149:6808/4266460479,v1:172.21.15.149:6809/4266460479]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T09:15:08.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:08 smithi149 ceph-mon[32592]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T09:15:08.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:08 smithi149 ceph-mon[32592]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-24T09:15:08.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:08 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:08.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:08 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:08.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:08 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:15:08.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:08 smithi114 ceph-mon[32501]: from='osd.5 [v2:172.21.15.149:6808/4266460479,v1:172.21.15.149:6809/4266460479]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T09:15:08.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:08 smithi114 ceph-mon[32501]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T09:15:08.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:08 smithi114 ceph-mon[32501]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-24T09:15:09.263 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:09 smithi149 ceph-mon[32592]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-24T09:15:09.263 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:09 smithi149 ceph-mon[32592]: osdmap e37: 6 total, 5 up, 6 in 2024-04-24T09:15:09.263 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:09 smithi149 ceph-mon[32592]: from='osd.5 [v2:172.21.15.149:6808/4266460479,v1:172.21.15.149:6809/4266460479]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-04-24T09:15:09.263 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:09 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T09:15:09.263 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:09 smithi149 ceph-mon[32592]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-04-24T09:15:09.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:09 smithi090 ceph-mon[27825]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-24T09:15:09.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:09 smithi090 ceph-mon[27825]: osdmap e37: 6 total, 5 up, 6 in 2024-04-24T09:15:09.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:09 smithi090 ceph-mon[27825]: from='osd.5 [v2:172.21.15.149:6808/4266460479,v1:172.21.15.149:6809/4266460479]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-04-24T09:15:09.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:09 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T09:15:09.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:09 smithi090 ceph-mon[27825]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-04-24T09:15:09.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:09 smithi114 ceph-mon[32501]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-24T09:15:09.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:09 smithi114 ceph-mon[32501]: osdmap e37: 6 total, 5 up, 6 in 2024-04-24T09:15:09.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:09 smithi114 ceph-mon[32501]: from='osd.5 [v2:172.21.15.149:6808/4266460479,v1:172.21.15.149:6809/4266460479]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-04-24T09:15:09.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:09 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T09:15:09.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:09 smithi114 ceph-mon[32501]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-04-24T09:15:09.612 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 09:15:09 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-5[44708]: 2024-04-24T09:15:09.500+0000 7f2d799b4700 -1 osd.5 0 waiting for initial osdmap 2024-04-24T09:15:09.612 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 09:15:09 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-5[44708]: 2024-04-24T09:15:09.507+0000 7f2d7179f700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T09:15:10.330 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:10 smithi090 ceph-mon[27825]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-04-24T09:15:10.330 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:10 smithi090 ceph-mon[27825]: osdmap e38: 6 total, 5 up, 6 in 2024-04-24T09:15:10.330 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:10 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T09:15:10.330 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:10 smithi090 ceph-mon[27825]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-24T09:15:10.330 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:10 smithi090 ceph-mon[27825]: from='osd.5 ' entity='osd.5' 2024-04-24T09:15:10.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:10 smithi149 ceph-mon[32592]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-04-24T09:15:10.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:10 smithi149 ceph-mon[32592]: osdmap e38: 6 total, 5 up, 6 in 2024-04-24T09:15:10.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:10 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T09:15:10.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:10 smithi149 ceph-mon[32592]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-24T09:15:10.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:10 smithi149 ceph-mon[32592]: from='osd.5 ' entity='osd.5' 2024-04-24T09:15:10.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:10 smithi114 ceph-mon[32501]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-04-24T09:15:10.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:10 smithi114 ceph-mon[32501]: osdmap e38: 6 total, 5 up, 6 in 2024-04-24T09:15:10.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:10 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T09:15:10.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:10 smithi114 ceph-mon[32501]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-24T09:15:10.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:10 smithi114 ceph-mon[32501]: from='osd.5 ' entity='osd.5' 2024-04-24T09:15:11.223 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:11 smithi149 ceph-mon[32592]: purged_snaps scrub starts 2024-04-24T09:15:11.224 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:11 smithi149 ceph-mon[32592]: purged_snaps scrub ok 2024-04-24T09:15:11.224 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:11 smithi149 ceph-mon[32592]: osd.5 [v2:172.21.15.149:6808/4266460479,v1:172.21.15.149:6809/4266460479] boot 2024-04-24T09:15:11.224 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:11 smithi149 ceph-mon[32592]: osdmap e39: 6 total, 6 up, 6 in 2024-04-24T09:15:11.224 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:11 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T09:15:11.224 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:11 smithi149 ceph-mon[32592]: osdmap e40: 6 total, 6 up, 6 in 2024-04-24T09:15:11.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:11 smithi090 ceph-mon[27825]: purged_snaps scrub starts 2024-04-24T09:15:11.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:11 smithi090 ceph-mon[27825]: purged_snaps scrub ok 2024-04-24T09:15:11.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:11 smithi090 ceph-mon[27825]: osd.5 [v2:172.21.15.149:6808/4266460479,v1:172.21.15.149:6809/4266460479] boot 2024-04-24T09:15:11.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:11 smithi090 ceph-mon[27825]: osdmap e39: 6 total, 6 up, 6 in 2024-04-24T09:15:11.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:11 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T09:15:11.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:11 smithi090 ceph-mon[27825]: osdmap e40: 6 total, 6 up, 6 in 2024-04-24T09:15:11.411 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-24T09:15:11.411 INFO:teuthology.orchestra.run.smithi090.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-24T09:15:11.411 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10+0 records in 2024-04-24T09:15:11.411 INFO:teuthology.orchestra.run.smithi090.stderr:10+0 records out 2024-04-24T09:15:11.412 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.058657 s, 179 MB/s 2024-04-24T09:15:11.412 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping successful for: 2024-04-24T09:15:11.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:11 smithi114 ceph-mon[32501]: purged_snaps scrub starts 2024-04-24T09:15:11.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:11 smithi114 ceph-mon[32501]: purged_snaps scrub ok 2024-04-24T09:15:11.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:11 smithi114 ceph-mon[32501]: osd.5 [v2:172.21.15.149:6808/4266460479,v1:172.21.15.149:6809/4266460479] boot 2024-04-24T09:15:11.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:11 smithi114 ceph-mon[32501]: osdmap e39: 6 total, 6 up, 6 in 2024-04-24T09:15:11.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:11 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T09:15:11.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:11 smithi114 ceph-mon[32501]: osdmap e40: 6 total, 6 up, 6 in 2024-04-24T09:15:12.420 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi090:vg_nvme/lv_2 2024-04-24T09:15:12.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:12 smithi090 ceph-mon[27825]: Detected new or changed devices on smithi149 2024-04-24T09:15:12.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:12 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:12.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:12 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:12.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:12 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:15:12.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:12 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:15:12.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:12 smithi090 ceph-mon[27825]: Adjusting osd_memory_target on smithi149 to 2908M 2024-04-24T09:15:12.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:12 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:12.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:12 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:15:12.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:12 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:15:12.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:12 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:12.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:12 smithi090 ceph-mon[27825]: pgmap v157: 1 pgs: 1 remapped+peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-04-24T09:15:12.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:12 smithi090 ceph-mon[27825]: osdmap e41: 6 total, 6 up, 6 in 2024-04-24T09:15:12.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:12 smithi149 ceph-mon[32592]: Detected new or changed devices on smithi149 2024-04-24T09:15:12.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:12 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:12.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:12 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:12.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:12 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:15:12.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:12 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:15:12.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:12 smithi149 ceph-mon[32592]: Adjusting osd_memory_target on smithi149 to 2908M 2024-04-24T09:15:12.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:12 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:12.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:12 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:15:12.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:12 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:15:12.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:12 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:12.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:12 smithi149 ceph-mon[32592]: pgmap v157: 1 pgs: 1 remapped+peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-04-24T09:15:12.613 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:12 smithi149 ceph-mon[32592]: osdmap e41: 6 total, 6 up, 6 in 2024-04-24T09:15:12.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:12 smithi114 ceph-mon[32501]: Detected new or changed devices on smithi149 2024-04-24T09:15:12.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:12 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:12.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:12 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:12.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:12 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:15:12.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:12 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:15:12.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:12 smithi114 ceph-mon[32501]: Adjusting osd_memory_target on smithi149 to 2908M 2024-04-24T09:15:12.677 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:12 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:12.677 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:12 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:15:12.677 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:12 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:15:12.677 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:12 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:12.677 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:12 smithi114 ceph-mon[32501]: pgmap v157: 1 pgs: 1 remapped+peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-04-24T09:15:12.677 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:12 smithi114 ceph-mon[32501]: osdmap e41: 6 total, 6 up, 6 in 2024-04-24T09:15:12.855 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:15:14.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:14 smithi090 ceph-mon[27825]: osdmap e42: 6 total, 6 up, 6 in 2024-04-24T09:15:14.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:14 smithi090 ceph-mon[27825]: pgmap v160: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-24T09:15:14.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:14 smithi149 ceph-mon[32592]: osdmap e42: 6 total, 6 up, 6 in 2024-04-24T09:15:14.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:14 smithi149 ceph-mon[32592]: pgmap v160: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-24T09:15:14.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:14 smithi114 ceph-mon[32501]: osdmap e42: 6 total, 6 up, 6 in 2024-04-24T09:15:14.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:14 smithi114 ceph-mon[32501]: pgmap v160: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-24T09:15:15.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:15.258+0000 7f517d600700 1 -- 172.21.15.90:0/3241718901 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5178101a50 msgr2=0x7f5178101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:15.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:15.258+0000 7f517d600700 1 --2- 172.21.15.90:0/3241718901 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5178101a50 0x7f5178101e70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f51680099c0 tx=0x7f5168009cd0 comp rx=0 tx=0).stop 2024-04-24T09:15:15.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:15.258+0000 7f517d600700 1 -- 172.21.15.90:0/3241718901 shutdown_connections 2024-04-24T09:15:15.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:15.258+0000 7f517d600700 1 --2- 172.21.15.90:0/3241718901 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51781035f0 0x7f5178109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:15.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:15.258+0000 7f517d600700 1 --2- 172.21.15.90:0/3241718901 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5178101a50 0x7f5178101e70 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:15.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:15.258+0000 7f517d600700 1 --2- 172.21.15.90:0/3241718901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5178102c50 0x7f51781030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:15.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:15.258+0000 7f517d600700 1 -- 172.21.15.90:0/3241718901 >> 172.21.15.90:0/3241718901 conn(0x7f51780fcff0 msgr2=0x7f51780ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:15.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:15.258+0000 7f517d600700 1 -- 172.21.15.90:0/3241718901 shutdown_connections 2024-04-24T09:15:15.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:15.258+0000 7f517d600700 1 -- 172.21.15.90:0/3241718901 wait complete. 2024-04-24T09:15:15.260 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:15 smithi090 ceph-mon[27825]: osdmap e43: 6 total, 6 up, 6 in 2024-04-24T09:15:15.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:15.260+0000 7f517d600700 1 Processor -- start 2024-04-24T09:15:15.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:15.260+0000 7f517d600700 1 -- start start 2024-04-24T09:15:15.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:15.260+0000 7f517d600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5178101a50 0x7f5178198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:15.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:15.260+0000 7f517d600700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5178102c50 0x7f5178199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:15.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:15.260+0000 7f517d600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51781035f0 0x7f51781a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:15.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:15.260+0000 7f517d600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f517810d610 con 0x7f5178101a50 2024-04-24T09:15:15.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:15.260+0000 7f517d600700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f517810d490 con 0x7f5178102c50 2024-04-24T09:15:15.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:15.260+0000 7f517d600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f517810d790 con 0x7f51781035f0 2024-04-24T09:15:15.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:15.260+0000 7f5176a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5178101a50 0x7f5178198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:15.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:15.261+0000 7f5176a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5178101a50 0x7f5178198de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51936/0 (socket says 172.21.15.90:51936) 2024-04-24T09:15:15.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:15.261+0000 7f5177400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51781035f0 0x7f51781a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:15.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:15.261+0000 7f5176a00700 1 -- 172.21.15.90:0/739111210 learned_addr learned my addr 172.21.15.90:0/739111210 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:15:15.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:15.261+0000 7f516e000700 1 --2- 172.21.15.90:0/739111210 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5178102c50 0x7f5178199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:15.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:15.261+0000 7f5176a00700 1 -- 172.21.15.90:0/739111210 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51781035f0 msgr2=0x7f51781a03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:15.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:15.261+0000 7f5176a00700 1 --2- 172.21.15.90:0/739111210 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51781035f0 0x7f51781a03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:15.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:15.261+0000 7f5176a00700 1 -- 172.21.15.90:0/739111210 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5178102c50 msgr2=0x7f5178199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:15.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:15.261+0000 7f5176a00700 1 --2- 172.21.15.90:0/739111210 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5178102c50 0x7f5178199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:15.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:15.261+0000 7f5176a00700 1 -- 172.21.15.90:0/739111210 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51781a0a40 con 0x7f5178101a50 2024-04-24T09:15:15.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:15.261+0000 7f5176a00700 1 --2- 172.21.15.90:0/739111210 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5178101a50 0x7f5178198de0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f516800b580 tx=0x7f516800bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:15.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:15.261+0000 7f516fe00700 1 -- 172.21.15.90:0/739111210 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5168009e60 con 0x7f5178101a50 2024-04-24T09:15:15.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:15.262+0000 7f516fe00700 1 -- 172.21.15.90:0/739111210 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5168003e50 con 0x7f5178101a50 2024-04-24T09:15:15.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:15.262+0000 7f517d600700 1 -- 172.21.15.90:0/739111210 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51781a0cd0 con 0x7f5178101a50 2024-04-24T09:15:15.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:15.262+0000 7f516fe00700 1 -- 172.21.15.90:0/739111210 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51680173f0 con 0x7f5178101a50 2024-04-24T09:15:15.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:15.262+0000 7f517d600700 1 -- 172.21.15.90:0/739111210 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51781a1120 con 0x7f5178101a50 2024-04-24T09:15:15.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:15.263+0000 7f517d600700 1 -- 172.21.15.90:0/739111210 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5178069810 con 0x7f5178101a50 2024-04-24T09:15:15.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:15.265+0000 7f516fe00700 1 -- 172.21.15.90:0/739111210 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f516800f5f0 con 0x7f5178101a50 2024-04-24T09:15:15.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:15.266+0000 7f516fe00700 1 --2- 172.21.15.90:0/739111210 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f515c0a3c40 0x7f515c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:15.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:15.266+0000 7f516fe00700 1 -- 172.21.15.90:0/739111210 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4533+0+0 (secure 0 0 0) 0x7f51680d02b0 con 0x7f5178101a50 2024-04-24T09:15:15.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:15.266+0000 7f516e000700 1 --2- 172.21.15.90:0/739111210 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f515c0a3c40 0x7f515c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:15.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:15.267+0000 7f516e000700 1 --2- 172.21.15.90:0/739111210 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f515c0a3c40 0x7f515c0a6100 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5160009680 tx=0x7f5160009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:15.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:15.272+0000 7f516fe00700 1 -- 172.21.15.90:0/739111210 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5178069810 con 0x7f5178101a50 2024-04-24T09:15:15.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:15 smithi149 ceph-mon[32592]: osdmap e43: 6 total, 6 up, 6 in 2024-04-24T09:15:15.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:15 smithi114 ceph-mon[32501]: osdmap e43: 6 total, 6 up, 6 in 2024-04-24T09:15:15.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:15.509+0000 7f517d600700 1 -- 172.21.15.90:0/739111210 --> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f51780ff230 con 0x7f515c0a3c40 2024-04-24T09:15:16.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:16 smithi090 ceph-mon[27825]: pgmap v162: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-24T09:15:16.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:16 smithi090 ceph-mon[27825]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:15:16.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:16 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:15:16.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:16 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:15:16.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:16 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:15:16.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:16 smithi149 ceph-mon[32592]: pgmap v162: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-24T09:15:16.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:16 smithi149 ceph-mon[32592]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:15:16.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:16 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:15:16.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:16 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:15:16.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:16 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:15:16.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:16 smithi114 ceph-mon[32501]: pgmap v162: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-24T09:15:16.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:16 smithi114 ceph-mon[32501]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:15:16.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:16 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:15:16.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:16 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:15:16.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:16 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:15:18.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:18 smithi090 ceph-mon[27825]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T09:15:18.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:18 smithi149 ceph-mon[32592]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T09:15:18.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:18 smithi114 ceph-mon[32501]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T09:15:20.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:20 smithi090 ceph-mon[27825]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-24T09:15:20.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:20 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/3377205627' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4287b5a1-f953-421c-bf62-5e390bee834d"}]: dispatch 2024-04-24T09:15:20.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:20 smithi090 ceph-mon[27825]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4287b5a1-f953-421c-bf62-5e390bee834d"}]: dispatch 2024-04-24T09:15:20.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:20 smithi090 ceph-mon[27825]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4287b5a1-f953-421c-bf62-5e390bee834d"}]': finished 2024-04-24T09:15:20.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:20 smithi090 ceph-mon[27825]: osdmap e44: 7 total, 6 up, 7 in 2024-04-24T09:15:20.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:20 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T09:15:20.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:20 smithi149 ceph-mon[32592]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-24T09:15:20.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:20 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/3377205627' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4287b5a1-f953-421c-bf62-5e390bee834d"}]: dispatch 2024-04-24T09:15:20.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:20 smithi149 ceph-mon[32592]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4287b5a1-f953-421c-bf62-5e390bee834d"}]: dispatch 2024-04-24T09:15:20.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:20 smithi149 ceph-mon[32592]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4287b5a1-f953-421c-bf62-5e390bee834d"}]': finished 2024-04-24T09:15:20.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:20 smithi149 ceph-mon[32592]: osdmap e44: 7 total, 6 up, 7 in 2024-04-24T09:15:20.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:20 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T09:15:20.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:20 smithi114 ceph-mon[32501]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-24T09:15:20.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:20 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/3377205627' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4287b5a1-f953-421c-bf62-5e390bee834d"}]: dispatch 2024-04-24T09:15:20.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:20 smithi114 ceph-mon[32501]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4287b5a1-f953-421c-bf62-5e390bee834d"}]: dispatch 2024-04-24T09:15:20.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:20 smithi114 ceph-mon[32501]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4287b5a1-f953-421c-bf62-5e390bee834d"}]': finished 2024-04-24T09:15:20.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:20 smithi114 ceph-mon[32501]: osdmap e44: 7 total, 6 up, 7 in 2024-04-24T09:15:20.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:20 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T09:15:21.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:21 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/3672950225' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:15:21.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:21 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/3672950225' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:15:21.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:21 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/3672950225' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:15:22.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:22 smithi090 ceph-mon[27825]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T09:15:22.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:22 smithi149 ceph-mon[32592]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T09:15:22.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:22 smithi114 ceph-mon[32501]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T09:15:24.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:24 smithi090 ceph-mon[27825]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-04-24T09:15:24.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:24 smithi149 ceph-mon[32592]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-04-24T09:15:24.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:24 smithi114 ceph-mon[32501]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-04-24T09:15:26.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:26 smithi090 ceph-mon[27825]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T09:15:26.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:26 smithi149 ceph-mon[32592]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T09:15:26.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:26 smithi114 ceph-mon[32501]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T09:15:28.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:28 smithi090 ceph-mon[27825]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-24T09:15:28.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:28 smithi149 ceph-mon[32592]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-24T09:15:28.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:28 smithi114 ceph-mon[32501]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-24T09:15:29.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:29 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-24T09:15:29.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:29 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:15:29.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:29 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-24T09:15:29.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:29 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:15:29.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:29 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-24T09:15:29.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:29 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:15:30.660 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:30 smithi090 ceph-mon[27825]: Deploying daemon osd.6 on smithi090 2024-04-24T09:15:30.660 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:30 smithi090 ceph-mon[27825]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-24T09:15:30.832 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:30 smithi114 ceph-mon[32501]: Deploying daemon osd.6 on smithi090 2024-04-24T09:15:30.832 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:30 smithi114 ceph-mon[32501]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-24T09:15:30.863 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:30 smithi149 ceph-mon[32592]: Deploying daemon osd.6 on smithi090 2024-04-24T09:15:30.864 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:30 smithi149 ceph-mon[32592]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-24T09:15:32.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:32 smithi090 ceph-mon[27825]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-24T09:15:32.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:32 smithi149 ceph-mon[32592]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-24T09:15:32.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:32 smithi114 ceph-mon[32501]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-24T09:15:34.800 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:34 smithi090 ceph-mon[27825]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-24T09:15:34.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:34 smithi149 ceph-mon[32592]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-24T09:15:34.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:34 smithi114 ceph-mon[32501]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-24T09:15:36.845 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:36 smithi090 ceph-mon[27825]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-24T09:15:36.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:36 smithi149 ceph-mon[32592]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-24T09:15:36.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:36 smithi114 ceph-mon[32501]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-24T09:15:38.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:38 smithi090 ceph-mon[27825]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-24T09:15:38.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:38 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:38.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:38 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:15:38.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:38 smithi149 ceph-mon[32592]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-24T09:15:38.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:38 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:38.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:38 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:15:38.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:38 smithi114 ceph-mon[32501]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-24T09:15:38.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:38 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:38.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:38 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:15:39.797 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:39 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:39.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:39 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:39.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:39 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:40.851 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:40 smithi090 ceph-mon[27825]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-24T09:15:40.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:40 smithi149 ceph-mon[32592]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-24T09:15:40.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:40 smithi114 ceph-mon[32501]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-24T09:15:42.801 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:42 smithi090 ceph-mon[27825]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-24T09:15:42.801 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:42 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:42.802 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:42 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:42.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:42 smithi149 ceph-mon[32592]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-24T09:15:42.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:42 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:42.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:42 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:42.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:42 smithi114 ceph-mon[32501]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-24T09:15:42.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:42 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:42.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:42 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:43.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:43 smithi090 ceph-mon[27825]: from='osd.6 [v2:172.21.15.90:6818/1121461276,v1:172.21.15.90:6819/1121461276]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-24T09:15:43.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:43 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:15:43.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:43 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:15:43.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:43 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:43.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:43 smithi149 ceph-mon[32592]: from='osd.6 [v2:172.21.15.90:6818/1121461276,v1:172.21.15.90:6819/1121461276]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-24T09:15:43.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:43 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:15:43.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:43 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:15:43.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:43 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:43.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:43 smithi114 ceph-mon[32501]: from='osd.6 [v2:172.21.15.90:6818/1121461276,v1:172.21.15.90:6819/1121461276]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-24T09:15:43.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:43 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:15:43.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:43 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:15:43.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:43 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:44.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:44.220+0000 7f516fe00700 1 -- 172.21.15.90:0/739111210 <== mgr.14152 v2:172.21.15.90:6800/4261662956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f51780ff230 con 0x7f515c0a3c40 2024-04-24T09:15:44.221 INFO:teuthology.orchestra.run.smithi090.stdout:Created osd(s) 6 on host 'smithi090' 2024-04-24T09:15:44.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:44.223+0000 7f517d600700 1 -- 172.21.15.90:0/739111210 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f515c0a3c40 msgr2=0x7f515c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:44.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:44.223+0000 7f517d600700 1 --2- 172.21.15.90:0/739111210 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f515c0a3c40 0x7f515c0a6100 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5160009680 tx=0x7f5160009260 comp rx=0 tx=0).stop 2024-04-24T09:15:44.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:44.224+0000 7f517d600700 1 -- 172.21.15.90:0/739111210 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5178101a50 msgr2=0x7f5178198de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:44.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:44.224+0000 7f517d600700 1 --2- 172.21.15.90:0/739111210 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5178101a50 0x7f5178198de0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f516800b580 tx=0x7f516800bee0 comp rx=0 tx=0).stop 2024-04-24T09:15:44.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:44.224+0000 7f517d600700 1 -- 172.21.15.90:0/739111210 shutdown_connections 2024-04-24T09:15:44.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:44.224+0000 7f517d600700 1 --2- 172.21.15.90:0/739111210 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51781035f0 0x7f51781a03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:44.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:44.224+0000 7f517d600700 1 --2- 172.21.15.90:0/739111210 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f515c0a3c40 0x7f515c0a6100 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:44.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:44.224+0000 7f517d600700 1 --2- 172.21.15.90:0/739111210 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5178101a50 0x7f5178198de0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:44.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:44.224+0000 7f517d600700 1 --2- 172.21.15.90:0/739111210 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5178102c50 0x7f5178199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:44.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:44.224+0000 7f517d600700 1 -- 172.21.15.90:0/739111210 >> 172.21.15.90:0/739111210 conn(0x7f51780fcff0 msgr2=0x7f51780feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:44.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:44.224+0000 7f517d600700 1 -- 172.21.15.90:0/739111210 shutdown_connections 2024-04-24T09:15:44.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:15:44.225+0000 7f517d600700 1 -- 172.21.15.90:0/739111210 wait complete. 2024-04-24T09:15:44.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:44 smithi114 ceph-mon[32501]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-24T09:15:44.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:44 smithi114 ceph-mon[32501]: from='osd.6 [v2:172.21.15.90:6818/1121461276,v1:172.21.15.90:6819/1121461276]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-24T09:15:44.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:44 smithi114 ceph-mon[32501]: osdmap e45: 7 total, 6 up, 7 in 2024-04-24T09:15:44.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:44 smithi114 ceph-mon[32501]: from='osd.6 [v2:172.21.15.90:6818/1121461276,v1:172.21.15.90:6819/1121461276]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-24T09:15:44.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:44 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T09:15:44.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:44 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:15:44.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:44 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:44.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:44 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:45.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:44 smithi090 ceph-mon[27825]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-24T09:15:45.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:44 smithi090 ceph-mon[27825]: from='osd.6 [v2:172.21.15.90:6818/1121461276,v1:172.21.15.90:6819/1121461276]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-24T09:15:45.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:44 smithi090 ceph-mon[27825]: osdmap e45: 7 total, 6 up, 7 in 2024-04-24T09:15:45.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:44 smithi090 ceph-mon[27825]: from='osd.6 [v2:172.21.15.90:6818/1121461276,v1:172.21.15.90:6819/1121461276]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-24T09:15:45.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:44 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T09:15:45.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:44 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:15:45.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:44 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:45.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:44 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:45.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:44 smithi149 ceph-mon[32592]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-24T09:15:45.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:44 smithi149 ceph-mon[32592]: from='osd.6 [v2:172.21.15.90:6818/1121461276,v1:172.21.15.90:6819/1121461276]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-24T09:15:45.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:44 smithi149 ceph-mon[32592]: osdmap e45: 7 total, 6 up, 7 in 2024-04-24T09:15:45.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:44 smithi149 ceph-mon[32592]: from='osd.6 [v2:172.21.15.90:6818/1121461276,v1:172.21.15.90:6819/1121461276]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-24T09:15:45.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:44 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T09:15:45.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:44 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:15:45.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:44 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:45.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:44 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:45.277 DEBUG:teuthology.orchestra.run.smithi090:osd.6> sudo journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@osd.6.service 2024-04-24T09:15:45.283 INFO:tasks.cephadm:Deploying osd.7 on smithi114 with /dev/vg_nvme/lv_2... 2024-04-24T09:15:45.284 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-24T09:15:45.529 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T09:15:45.843 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:45 smithi090 ceph-mon[27825]: from='osd.6 [v2:172.21.15.90:6818/1121461276,v1:172.21.15.90:6819/1121461276]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-04-24T09:15:45.843 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:45 smithi090 ceph-mon[27825]: osdmap e46: 7 total, 6 up, 7 in 2024-04-24T09:15:45.844 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:45 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T09:15:45.844 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:45 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T09:15:45.882 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:45 smithi114 ceph-mon[32501]: from='osd.6 [v2:172.21.15.90:6818/1121461276,v1:172.21.15.90:6819/1121461276]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-04-24T09:15:45.882 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:45 smithi114 ceph-mon[32501]: osdmap e46: 7 total, 6 up, 7 in 2024-04-24T09:15:45.882 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:45 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T09:15:45.882 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:45 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T09:15:46.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:45 smithi149 ceph-mon[32592]: from='osd.6 [v2:172.21.15.90:6818/1121461276,v1:172.21.15.90:6819/1121461276]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-04-24T09:15:46.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:45 smithi149 ceph-mon[32592]: osdmap e46: 7 total, 6 up, 7 in 2024-04-24T09:15:46.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:45 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T09:15:46.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:45 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T09:15:46.905 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:46 smithi114 ceph-mon[32501]: purged_snaps scrub starts 2024-04-24T09:15:46.905 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:46 smithi114 ceph-mon[32501]: purged_snaps scrub ok 2024-04-24T09:15:46.905 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:46 smithi114 ceph-mon[32501]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-24T09:15:46.905 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:46 smithi114 ceph-mon[32501]: osd.6 [v2:172.21.15.90:6818/1121461276,v1:172.21.15.90:6819/1121461276] boot 2024-04-24T09:15:46.905 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:46 smithi114 ceph-mon[32501]: osdmap e47: 7 total, 7 up, 7 in 2024-04-24T09:15:46.905 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:46 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T09:15:47.064 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:46 smithi090 ceph-mon[27825]: purged_snaps scrub starts 2024-04-24T09:15:47.064 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:46 smithi090 ceph-mon[27825]: purged_snaps scrub ok 2024-04-24T09:15:47.064 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:46 smithi090 ceph-mon[27825]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-24T09:15:47.064 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:46 smithi090 ceph-mon[27825]: osd.6 [v2:172.21.15.90:6818/1121461276,v1:172.21.15.90:6819/1121461276] boot 2024-04-24T09:15:47.064 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:46 smithi090 ceph-mon[27825]: osdmap e47: 7 total, 7 up, 7 in 2024-04-24T09:15:47.064 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:46 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T09:15:47.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:46 smithi149 ceph-mon[32592]: purged_snaps scrub starts 2024-04-24T09:15:47.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:46 smithi149 ceph-mon[32592]: purged_snaps scrub ok 2024-04-24T09:15:47.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:46 smithi149 ceph-mon[32592]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-24T09:15:47.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:46 smithi149 ceph-mon[32592]: osd.6 [v2:172.21.15.90:6818/1121461276,v1:172.21.15.90:6819/1121461276] boot 2024-04-24T09:15:47.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:46 smithi149 ceph-mon[32592]: osdmap e47: 7 total, 7 up, 7 in 2024-04-24T09:15:47.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:46 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T09:15:47.445 INFO:teuthology.orchestra.run.smithi114.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-24T09:15:47.445 INFO:teuthology.orchestra.run.smithi114.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-24T09:15:47.445 INFO:teuthology.orchestra.run.smithi114.stderr: stderr: 10+0 records in 2024-04-24T09:15:47.445 INFO:teuthology.orchestra.run.smithi114.stderr:10+0 records out 2024-04-24T09:15:47.445 INFO:teuthology.orchestra.run.smithi114.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115095 s, 911 MB/s 2024-04-24T09:15:47.445 INFO:teuthology.orchestra.run.smithi114.stderr:--> Zapping successful for: 2024-04-24T09:15:47.743 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:47 smithi114 ceph-mon[32501]: osdmap e48: 7 total, 7 up, 7 in 2024-04-24T09:15:48.097 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi114:vg_nvme/lv_2 2024-04-24T09:15:48.101 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:47 smithi090 ceph-mon[27825]: osdmap e48: 7 total, 7 up, 7 in 2024-04-24T09:15:48.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:47 smithi149 ceph-mon[32592]: osdmap e48: 7 total, 7 up, 7 in 2024-04-24T09:15:48.344 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T09:15:48.814 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:48 smithi114 ceph-mon[32501]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:15:49.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:48 smithi090 ceph-mon[27825]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:15:49.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:48 smithi149 ceph-mon[32592]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:15:49.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:15:49.949+0000 7f2fc7e1e700 1 -- 172.21.15.114:0/561526413 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2fc0103630 msgr2=0x7f2fc0105a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:49.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:15:49.949+0000 7f2fc7e1e700 1 --2- 172.21.15.114:0/561526413 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2fc0103630 0x7f2fc0105a20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc009990 tx=0x7f2fbc009ca0 comp rx=0 tx=0).stop 2024-04-24T09:15:49.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:15:49.950+0000 7f2fc7e1e700 1 -- 172.21.15.114:0/561526413 shutdown_connections 2024-04-24T09:15:49.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:15:49.950+0000 7f2fc7e1e700 1 --2- 172.21.15.114:0/561526413 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fc0108890 0x7f2fc010aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:49.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:15:49.950+0000 7f2fc7e1e700 1 --2- 172.21.15.114:0/561526413 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2fc0103630 0x7f2fc0105a20 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:49.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:15:49.950+0000 7f2fc7e1e700 1 --2- 172.21.15.114:0/561526413 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2fc0105f60 0x7f2fc0108350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:49.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:15:49.950+0000 7f2fc7e1e700 1 -- 172.21.15.114:0/561526413 >> 172.21.15.114:0/561526413 conn(0x7f2fc00fcfe0 msgr2=0x7f2fc00ff420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:49.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:15:49.950+0000 7f2fc7e1e700 1 -- 172.21.15.114:0/561526413 shutdown_connections 2024-04-24T09:15:49.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:15:49.950+0000 7f2fc7e1e700 1 -- 172.21.15.114:0/561526413 wait complete. 2024-04-24T09:15:49.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:15:49.950+0000 7f2fc7e1e700 1 Processor -- start 2024-04-24T09:15:49.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:15:49.950+0000 7f2fc7e1e700 1 -- start start 2024-04-24T09:15:49.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:15:49.950+0000 7f2fc7e1e700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2fc0103630 0x7f2fc0198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:49.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:15:49.950+0000 7f2fc7e1e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fc0105f60 0x7f2fc01992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:49.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:15:49.950+0000 7f2fc7e1e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2fc0108890 0x7f2fc01a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:49.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:15:49.950+0000 7f2fc7e1e700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc010e790 con 0x7f2fc0105f60 2024-04-24T09:15:49.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:15:49.950+0000 7f2fc7e1e700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc010e610 con 0x7f2fc0108890 2024-04-24T09:15:49.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:15:49.950+0000 7f2fc7e1e700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc010e910 con 0x7f2fc0103630 2024-04-24T09:15:49.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:15:49.950+0000 7f2fc63bb700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2fc0108890 0x7f2fc01a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:49.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:15:49.951+0000 7f2fc63bb700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2fc0108890 0x7f2fc01a0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39216/0 (socket says 172.21.15.114:39216) 2024-04-24T09:15:49.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:15:49.951+0000 7f2fc63bb700 1 -- 172.21.15.114:0/940296516 learned_addr learned my addr 172.21.15.114:0/940296516 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-24T09:15:49.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:15:49.951+0000 7f2fc5bba700 1 --2- 172.21.15.114:0/940296516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2fc0103630 0x7f2fc0198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:49.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:15:49.951+0000 7f2fc63bb700 1 -- 172.21.15.114:0/940296516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2fc0103630 msgr2=0x7f2fc0198da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:49.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:15:49.951+0000 7f2fc63bb700 1 --2- 172.21.15.114:0/940296516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2fc0103630 0x7f2fc0198da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:49.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:15:49.951+0000 7f2fc63bb700 1 -- 172.21.15.114:0/940296516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fc0105f60 msgr2=0x7f2fc01992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:49.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:15:49.951+0000 7f2fc63bb700 1 --2- 172.21.15.114:0/940296516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fc0105f60 0x7f2fc01992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:49.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:15:49.951+0000 7f2fc63bb700 1 -- 172.21.15.114:0/940296516 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fc01a09f0 con 0x7f2fc0108890 2024-04-24T09:15:49.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:15:49.951+0000 7f2fc63bb700 1 --2- 172.21.15.114:0/940296516 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2fc0108890 0x7f2fc01a0320 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2fb800b9e0 tx=0x7f2fb800bcf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:49.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:15:49.951+0000 7f2fb6ffd700 1 -- 172.21.15.114:0/940296516 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb8019070 con 0x7f2fc0108890 2024-04-24T09:15:49.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:15:49.951+0000 7f2fb6ffd700 1 -- 172.21.15.114:0/940296516 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fb800ce10 con 0x7f2fc0108890 2024-04-24T09:15:49.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:15:49.951+0000 7f2fc7e1e700 1 -- 172.21.15.114:0/940296516 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fc01a0ce0 con 0x7f2fc0108890 2024-04-24T09:15:49.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:15:49.951+0000 7f2fb6ffd700 1 -- 172.21.15.114:0/940296516 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb8012360 con 0x7f2fc0108890 2024-04-24T09:15:49.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:15:49.951+0000 7f2fc7e1e700 1 -- 172.21.15.114:0/940296516 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fc01a1240 con 0x7f2fc0108890 2024-04-24T09:15:49.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:15:49.952+0000 7f2fc7e1e700 1 -- 172.21.15.114:0/940296516 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fc002dfe0 con 0x7f2fc0108890 2024-04-24T09:15:49.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:15:49.953+0000 7f2fb6ffd700 1 -- 172.21.15.114:0/940296516 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2fb801c070 con 0x7f2fc0108890 2024-04-24T09:15:49.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:15:49.954+0000 7f2fb6ffd700 1 --2- 172.21.15.114:0/940296516 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f2fac0a3c40 0x7f2fac0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:49.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:15:49.954+0000 7f2fb6ffd700 1 -- 172.21.15.114:0/940296516 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7f2fb80ceeb0 con 0x7f2fc0108890 2024-04-24T09:15:49.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:15:49.954+0000 7f2fc5bba700 1 --2- 172.21.15.114:0/940296516 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f2fac0a3c40 0x7f2fac0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:49.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:15:49.955+0000 7f2fc5bba700 1 --2- 172.21.15.114:0/940296516 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f2fac0a3c40 0x7f2fac0a6100 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc003890 tx=0x7f2fbc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:49.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:15:49.958+0000 7f2fb6ffd700 1 -- 172.21.15.114:0/940296516 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2fb80fdff0 con 0x7f2fc0108890 2024-04-24T09:15:50.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:15:50.163+0000 7f2fc7e1e700 1 -- 172.21.15.114:0/940296516 --> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f2fc0062f30 con 0x7f2fac0a3c40 2024-04-24T09:15:50.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:50 smithi090 ceph-mon[27825]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:15:50.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:50 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:15:50.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:50 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:15:50.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:50 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:15:50.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:50 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:50.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:50 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:50.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:50 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:15:50.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:50 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:15:50.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:50 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:15:50.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:50 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:50.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:50 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:15:50.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:50 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:15:50.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:50 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:50.913 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:50 smithi114 ceph-mon[32501]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:15:50.913 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:50 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:15:50.913 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:50 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:15:50.913 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:50 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:15:50.913 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:50 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:50.913 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:50 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:50.913 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:50 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:15:50.913 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:50 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:15:50.913 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:50 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:15:50.914 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:50 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:50.914 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:50 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:15:50.914 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:50 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:15:50.914 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:50 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:51.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:50 smithi149 ceph-mon[32592]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:15:51.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:50 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:15:51.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:50 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:15:51.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:50 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:15:51.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:50 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:51.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:50 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:51.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:50 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:15:51.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:50 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:15:51.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:50 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:15:51.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:50 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:51.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:50 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:15:51.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:50 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:15:51.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:50 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:15:51.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:51 smithi114 ceph-mon[32501]: from='client.24296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:15:51.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:51 smithi114 ceph-mon[32501]: Detected new or changed devices on smithi090 2024-04-24T09:15:51.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:51 smithi114 ceph-mon[32501]: Adjusting osd_memory_target on smithi090 to 5696M 2024-04-24T09:15:52.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:51 smithi090 ceph-mon[27825]: from='client.24296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:15:52.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:51 smithi090 ceph-mon[27825]: Detected new or changed devices on smithi090 2024-04-24T09:15:52.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:51 smithi090 ceph-mon[27825]: Adjusting osd_memory_target on smithi090 to 5696M 2024-04-24T09:15:52.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:51 smithi149 ceph-mon[32592]: from='client.24296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:15:52.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:51 smithi149 ceph-mon[32592]: Detected new or changed devices on smithi090 2024-04-24T09:15:52.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:51 smithi149 ceph-mon[32592]: Adjusting osd_memory_target on smithi090 to 5696M 2024-04-24T09:15:52.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:52 smithi090 ceph-mon[27825]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:15:52.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:52 smithi114 ceph-mon[32501]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:15:53.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:52 smithi149 ceph-mon[32592]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:15:53.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:53 smithi114 ceph-mon[32501]: from='client.? 172.21.15.114:0/2045726771' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a54e507d-d6ba-4bf2-9a7f-a7b95c3859bd"}]: dispatch 2024-04-24T09:15:53.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:53 smithi114 ceph-mon[32501]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a54e507d-d6ba-4bf2-9a7f-a7b95c3859bd"}]: dispatch 2024-04-24T09:15:53.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:53 smithi114 ceph-mon[32501]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a54e507d-d6ba-4bf2-9a7f-a7b95c3859bd"}]': finished 2024-04-24T09:15:53.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:53 smithi114 ceph-mon[32501]: osdmap e49: 8 total, 7 up, 8 in 2024-04-24T09:15:53.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:53 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T09:15:53.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:53 smithi114 ceph-mon[32501]: from='client.? 172.21.15.114:0/2725987219' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:15:54.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:53 smithi090 ceph-mon[27825]: from='client.? 172.21.15.114:0/2045726771' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a54e507d-d6ba-4bf2-9a7f-a7b95c3859bd"}]: dispatch 2024-04-24T09:15:54.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:53 smithi090 ceph-mon[27825]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a54e507d-d6ba-4bf2-9a7f-a7b95c3859bd"}]: dispatch 2024-04-24T09:15:54.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:53 smithi090 ceph-mon[27825]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a54e507d-d6ba-4bf2-9a7f-a7b95c3859bd"}]': finished 2024-04-24T09:15:54.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:53 smithi090 ceph-mon[27825]: osdmap e49: 8 total, 7 up, 8 in 2024-04-24T09:15:54.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:53 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T09:15:54.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:53 smithi090 ceph-mon[27825]: from='client.? 172.21.15.114:0/2725987219' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:15:54.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:53 smithi149 ceph-mon[32592]: from='client.? 172.21.15.114:0/2045726771' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a54e507d-d6ba-4bf2-9a7f-a7b95c3859bd"}]: dispatch 2024-04-24T09:15:54.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:53 smithi149 ceph-mon[32592]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a54e507d-d6ba-4bf2-9a7f-a7b95c3859bd"}]: dispatch 2024-04-24T09:15:54.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:53 smithi149 ceph-mon[32592]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a54e507d-d6ba-4bf2-9a7f-a7b95c3859bd"}]': finished 2024-04-24T09:15:54.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:53 smithi149 ceph-mon[32592]: osdmap e49: 8 total, 7 up, 8 in 2024-04-24T09:15:54.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:53 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T09:15:54.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:53 smithi149 ceph-mon[32592]: from='client.? 172.21.15.114:0/2725987219' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:15:54.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:54 smithi114 ceph-mon[32501]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:15:55.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:54 smithi090 ceph-mon[27825]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:15:55.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:54 smithi149 ceph-mon[32592]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:15:56.914 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:56 smithi114 ceph-mon[32501]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:15:57.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:56 smithi090 ceph-mon[27825]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:15:57.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:56 smithi149 ceph-mon[32592]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:15:58.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:15:58 smithi114 ceph-mon[32501]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:15:59.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:15:58 smithi090 ceph-mon[27825]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:15:59.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:15:58 smithi149 ceph-mon[32592]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:16:00.985 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:00 smithi114 ceph-mon[32501]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:16:00.985 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:00 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-24T09:16:00.985 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:00 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:16:01.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:00 smithi090 ceph-mon[27825]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:16:01.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:00 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-24T09:16:01.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:00 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:16:01.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:00 smithi149 ceph-mon[32592]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:16:01.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:00 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-24T09:16:01.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:00 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:16:02.059 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:01 smithi114 ceph-mon[32501]: Deploying daemon osd.7 on smithi114 2024-04-24T09:16:02.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:01 smithi090 ceph-mon[27825]: Deploying daemon osd.7 on smithi114 2024-04-24T09:16:02.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:01 smithi149 ceph-mon[32592]: Deploying daemon osd.7 on smithi114 2024-04-24T09:16:02.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:02 smithi114 ceph-mon[32501]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:16:03.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:02 smithi090 ceph-mon[27825]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:16:03.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:02 smithi149 ceph-mon[32592]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:16:03.904 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:03 smithi114 ceph-mon[32501]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:16:04.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:03 smithi090 ceph-mon[27825]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:16:04.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:03 smithi149 ceph-mon[32592]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:16:06.659 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:06 smithi114 ceph-mon[32501]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:16:06.659 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:06 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:06.659 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:06 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:06.660 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:06 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:16:06.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:06 smithi090 ceph-mon[27825]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:16:06.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:06 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:06.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:06 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:06.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:06 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:16:06.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:06 smithi149 ceph-mon[32592]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:16:06.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:06 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:06.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:06 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:06.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:06 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:16:08.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:08 smithi114 ceph-mon[32501]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:16:08.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:08 smithi090 ceph-mon[27825]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:16:08.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:08 smithi149 ceph-mon[32592]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:16:09.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:16:09.596+0000 7f2fb6ffd700 1 -- 172.21.15.114:0/940296516 <== mgr.14152 v2:172.21.15.90:6800/4261662956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2fc0062f30 con 0x7f2fac0a3c40 2024-04-24T09:16:09.596 INFO:teuthology.orchestra.run.smithi114.stdout:Created osd(s) 7 on host 'smithi114' 2024-04-24T09:16:09.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:16:09.599+0000 7f2fc7e1e700 1 -- 172.21.15.114:0/940296516 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f2fac0a3c40 msgr2=0x7f2fac0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:09.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:16:09.599+0000 7f2fc7e1e700 1 --2- 172.21.15.114:0/940296516 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f2fac0a3c40 0x7f2fac0a6100 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc003890 tx=0x7f2fbc005fd0 comp rx=0 tx=0).stop 2024-04-24T09:16:09.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:16:09.599+0000 7f2fc7e1e700 1 -- 172.21.15.114:0/940296516 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2fc0108890 msgr2=0x7f2fc01a0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:09.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:16:09.599+0000 7f2fc7e1e700 1 --2- 172.21.15.114:0/940296516 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2fc0108890 0x7f2fc01a0320 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2fb800b9e0 tx=0x7f2fb800bcf0 comp rx=0 tx=0).stop 2024-04-24T09:16:09.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:16:09.599+0000 7f2fc7e1e700 1 -- 172.21.15.114:0/940296516 shutdown_connections 2024-04-24T09:16:09.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:16:09.599+0000 7f2fc7e1e700 1 --2- 172.21.15.114:0/940296516 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2fc0108890 0x7f2fc01a0320 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:09.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:16:09.599+0000 7f2fc7e1e700 1 --2- 172.21.15.114:0/940296516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2fc0103630 0x7f2fc0198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:09.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:16:09.599+0000 7f2fc7e1e700 1 --2- 172.21.15.114:0/940296516 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f2fac0a3c40 0x7f2fac0a6100 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:09.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:16:09.599+0000 7f2fc7e1e700 1 --2- 172.21.15.114:0/940296516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fc0105f60 0x7f2fc01992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:09.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:16:09.599+0000 7f2fc7e1e700 1 -- 172.21.15.114:0/940296516 >> 172.21.15.114:0/940296516 conn(0x7f2fc00fcfe0 msgr2=0x7f2fc0103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:09.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:16:09.599+0000 7f2fc7e1e700 1 -- 172.21.15.114:0/940296516 shutdown_connections 2024-04-24T09:16:09.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:16:09.599+0000 7f2fc7e1e700 1 -- 172.21.15.114:0/940296516 wait complete. 2024-04-24T09:16:10.300 DEBUG:teuthology.orchestra.run.smithi114:osd.7> sudo journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@osd.7.service 2024-04-24T09:16:10.302 INFO:tasks.cephadm:Deploying osd.8 on smithi149 with /dev/vg_nvme/lv_2... 2024-04-24T09:16:10.302 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-24T09:16:10.551 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T09:16:10.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:10 smithi149 ceph-mon[32592]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:16:10.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:10 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:10.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:10 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:10.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:10 smithi149 ceph-mon[32592]: from='osd.7 [v2:172.21.15.114:6816/3222546280,v1:172.21.15.114:6817/3222546280]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T09:16:10.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:10 smithi149 ceph-mon[32592]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T09:16:10.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:10 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:10.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:10 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:10.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:10 smithi090 ceph-mon[27825]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:16:10.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:10 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:10.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:10 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:10.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:10 smithi090 ceph-mon[27825]: from='osd.7 [v2:172.21.15.114:6816/3222546280,v1:172.21.15.114:6817/3222546280]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T09:16:10.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:10 smithi090 ceph-mon[27825]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T09:16:10.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:10 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:10.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:10 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:10.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:10 smithi114 ceph-mon[32501]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:16:10.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:10 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:10.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:10 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:10.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:10 smithi114 ceph-mon[32501]: from='osd.7 [v2:172.21.15.114:6816/3222546280,v1:172.21.15.114:6817/3222546280]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T09:16:10.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:10 smithi114 ceph-mon[32501]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T09:16:10.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:10 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:10.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:10 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:11.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:11 smithi114 ceph-mon[32501]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-24T09:16:11.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:11 smithi114 ceph-mon[32501]: from='osd.7 [v2:172.21.15.114:6816/3222546280,v1:172.21.15.114:6817/3222546280]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-04-24T09:16:11.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:11 smithi114 ceph-mon[32501]: osdmap e50: 8 total, 7 up, 8 in 2024-04-24T09:16:11.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:11 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T09:16:11.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:11 smithi114 ceph-mon[32501]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-04-24T09:16:11.830 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:11 smithi149 ceph-mon[32592]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-24T09:16:11.831 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:11 smithi149 ceph-mon[32592]: from='osd.7 [v2:172.21.15.114:6816/3222546280,v1:172.21.15.114:6817/3222546280]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-04-24T09:16:11.831 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:11 smithi149 ceph-mon[32592]: osdmap e50: 8 total, 7 up, 8 in 2024-04-24T09:16:11.831 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:11 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T09:16:11.831 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:11 smithi149 ceph-mon[32592]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-04-24T09:16:11.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:11 smithi090 ceph-mon[27825]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-24T09:16:11.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:11 smithi090 ceph-mon[27825]: from='osd.7 [v2:172.21.15.114:6816/3222546280,v1:172.21.15.114:6817/3222546280]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-04-24T09:16:11.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:11 smithi090 ceph-mon[27825]: osdmap e50: 8 total, 7 up, 8 in 2024-04-24T09:16:11.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:11 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T09:16:11.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:11 smithi090 ceph-mon[27825]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-04-24T09:16:12.069 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:16:11 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:16:11.982+0000 7f47f0c17700 -1 osd.7 0 waiting for initial osdmap 2024-04-24T09:16:12.069 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:16:11 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:16:11.990+0000 7f47e8201700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T09:16:12.552 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-24T09:16:12.552 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-24T09:16:12.552 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2024-04-24T09:16:12.552 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2024-04-24T09:16:12.552 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0131299 s, 799 MB/s 2024-04-24T09:16:12.552 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2024-04-24T09:16:12.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:12 smithi114 ceph-mon[32501]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:16:12.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:12 smithi114 ceph-mon[32501]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-04-24T09:16:12.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:12 smithi114 ceph-mon[32501]: osdmap e51: 8 total, 7 up, 8 in 2024-04-24T09:16:12.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:12 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T09:16:12.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:12 smithi114 ceph-mon[32501]: from='osd.7 ' entity='osd.7' 2024-04-24T09:16:12.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:12 smithi090 ceph-mon[27825]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:16:12.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:12 smithi090 ceph-mon[27825]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-04-24T09:16:12.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:12 smithi090 ceph-mon[27825]: osdmap e51: 8 total, 7 up, 8 in 2024-04-24T09:16:12.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:12 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T09:16:12.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:12 smithi090 ceph-mon[27825]: from='osd.7 ' entity='osd.7' 2024-04-24T09:16:12.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:12 smithi149 ceph-mon[32592]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:16:12.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:12 smithi149 ceph-mon[32592]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-04-24T09:16:12.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:12 smithi149 ceph-mon[32592]: osdmap e51: 8 total, 7 up, 8 in 2024-04-24T09:16:12.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:12 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T09:16:12.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:12 smithi149 ceph-mon[32592]: from='osd.7 ' entity='osd.7' 2024-04-24T09:16:13.228 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi149:vg_nvme/lv_2 2024-04-24T09:16:13.468 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T09:16:13.757 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:13 smithi114 ceph-mon[32501]: purged_snaps scrub starts 2024-04-24T09:16:13.757 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:13 smithi114 ceph-mon[32501]: purged_snaps scrub ok 2024-04-24T09:16:13.757 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:13 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T09:16:13.757 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:13 smithi114 ceph-mon[32501]: osd.7 [v2:172.21.15.114:6816/3222546280,v1:172.21.15.114:6817/3222546280] boot 2024-04-24T09:16:13.758 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:13 smithi114 ceph-mon[32501]: osdmap e52: 8 total, 8 up, 8 in 2024-04-24T09:16:13.758 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:13 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T09:16:13.845 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:13 smithi149 ceph-mon[32592]: purged_snaps scrub starts 2024-04-24T09:16:13.845 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:13 smithi149 ceph-mon[32592]: purged_snaps scrub ok 2024-04-24T09:16:13.845 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:13 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T09:16:13.845 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:13 smithi149 ceph-mon[32592]: osd.7 [v2:172.21.15.114:6816/3222546280,v1:172.21.15.114:6817/3222546280] boot 2024-04-24T09:16:13.845 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:13 smithi149 ceph-mon[32592]: osdmap e52: 8 total, 8 up, 8 in 2024-04-24T09:16:13.845 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:13 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T09:16:13.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:13 smithi090 ceph-mon[27825]: purged_snaps scrub starts 2024-04-24T09:16:13.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:13 smithi090 ceph-mon[27825]: purged_snaps scrub ok 2024-04-24T09:16:13.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:13 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T09:16:13.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:13 smithi090 ceph-mon[27825]: osd.7 [v2:172.21.15.114:6816/3222546280,v1:172.21.15.114:6817/3222546280] boot 2024-04-24T09:16:13.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:13 smithi090 ceph-mon[27825]: osdmap e52: 8 total, 8 up, 8 in 2024-04-24T09:16:13.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:13 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T09:16:14.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:14 smithi149 ceph-mon[32592]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:16:14.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:14 smithi149 ceph-mon[32592]: osdmap e53: 8 total, 8 up, 8 in 2024-04-24T09:16:14.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:14 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:14.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:14 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:14.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:14 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:16:14.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:14 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:16:14.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:14 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:16:14.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:14 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:14.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:14 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:16:14.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:14 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:16:14.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:14 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:14.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:14 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:16:14.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:14 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:16:14.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:14 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:16:14.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:14 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:14.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:14 smithi114 ceph-mon[32501]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:16:14.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:14 smithi114 ceph-mon[32501]: osdmap e53: 8 total, 8 up, 8 in 2024-04-24T09:16:14.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:14 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:14.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:14 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:14.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:14 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:16:14.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:14 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:16:14.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:14 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:16:14.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:14 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:14.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:14 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:16:14.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:14 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:16:14.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:14 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:14.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:14 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:16:14.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:14 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:16:14.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:14 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:16:14.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:14 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:15.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:14 smithi090 ceph-mon[27825]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-24T09:16:15.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:14 smithi090 ceph-mon[27825]: osdmap e53: 8 total, 8 up, 8 in 2024-04-24T09:16:15.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:14 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:15.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:14 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:15.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:14 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:16:15.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:14 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:16:15.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:14 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:16:15.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:14 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:15.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:14 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:16:15.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:14 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:16:15.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:14 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:15.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:14 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:16:15.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:14 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:16:15.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:14 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:16:15.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:14 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:15.159 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:15.155+0000 7fab61f4e700 1 -- 172.21.15.149:0/1535525952 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab5c101a50 msgr2=0x7fab5c101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:15.159 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:15.155+0000 7fab61f4e700 1 --2- 172.21.15.149:0/1535525952 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab5c101a50 0x7fab5c101e70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fab50009990 tx=0x7fab50009ca0 comp rx=0 tx=0).stop 2024-04-24T09:16:15.159 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:15.156+0000 7fab61f4e700 1 -- 172.21.15.149:0/1535525952 shutdown_connections 2024-04-24T09:16:15.159 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:15.156+0000 7fab61f4e700 1 --2- 172.21.15.149:0/1535525952 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fab5c1035f0 0x7fab5c109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:15.159 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:15.156+0000 7fab61f4e700 1 --2- 172.21.15.149:0/1535525952 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab5c101a50 0x7fab5c101e70 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:15.160 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:15.156+0000 7fab61f4e700 1 --2- 172.21.15.149:0/1535525952 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab5c102c50 0x7fab5c1030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:15.160 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:15.156+0000 7fab61f4e700 1 -- 172.21.15.149:0/1535525952 >> 172.21.15.149:0/1535525952 conn(0x7fab5c0fcff0 msgr2=0x7fab5c0ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:15.160 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:15.156+0000 7fab61f4e700 1 -- 172.21.15.149:0/1535525952 shutdown_connections 2024-04-24T09:16:15.160 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:15.156+0000 7fab61f4e700 1 -- 172.21.15.149:0/1535525952 wait complete. 2024-04-24T09:16:15.160 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:15.156+0000 7fab61f4e700 1 Processor -- start 2024-04-24T09:16:15.160 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:15.156+0000 7fab61f4e700 1 -- start start 2024-04-24T09:16:15.160 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:15.156+0000 7fab61f4e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fab5c101a50 0x7fab5c198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:15.160 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:15.156+0000 7fab61f4e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab5c102c50 0x7fab5c199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:15.160 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:15.157+0000 7fab61f4e700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab5c1035f0 0x7fab5c1a0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:15.160 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:15.157+0000 7fab61f4e700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fab5c10d760 con 0x7fab5c102c50 2024-04-24T09:16:15.160 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:15.157+0000 7fab61f4e700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fab5c10d5e0 con 0x7fab5c101a50 2024-04-24T09:16:15.160 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:15.157+0000 7fab61f4e700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fab5c10d8e0 con 0x7fab5c1035f0 2024-04-24T09:16:15.160 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:15.157+0000 7fab5bfff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab5c1035f0 0x7fab5c1a0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:15.161 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:15.157+0000 7fab5bfff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab5c1035f0 0x7fab5c1a0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:55374/0 (socket says 172.21.15.149:55374) 2024-04-24T09:16:15.161 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:15.157+0000 7fab5bfff700 1 -- 172.21.15.149:0/4215488581 learned_addr learned my addr 172.21.15.149:0/4215488581 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-04-24T09:16:15.161 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:15.157+0000 7fab5b7fe700 1 --2- 172.21.15.149:0/4215488581 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fab5c101a50 0x7fab5c198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:15.161 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:15.157+0000 7fab5bfff700 1 -- 172.21.15.149:0/4215488581 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fab5c101a50 msgr2=0x7fab5c198f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:15.161 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:15.157+0000 7fab5bfff700 1 --2- 172.21.15.149:0/4215488581 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fab5c101a50 0x7fab5c198f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:15.161 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:15.157+0000 7fab5bfff700 1 -- 172.21.15.149:0/4215488581 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab5c102c50 msgr2=0x7fab5c199490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:15.161 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:15.157+0000 7fab5bfff700 1 --2- 172.21.15.149:0/4215488581 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab5c102c50 0x7fab5c199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:15.161 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:15.157+0000 7fab5bfff700 1 -- 172.21.15.149:0/4215488581 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab5c1a0bb0 con 0x7fab5c1035f0 2024-04-24T09:16:15.161 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:15.157+0000 7fab5bfff700 1 --2- 172.21.15.149:0/4215488581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab5c1035f0 0x7fab5c1a0510 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fab5400b9a0 tx=0x7fab5400bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:15.161 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:15.157+0000 7fab58ff9700 1 -- 172.21.15.149:0/4215488581 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab54019070 con 0x7fab5c1035f0 2024-04-24T09:16:15.161 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:15.157+0000 7fab58ff9700 1 -- 172.21.15.149:0/4215488581 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab54014440 con 0x7fab5c1035f0 2024-04-24T09:16:15.161 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:15.157+0000 7fab61f4e700 1 -- 172.21.15.149:0/4215488581 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab5c1a0ea0 con 0x7fab5c1035f0 2024-04-24T09:16:15.161 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:15.157+0000 7fab58ff9700 1 -- 172.21.15.149:0/4215488581 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab540075d0 con 0x7fab5c1035f0 2024-04-24T09:16:15.162 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:15.157+0000 7fab61f4e700 1 -- 172.21.15.149:0/4215488581 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab5c1a1380 con 0x7fab5c1035f0 2024-04-24T09:16:15.162 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:15.158+0000 7fab61f4e700 1 -- 172.21.15.149:0/4215488581 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab5c069810 con 0x7fab5c1035f0 2024-04-24T09:16:15.163 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:15.159+0000 7fab58ff9700 1 -- 172.21.15.149:0/4215488581 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fab5401c070 con 0x7fab5c1035f0 2024-04-24T09:16:15.163 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:15.160+0000 7fab58ff9700 1 --2- 172.21.15.149:0/4215488581 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fab440a3c40 0x7fab440a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:15.163 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:15.160+0000 7fab58ff9700 1 -- 172.21.15.149:0/4215488581 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5426+0+0 (secure 0 0 0) 0x7fab540cecc0 con 0x7fab5c1035f0 2024-04-24T09:16:15.163 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:15.160+0000 7fab5b7fe700 1 --2- 172.21.15.149:0/4215488581 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fab440a3c40 0x7fab440a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:15.163 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:15.161+0000 7fab5b7fe700 1 --2- 172.21.15.149:0/4215488581 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fab440a3c40 0x7fab440a6100 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fab50006010 tx=0x7fab50005c20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:15.165 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:15.165+0000 7fab58ff9700 1 -- 172.21.15.149:0/4215488581 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fab5c069810 con 0x7fab5c1035f0 2024-04-24T09:16:15.372 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:15.372+0000 7fab61f4e700 1 -- 172.21.15.149:0/4215488581 --> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fab5c0ff230 con 0x7fab440a3c40 2024-04-24T09:16:15.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:15 smithi149 ceph-mon[32592]: Detected new or changed devices on smithi114 2024-04-24T09:16:15.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:15 smithi149 ceph-mon[32592]: Adjusting osd_memory_target on smithi114 to 5696M 2024-04-24T09:16:15.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:15 smithi149 ceph-mon[32592]: osdmap e54: 8 total, 8 up, 8 in 2024-04-24T09:16:15.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:15 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:16:15.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:15 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:16:15.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:15 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:16:15.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:15 smithi114 ceph-mon[32501]: Detected new or changed devices on smithi114 2024-04-24T09:16:15.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:15 smithi114 ceph-mon[32501]: Adjusting osd_memory_target on smithi114 to 5696M 2024-04-24T09:16:15.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:15 smithi114 ceph-mon[32501]: osdmap e54: 8 total, 8 up, 8 in 2024-04-24T09:16:15.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:15 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:16:15.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:15 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:16:15.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:15 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:16:16.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:15 smithi090 ceph-mon[27825]: Detected new or changed devices on smithi114 2024-04-24T09:16:16.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:15 smithi090 ceph-mon[27825]: Adjusting osd_memory_target on smithi114 to 5696M 2024-04-24T09:16:16.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:15 smithi090 ceph-mon[27825]: osdmap e54: 8 total, 8 up, 8 in 2024-04-24T09:16:16.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:15 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:16:16.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:15 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:16:16.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:15 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:16:16.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:16 smithi114 ceph-mon[32501]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:16:16.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:16 smithi114 ceph-mon[32501]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:16:16.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:16 smithi114 ceph-mon[32501]: osdmap e55: 8 total, 8 up, 8 in 2024-04-24T09:16:16.959 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:16 smithi149 ceph-mon[32592]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:16:16.959 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:16 smithi149 ceph-mon[32592]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:16:16.959 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:16 smithi149 ceph-mon[32592]: osdmap e55: 8 total, 8 up, 8 in 2024-04-24T09:16:17.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:16 smithi090 ceph-mon[27825]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:16:17.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:16 smithi090 ceph-mon[27825]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:16:17.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:16 smithi090 ceph-mon[27825]: osdmap e55: 8 total, 8 up, 8 in 2024-04-24T09:16:17.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:17 smithi149 ceph-mon[32592]: osdmap e56: 8 total, 8 up, 8 in 2024-04-24T09:16:17.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:17 smithi114 ceph-mon[32501]: osdmap e56: 8 total, 8 up, 8 in 2024-04-24T09:16:18.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:17 smithi090 ceph-mon[27825]: osdmap e56: 8 total, 8 up, 8 in 2024-04-24T09:16:18.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:18 smithi149 ceph-mon[32592]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:16:18.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:18 smithi149 ceph-mon[32592]: from='client.? 172.21.15.149:0/2504176730' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9950b010-5865-4109-ae37-582b1dbd2f41"}]: dispatch 2024-04-24T09:16:18.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:18 smithi149 ceph-mon[32592]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9950b010-5865-4109-ae37-582b1dbd2f41"}]: dispatch 2024-04-24T09:16:18.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:18 smithi149 ceph-mon[32592]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9950b010-5865-4109-ae37-582b1dbd2f41"}]': finished 2024-04-24T09:16:18.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:18 smithi149 ceph-mon[32592]: osdmap e57: 9 total, 8 up, 9 in 2024-04-24T09:16:18.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:18 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T09:16:18.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:18 smithi114 ceph-mon[32501]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:16:18.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:18 smithi114 ceph-mon[32501]: from='client.? 172.21.15.149:0/2504176730' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9950b010-5865-4109-ae37-582b1dbd2f41"}]: dispatch 2024-04-24T09:16:18.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:18 smithi114 ceph-mon[32501]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9950b010-5865-4109-ae37-582b1dbd2f41"}]: dispatch 2024-04-24T09:16:18.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:18 smithi114 ceph-mon[32501]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9950b010-5865-4109-ae37-582b1dbd2f41"}]': finished 2024-04-24T09:16:18.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:18 smithi114 ceph-mon[32501]: osdmap e57: 9 total, 8 up, 9 in 2024-04-24T09:16:18.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:18 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T09:16:19.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:18 smithi090 ceph-mon[27825]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:16:19.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:18 smithi090 ceph-mon[27825]: from='client.? 172.21.15.149:0/2504176730' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9950b010-5865-4109-ae37-582b1dbd2f41"}]: dispatch 2024-04-24T09:16:19.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:18 smithi090 ceph-mon[27825]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9950b010-5865-4109-ae37-582b1dbd2f41"}]: dispatch 2024-04-24T09:16:19.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:18 smithi090 ceph-mon[27825]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9950b010-5865-4109-ae37-582b1dbd2f41"}]': finished 2024-04-24T09:16:19.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:18 smithi090 ceph-mon[27825]: osdmap e57: 9 total, 8 up, 9 in 2024-04-24T09:16:19.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:18 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T09:16:19.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:19 smithi114 ceph-mon[32501]: from='client.? 172.21.15.149:0/1292705741' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:16:20.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:19 smithi090 ceph-mon[27825]: from='client.? 172.21.15.149:0/1292705741' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:16:20.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:19 smithi149 ceph-mon[32592]: from='client.? 172.21.15.149:0/1292705741' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:16:20.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:20 smithi114 ceph-mon[32501]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:16:21.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:20 smithi090 ceph-mon[27825]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:16:21.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:20 smithi149 ceph-mon[32592]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:16:22.821 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:22 smithi149 ceph-mon[32592]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:16:22.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:22 smithi114 ceph-mon[32501]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:16:23.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:22 smithi090 ceph-mon[27825]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:16:24.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:24 smithi149 ceph-mon[32592]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:16:24.925 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:24 smithi114 ceph-mon[32501]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:16:25.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:24 smithi090 ceph-mon[27825]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:16:25.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:25 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-24T09:16:25.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:25 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:16:26.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:25 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-24T09:16:26.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:25 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:16:26.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:25 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-24T09:16:26.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:25 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:16:26.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:26 smithi149 ceph-mon[32592]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:16:26.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:26 smithi149 ceph-mon[32592]: Deploying daemon osd.8 on smithi149 2024-04-24T09:16:27.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:26 smithi090 ceph-mon[27825]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:16:27.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:26 smithi090 ceph-mon[27825]: Deploying daemon osd.8 on smithi149 2024-04-24T09:16:27.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:26 smithi114 ceph-mon[32501]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:16:27.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:26 smithi114 ceph-mon[32501]: Deploying daemon osd.8 on smithi149 2024-04-24T09:16:29.069 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:28 smithi149 ceph-mon[32592]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:16:29.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:28 smithi090 ceph-mon[27825]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:16:29.175 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:28 smithi114 ceph-mon[32501]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:16:30.077 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:29 smithi114 ceph-mon[32501]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:16:30.101 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:29 smithi090 ceph-mon[27825]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:16:30.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:29 smithi149 ceph-mon[32592]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:16:32.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:32 smithi090 ceph-mon[27825]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:16:32.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:32 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:32.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:32 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:32.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:32 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:16:32.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:32 smithi149 ceph-mon[32592]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:16:32.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:32 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:32.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:32 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:32.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:32 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:16:32.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:32 smithi114 ceph-mon[32501]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:16:32.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:32 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:32.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:32 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:32.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:32 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:16:34.805 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:34 smithi149 ceph-mon[32592]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:16:34.851 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:34 smithi090 ceph-mon[27825]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:16:34.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:34 smithi114 ceph-mon[32501]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:16:35.158 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:35.158+0000 7fab58ff9700 1 -- 172.21.15.149:0/4215488581 <== mgr.14152 v2:172.21.15.90:6800/4261662956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fab5c0ff230 con 0x7fab440a3c40 2024-04-24T09:16:35.158 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 8 on host 'smithi149' 2024-04-24T09:16:35.160 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:35.161+0000 7fab61f4e700 1 -- 172.21.15.149:0/4215488581 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fab440a3c40 msgr2=0x7fab440a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:35.160 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:35.161+0000 7fab61f4e700 1 --2- 172.21.15.149:0/4215488581 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fab440a3c40 0x7fab440a6100 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fab50006010 tx=0x7fab50005c20 comp rx=0 tx=0).stop 2024-04-24T09:16:35.160 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:35.161+0000 7fab61f4e700 1 -- 172.21.15.149:0/4215488581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab5c1035f0 msgr2=0x7fab5c1a0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:35.160 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:35.161+0000 7fab61f4e700 1 --2- 172.21.15.149:0/4215488581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab5c1035f0 0x7fab5c1a0510 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fab5400b9a0 tx=0x7fab5400bd60 comp rx=0 tx=0).stop 2024-04-24T09:16:35.161 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:35.161+0000 7fab61f4e700 1 -- 172.21.15.149:0/4215488581 shutdown_connections 2024-04-24T09:16:35.161 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:35.161+0000 7fab61f4e700 1 --2- 172.21.15.149:0/4215488581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab5c1035f0 0x7fab5c1a0510 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:35.161 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:35.161+0000 7fab61f4e700 1 --2- 172.21.15.149:0/4215488581 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fab5c101a50 0x7fab5c198f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:35.161 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:35.161+0000 7fab61f4e700 1 --2- 172.21.15.149:0/4215488581 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fab440a3c40 0x7fab440a6100 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:35.161 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:35.161+0000 7fab61f4e700 1 --2- 172.21.15.149:0/4215488581 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab5c102c50 0x7fab5c199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:35.161 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:35.161+0000 7fab61f4e700 1 -- 172.21.15.149:0/4215488581 >> 172.21.15.149:0/4215488581 conn(0x7fab5c0fcff0 msgr2=0x7fab5c0feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:35.161 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:35.161+0000 7fab61f4e700 1 -- 172.21.15.149:0/4215488581 shutdown_connections 2024-04-24T09:16:35.161 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:16:35.161+0000 7fab61f4e700 1 -- 172.21.15.149:0/4215488581 wait complete. 2024-04-24T09:16:35.815 DEBUG:teuthology.orchestra.run.smithi149:osd.8> sudo journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@osd.8.service 2024-04-24T09:16:35.836 INFO:tasks.cephadm:Deploying osd.9 on smithi090 with /dev/vg_nvme/lv_1... 2024-04-24T09:16:35.836 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-24T09:16:36.291 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:16:36.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:36 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:36.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:36 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:36.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:36 smithi090 ceph-mon[27825]: from='osd.8 [v2:172.21.15.149:6816/157615149,v1:172.21.15.149:6817/157615149]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T09:16:36.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:36 smithi090 ceph-mon[27825]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T09:16:36.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:36 smithi090 ceph-mon[27825]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:16:36.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:36 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:36.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:36 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:36.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:36 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:36.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:36 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:36.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:36 smithi114 ceph-mon[32501]: from='osd.8 [v2:172.21.15.149:6816/157615149,v1:172.21.15.149:6817/157615149]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T09:16:36.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:36 smithi114 ceph-mon[32501]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T09:16:36.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:36 smithi114 ceph-mon[32501]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:16:36.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:36 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:36.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:36 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:36.588 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:36 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:36.588 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:36 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:36.588 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:36 smithi149 ceph-mon[32592]: from='osd.8 [v2:172.21.15.149:6816/157615149,v1:172.21.15.149:6817/157615149]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T09:16:36.588 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:36 smithi149 ceph-mon[32592]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T09:16:36.588 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:36 smithi149 ceph-mon[32592]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:16:36.588 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:36 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:36.588 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:36 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:37.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:37 smithi114 ceph-mon[32501]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-24T09:16:37.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:37 smithi114 ceph-mon[32501]: osdmap e58: 9 total, 8 up, 9 in 2024-04-24T09:16:37.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:37 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T09:16:37.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:37 smithi114 ceph-mon[32501]: from='osd.8 [v2:172.21.15.149:6816/157615149,v1:172.21.15.149:6817/157615149]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-04-24T09:16:37.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:37 smithi114 ceph-mon[32501]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-04-24T09:16:37.476 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:37 smithi090 ceph-mon[27825]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-24T09:16:37.476 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:37 smithi090 ceph-mon[27825]: osdmap e58: 9 total, 8 up, 9 in 2024-04-24T09:16:37.476 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:37 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T09:16:37.476 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:37 smithi090 ceph-mon[27825]: from='osd.8 [v2:172.21.15.149:6816/157615149,v1:172.21.15.149:6817/157615149]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-04-24T09:16:37.476 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:37 smithi090 ceph-mon[27825]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-04-24T09:16:37.594 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:37 smithi149 ceph-mon[32592]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-24T09:16:37.594 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:37 smithi149 ceph-mon[32592]: osdmap e58: 9 total, 8 up, 9 in 2024-04-24T09:16:37.594 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:37 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T09:16:37.594 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:37 smithi149 ceph-mon[32592]: from='osd.8 [v2:172.21.15.149:6816/157615149,v1:172.21.15.149:6817/157615149]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-04-24T09:16:37.594 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:37 smithi149 ceph-mon[32592]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-04-24T09:16:37.861 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:16:37 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:16:37.593+0000 7fce01883700 -1 osd.8 0 waiting for initial osdmap 2024-04-24T09:16:37.861 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:16:37 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:16:37.604+0000 7fcdf966e700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T09:16:38.279 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:38 smithi090 ceph-mon[27825]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-04-24T09:16:38.279 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:38 smithi090 ceph-mon[27825]: osdmap e59: 9 total, 8 up, 9 in 2024-04-24T09:16:38.279 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:38 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T09:16:38.280 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:38 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T09:16:38.280 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:38 smithi090 ceph-mon[27825]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:16:38.280 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:38 smithi090 ceph-mon[27825]: from='osd.8 ' entity='osd.8' 2024-04-24T09:16:38.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:38 smithi114 ceph-mon[32501]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-04-24T09:16:38.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:38 smithi114 ceph-mon[32501]: osdmap e59: 9 total, 8 up, 9 in 2024-04-24T09:16:38.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:38 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T09:16:38.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:38 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T09:16:38.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:38 smithi114 ceph-mon[32501]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:16:38.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:38 smithi114 ceph-mon[32501]: from='osd.8 ' entity='osd.8' 2024-04-24T09:16:38.511 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:38 smithi149 ceph-mon[32592]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-04-24T09:16:38.511 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:38 smithi149 ceph-mon[32592]: osdmap e59: 9 total, 8 up, 9 in 2024-04-24T09:16:38.512 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:38 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T09:16:38.512 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:38 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T09:16:38.512 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:38 smithi149 ceph-mon[32592]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-24T09:16:38.512 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:38 smithi149 ceph-mon[32592]: from='osd.8 ' entity='osd.8' 2024-04-24T09:16:39.447 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:39 smithi149 ceph-mon[32592]: purged_snaps scrub starts 2024-04-24T09:16:39.447 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:39 smithi149 ceph-mon[32592]: purged_snaps scrub ok 2024-04-24T09:16:39.448 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:39 smithi149 ceph-mon[32592]: osd.8 [v2:172.21.15.149:6816/157615149,v1:172.21.15.149:6817/157615149] boot 2024-04-24T09:16:39.448 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:39 smithi149 ceph-mon[32592]: osdmap e60: 9 total, 9 up, 9 in 2024-04-24T09:16:39.448 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:39 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T09:16:39.535 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-24T09:16:39.535 INFO:teuthology.orchestra.run.smithi090.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-24T09:16:39.535 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10+0 records in 2024-04-24T09:16:39.536 INFO:teuthology.orchestra.run.smithi090.stderr:10+0 records out 2024-04-24T09:16:39.536 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0343751 s, 305 MB/s 2024-04-24T09:16:39.536 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping successful for: 2024-04-24T09:16:39.536 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:39 smithi090 ceph-mon[27825]: purged_snaps scrub starts 2024-04-24T09:16:39.536 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:39 smithi090 ceph-mon[27825]: purged_snaps scrub ok 2024-04-24T09:16:39.536 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:39 smithi090 ceph-mon[27825]: osd.8 [v2:172.21.15.149:6816/157615149,v1:172.21.15.149:6817/157615149] boot 2024-04-24T09:16:39.536 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:39 smithi090 ceph-mon[27825]: osdmap e60: 9 total, 9 up, 9 in 2024-04-24T09:16:39.536 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:39 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T09:16:39.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:39 smithi114 ceph-mon[32501]: purged_snaps scrub starts 2024-04-24T09:16:39.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:39 smithi114 ceph-mon[32501]: purged_snaps scrub ok 2024-04-24T09:16:39.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:39 smithi114 ceph-mon[32501]: osd.8 [v2:172.21.15.149:6816/157615149,v1:172.21.15.149:6817/157615149] boot 2024-04-24T09:16:39.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:39 smithi114 ceph-mon[32501]: osdmap e60: 9 total, 9 up, 9 in 2024-04-24T09:16:39.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:39 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T09:16:40.456 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:40 smithi090 ceph-mon[27825]: osdmap e61: 9 total, 9 up, 9 in 2024-04-24T09:16:40.456 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:40 smithi090 ceph-mon[27825]: pgmap v222: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:16:40.456 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:40 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:40.456 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:40 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:40.456 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:40 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:16:40.456 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:40 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:16:40.456 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:40 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:16:40.456 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:40 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:40.456 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:40 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:16:40.456 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:40 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:16:40.456 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:40 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:40.457 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:40 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:16:40.512 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi090:vg_nvme/lv_1 2024-04-24T09:16:40.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:40 smithi149 ceph-mon[32592]: osdmap e61: 9 total, 9 up, 9 in 2024-04-24T09:16:40.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:40 smithi149 ceph-mon[32592]: pgmap v222: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:16:40.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:40 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:40.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:40 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:40.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:40 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:16:40.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:40 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:16:40.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:40 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:16:40.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:40 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:40.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:40 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:16:40.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:40 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:16:40.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:40 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:40.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:40 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:16:40.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:40 smithi114 ceph-mon[32501]: osdmap e61: 9 total, 9 up, 9 in 2024-04-24T09:16:40.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:40 smithi114 ceph-mon[32501]: pgmap v222: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:16:40.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:40 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:40.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:40 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:40.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:40 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:16:40.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:40 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:16:40.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:40 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:16:40.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:40 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:40.677 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:40 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:16:40.677 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:40 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:16:40.677 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:40 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:40.677 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:40 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:16:40.960 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:16:41.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:41 smithi090 ceph-mon[27825]: Detected new or changed devices on smithi149 2024-04-24T09:16:41.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:41 smithi090 ceph-mon[27825]: Adjusting osd_memory_target on smithi149 to 1938M 2024-04-24T09:16:41.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:41 smithi090 ceph-mon[27825]: osdmap e62: 9 total, 9 up, 9 in 2024-04-24T09:16:41.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:41 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:16:41.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:41 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:16:41.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:41 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:41.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:41 smithi090 ceph-mon[27825]: osdmap e63: 9 total, 9 up, 9 in 2024-04-24T09:16:41.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:41 smithi149 ceph-mon[32592]: Detected new or changed devices on smithi149 2024-04-24T09:16:41.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:41 smithi149 ceph-mon[32592]: Adjusting osd_memory_target on smithi149 to 1938M 2024-04-24T09:16:41.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:41 smithi149 ceph-mon[32592]: osdmap e62: 9 total, 9 up, 9 in 2024-04-24T09:16:41.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:41 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:16:41.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:41 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:16:41.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:41 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:41.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:41 smithi149 ceph-mon[32592]: osdmap e63: 9 total, 9 up, 9 in 2024-04-24T09:16:41.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:41 smithi114 ceph-mon[32501]: Detected new or changed devices on smithi149 2024-04-24T09:16:41.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:41 smithi114 ceph-mon[32501]: Adjusting osd_memory_target on smithi149 to 1938M 2024-04-24T09:16:41.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:41 smithi114 ceph-mon[32501]: osdmap e62: 9 total, 9 up, 9 in 2024-04-24T09:16:41.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:41 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:16:41.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:41 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:16:41.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:41 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:16:41.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:41 smithi114 ceph-mon[32501]: osdmap e63: 9 total, 9 up, 9 in 2024-04-24T09:16:42.513 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:42 smithi149 ceph-mon[32592]: pgmap v225: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:16:42.513 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:42 smithi149 ceph-mon[32592]: osdmap e64: 9 total, 9 up, 9 in 2024-04-24T09:16:42.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:42 smithi090 ceph-mon[27825]: pgmap v225: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:16:42.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:42 smithi090 ceph-mon[27825]: osdmap e64: 9 total, 9 up, 9 in 2024-04-24T09:16:42.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:42 smithi114 ceph-mon[32501]: pgmap v225: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:16:42.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:42 smithi114 ceph-mon[32501]: osdmap e64: 9 total, 9 up, 9 in 2024-04-24T09:16:43.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:16:43.702+0000 7fb5d6e00700 1 -- 172.21.15.90:0/1238901062 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5d0108840 msgr2=0x7fb5d010acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:43.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:16:43.702+0000 7fb5d6e00700 1 --2- 172.21.15.90:0/1238901062 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5d0108840 0x7fb5d010acb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb5c800b0a0 tx=0x7fb5c800b3b0 comp rx=0 tx=0).stop 2024-04-24T09:16:43.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:16:43.702+0000 7fb5d6e00700 1 -- 172.21.15.90:0/1238901062 shutdown_connections 2024-04-24T09:16:43.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:16:43.702+0000 7fb5d6e00700 1 --2- 172.21.15.90:0/1238901062 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5d0108840 0x7fb5d010acb0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:43.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:16:43.702+0000 7fb5d6e00700 1 --2- 172.21.15.90:0/1238901062 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb5d01035e0 0x7fb5d01059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:43.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:16:43.702+0000 7fb5d6e00700 1 --2- 172.21.15.90:0/1238901062 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5d0105f10 0x7fb5d0108300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:43.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:16:43.702+0000 7fb5d6e00700 1 -- 172.21.15.90:0/1238901062 >> 172.21.15.90:0/1238901062 conn(0x7fb5d00fcff0 msgr2=0x7fb5d00ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:43.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:16:43.703+0000 7fb5d6e00700 1 -- 172.21.15.90:0/1238901062 shutdown_connections 2024-04-24T09:16:43.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:16:43.703+0000 7fb5d6e00700 1 -- 172.21.15.90:0/1238901062 wait complete. 2024-04-24T09:16:43.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:16:43.704+0000 7fb5d6e00700 1 Processor -- start 2024-04-24T09:16:43.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:16:43.704+0000 7fb5d6e00700 1 -- start start 2024-04-24T09:16:43.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:16:43.704+0000 7fb5d6e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5d01035e0 0x7fb5d0198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:43.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:16:43.705+0000 7fb5d6e00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb5d0105f10 0x7fb5d0199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:43.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:16:43.705+0000 7fb5d6e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5d0108840 0x7fb5d01a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:43.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:16:43.705+0000 7fb5d6e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d010e700 con 0x7fb5d0108840 2024-04-24T09:16:43.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:16:43.705+0000 7fb5d6e00700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d010e580 con 0x7fb5d0105f10 2024-04-24T09:16:43.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:16:43.705+0000 7fb5d6e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d010e880 con 0x7fb5d01035e0 2024-04-24T09:16:43.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:16:43.705+0000 7fb5cf400700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb5d0105f10 0x7fb5d0199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:43.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:16:43.705+0000 7fb5cfe00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5d01035e0 0x7fb5d0198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:43.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:16:43.705+0000 7fb5cf400700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb5d0105f10 0x7fb5d0199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.90:57162/0 (socket says 172.21.15.90:57162) 2024-04-24T09:16:43.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:16:43.705+0000 7fb5d4e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5d0108840 0x7fb5d01a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:43.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:16:43.705+0000 7fb5cf400700 1 -- 172.21.15.90:0/1211665286 learned_addr learned my addr 172.21.15.90:0/1211665286 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:16:43.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:16:43.705+0000 7fb5cfe00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5d01035e0 0x7fb5d0198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:46232/0 (socket says 172.21.15.90:46232) 2024-04-24T09:16:43.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:16:43.705+0000 7fb5cf400700 1 -- 172.21.15.90:0/1211665286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5d01035e0 msgr2=0x7fb5d0198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:43.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:16:43.705+0000 7fb5cf400700 1 --2- 172.21.15.90:0/1211665286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5d01035e0 0x7fb5d0198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:43.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:16:43.705+0000 7fb5cf400700 1 -- 172.21.15.90:0/1211665286 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5d0108840 msgr2=0x7fb5d01a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:43.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:16:43.705+0000 7fb5cf400700 1 --2- 172.21.15.90:0/1211665286 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5d0108840 0x7fb5d01a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:43.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:16:43.705+0000 7fb5cf400700 1 -- 172.21.15.90:0/1211665286 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5d01a09b0 con 0x7fb5d0105f10 2024-04-24T09:16:43.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:16:43.706+0000 7fb5cf400700 1 --2- 172.21.15.90:0/1211665286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb5d0105f10 0x7fb5d0199290 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc00e9a0 tx=0x7fb5bc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:43.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:16:43.706+0000 7fb5ccc00700 1 -- 172.21.15.90:0/1211665286 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc00ca10 con 0x7fb5d0105f10 2024-04-24T09:16:43.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:16:43.706+0000 7fb5d6e00700 1 -- 172.21.15.90:0/1211665286 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5d0101430 con 0x7fb5d0105f10 2024-04-24T09:16:43.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:16:43.706+0000 7fb5ccc00700 1 -- 172.21.15.90:0/1211665286 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5bc00cbb0 con 0x7fb5d0105f10 2024-04-24T09:16:43.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:16:43.706+0000 7fb5ccc00700 1 -- 172.21.15.90:0/1211665286 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc018710 con 0x7fb5d0105f10 2024-04-24T09:16:43.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:16:43.706+0000 7fb5d6e00700 1 -- 172.21.15.90:0/1211665286 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5d01019c0 con 0x7fb5d0105f10 2024-04-24T09:16:43.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:16:43.707+0000 7fb5d6e00700 1 -- 172.21.15.90:0/1211665286 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5d00697e0 con 0x7fb5d0105f10 2024-04-24T09:16:43.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:16:43.709+0000 7fb5ccc00700 1 -- 172.21.15.90:0/1211665286 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb5bc0188b0 con 0x7fb5d0105f10 2024-04-24T09:16:43.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:16:43.710+0000 7fb5ccc00700 1 --2- 172.21.15.90:0/1211665286 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fb5b40a3c40 0x7fb5b40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:43.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:16:43.711+0000 7fb5ccc00700 1 -- 172.21.15.90:0/1211665286 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5829+0+0 (secure 0 0 0) 0x7fb5bc0cfbc0 con 0x7fb5d0105f10 2024-04-24T09:16:43.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:16:43.711+0000 7fb5cfe00700 1 --2- 172.21.15.90:0/1211665286 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fb5b40a3c40 0x7fb5b40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:43.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:16:43.711+0000 7fb5cfe00700 1 --2- 172.21.15.90:0/1211665286 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fb5b40a3c40 0x7fb5b40a6100 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0004150 tx=0x7fb5c000f7a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:43.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:16:43.713+0000 7fb5ccc00700 1 -- 172.21.15.90:0/1211665286 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb5bc09ceb0 con 0x7fb5d0105f10 2024-04-24T09:16:43.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:16:43.951+0000 7fb5d6e00700 1 -- 172.21.15.90:0/1211665286 --> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fb5d0062ff0 con 0x7fb5b40a3c40 2024-04-24T09:16:44.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:44 smithi090 ceph-mon[27825]: pgmap v227: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:16:44.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:44 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:16:44.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:44 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:16:44.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:44 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:16:44.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:44 smithi149 ceph-mon[32592]: pgmap v227: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:16:44.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:44 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:16:44.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:44 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:16:44.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:44 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:16:44.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:44 smithi114 ceph-mon[32501]: pgmap v227: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:16:44.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:44 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:16:44.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:44 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:16:44.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:44 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:16:45.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:45 smithi090 ceph-mon[27825]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:16:45.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:45 smithi149 ceph-mon[32592]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:16:45.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:45 smithi114 ceph-mon[32501]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:16:46.833 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:46 smithi090 ceph-mon[27825]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T09:16:46.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:46 smithi149 ceph-mon[32592]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T09:16:46.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:46 smithi114 ceph-mon[32501]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T09:16:48.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:48 smithi090 ceph-mon[27825]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-04-24T09:16:48.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:48 smithi149 ceph-mon[32592]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-04-24T09:16:48.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:48 smithi114 ceph-mon[32501]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-04-24T09:16:49.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:49 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/4240549372' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "31e2635c-e234-421f-9757-592fd673c936"}]: dispatch 2024-04-24T09:16:49.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:49 smithi090 ceph-mon[27825]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "31e2635c-e234-421f-9757-592fd673c936"}]: dispatch 2024-04-24T09:16:49.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:49 smithi090 ceph-mon[27825]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "31e2635c-e234-421f-9757-592fd673c936"}]': finished 2024-04-24T09:16:49.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:49 smithi090 ceph-mon[27825]: osdmap e65: 10 total, 9 up, 10 in 2024-04-24T09:16:49.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:49 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T09:16:49.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:49 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/1222208833' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:16:49.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:49 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/4240549372' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "31e2635c-e234-421f-9757-592fd673c936"}]: dispatch 2024-04-24T09:16:49.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:49 smithi149 ceph-mon[32592]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "31e2635c-e234-421f-9757-592fd673c936"}]: dispatch 2024-04-24T09:16:49.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:49 smithi149 ceph-mon[32592]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "31e2635c-e234-421f-9757-592fd673c936"}]': finished 2024-04-24T09:16:49.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:49 smithi149 ceph-mon[32592]: osdmap e65: 10 total, 9 up, 10 in 2024-04-24T09:16:49.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:49 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T09:16:49.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:49 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/1222208833' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:16:49.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:49 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/4240549372' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "31e2635c-e234-421f-9757-592fd673c936"}]: dispatch 2024-04-24T09:16:49.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:49 smithi114 ceph-mon[32501]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "31e2635c-e234-421f-9757-592fd673c936"}]: dispatch 2024-04-24T09:16:49.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:49 smithi114 ceph-mon[32501]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "31e2635c-e234-421f-9757-592fd673c936"}]': finished 2024-04-24T09:16:49.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:49 smithi114 ceph-mon[32501]: osdmap e65: 10 total, 9 up, 10 in 2024-04-24T09:16:49.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:49 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T09:16:49.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:49 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/1222208833' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:16:50.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:50 smithi090 ceph-mon[27825]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T09:16:50.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:50 smithi149 ceph-mon[32592]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T09:16:50.925 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:50 smithi114 ceph-mon[32501]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T09:16:52.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:52 smithi090 ceph-mon[27825]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-04-24T09:16:52.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:52 smithi149 ceph-mon[32592]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-04-24T09:16:52.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:52 smithi114 ceph-mon[32501]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-04-24T09:16:54.851 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:54 smithi090 ceph-mon[27825]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T09:16:54.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:54 smithi149 ceph-mon[32592]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T09:16:54.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:54 smithi114 ceph-mon[32501]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T09:16:56.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:56 smithi090 ceph-mon[27825]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:16:56.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:56 smithi149 ceph-mon[32592]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:16:56.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:56 smithi114 ceph-mon[32501]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:16:58.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:58 smithi090 ceph-mon[27825]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:16:58.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:58 smithi149 ceph-mon[32592]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:16:58.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:58 smithi114 ceph-mon[32501]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:16:59.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:59 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-24T09:16:59.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:16:59 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:16:59.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:59 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-24T09:16:59.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:16:59 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:16:59.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:59 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-24T09:16:59.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:16:59 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:17:00.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:00 smithi090 ceph-mon[27825]: Deploying daemon osd.9 on smithi090 2024-04-24T09:17:00.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:00 smithi090 ceph-mon[27825]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:17:00.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:00 smithi149 ceph-mon[32592]: Deploying daemon osd.9 on smithi090 2024-04-24T09:17:00.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:00 smithi149 ceph-mon[32592]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:17:00.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:00 smithi114 ceph-mon[32501]: Deploying daemon osd.9 on smithi090 2024-04-24T09:17:00.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:00 smithi114 ceph-mon[32501]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:17:02.839 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:02 smithi090 ceph-mon[27825]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:17:02.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:02 smithi114 ceph-mon[32501]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:17:03.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:02 smithi149 ceph-mon[32592]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:17:04.921 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:04 smithi090 ceph-mon[27825]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:17:04.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:04 smithi114 ceph-mon[32501]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:17:05.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:04 smithi149 ceph-mon[32592]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:17:06.911 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:06 smithi090 ceph-mon[27825]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:17:06.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:06 smithi114 ceph-mon[32501]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:17:07.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:06 smithi149 ceph-mon[32592]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:17:08.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:08 smithi090 ceph-mon[27825]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:17:08.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:08 smithi114 ceph-mon[32501]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:17:09.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:08 smithi149 ceph-mon[32592]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:17:10.238 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:09 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:10.238 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:09 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:10.238 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:09 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:17:10.238 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:09 smithi090 ceph-mon[27825]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:17:10.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:09 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:10.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:09 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:10.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:09 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:17:10.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:09 smithi149 ceph-mon[32592]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:17:10.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:09 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:10.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:09 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:10.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:09 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:17:10.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:09 smithi114 ceph-mon[32501]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:17:12.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:12 smithi090 ceph-mon[27825]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:17:12.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:12 smithi149 ceph-mon[32592]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:17:12.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:12 smithi114 ceph-mon[32501]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:17:13.823 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:13 smithi090 ceph-mon[27825]: from='osd.9 [v2:172.21.15.90:6826/1307318076,v1:172.21.15.90:6827/1307318076]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-24T09:17:13.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:13 smithi149 ceph-mon[32592]: from='osd.9 [v2:172.21.15.90:6826/1307318076,v1:172.21.15.90:6827/1307318076]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-24T09:17:13.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:13 smithi114 ceph-mon[32501]: from='osd.9 [v2:172.21.15.90:6826/1307318076,v1:172.21.15.90:6827/1307318076]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-24T09:17:14.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:14 smithi090 ceph-mon[27825]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:17:14.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:14 smithi090 ceph-mon[27825]: from='osd.9 [v2:172.21.15.90:6826/1307318076,v1:172.21.15.90:6827/1307318076]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-24T09:17:14.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:14 smithi090 ceph-mon[27825]: osdmap e66: 10 total, 9 up, 10 in 2024-04-24T09:17:14.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:14 smithi090 ceph-mon[27825]: from='osd.9 [v2:172.21.15.90:6826/1307318076,v1:172.21.15.90:6827/1307318076]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-24T09:17:14.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:14 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T09:17:14.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:14 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:14.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:14 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:14.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:14 smithi149 ceph-mon[32592]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:17:14.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:14 smithi149 ceph-mon[32592]: from='osd.9 [v2:172.21.15.90:6826/1307318076,v1:172.21.15.90:6827/1307318076]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-24T09:17:14.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:14 smithi149 ceph-mon[32592]: osdmap e66: 10 total, 9 up, 10 in 2024-04-24T09:17:14.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:14 smithi149 ceph-mon[32592]: from='osd.9 [v2:172.21.15.90:6826/1307318076,v1:172.21.15.90:6827/1307318076]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-24T09:17:14.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:14 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T09:17:14.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:14 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:14.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:14 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:14.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:14 smithi114 ceph-mon[32501]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:17:14.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:14 smithi114 ceph-mon[32501]: from='osd.9 [v2:172.21.15.90:6826/1307318076,v1:172.21.15.90:6827/1307318076]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-24T09:17:14.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:14 smithi114 ceph-mon[32501]: osdmap e66: 10 total, 9 up, 10 in 2024-04-24T09:17:14.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:14 smithi114 ceph-mon[32501]: from='osd.9 [v2:172.21.15.90:6826/1307318076,v1:172.21.15.90:6827/1307318076]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-24T09:17:14.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:14 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T09:17:14.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:14 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:14.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:14 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:15.649 INFO:teuthology.orchestra.run.smithi090.stdout:Created osd(s) 9 on host 'smithi090' 2024-04-24T09:17:15.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:17:15.648+0000 7fb5ccc00700 1 -- 172.21.15.90:0/1211665286 <== mgr.14152 v2:172.21.15.90:6800/4261662956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb5d0062ff0 con 0x7fb5b40a3c40 2024-04-24T09:17:15.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:17:15.651+0000 7fb5d6e00700 1 -- 172.21.15.90:0/1211665286 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fb5b40a3c40 msgr2=0x7fb5b40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:15.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:17:15.651+0000 7fb5d6e00700 1 --2- 172.21.15.90:0/1211665286 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fb5b40a3c40 0x7fb5b40a6100 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0004150 tx=0x7fb5c000f7a0 comp rx=0 tx=0).stop 2024-04-24T09:17:15.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:17:15.652+0000 7fb5d6e00700 1 -- 172.21.15.90:0/1211665286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb5d0105f10 msgr2=0x7fb5d0199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:15.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:17:15.652+0000 7fb5d6e00700 1 --2- 172.21.15.90:0/1211665286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb5d0105f10 0x7fb5d0199290 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc00e9a0 tx=0x7fb5bc00ecb0 comp rx=0 tx=0).stop 2024-04-24T09:17:15.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:17:15.652+0000 7fb5d6e00700 1 -- 172.21.15.90:0/1211665286 shutdown_connections 2024-04-24T09:17:15.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:17:15.652+0000 7fb5d6e00700 1 --2- 172.21.15.90:0/1211665286 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fb5b40a3c40 0x7fb5b40a6100 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:15.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:17:15.652+0000 7fb5d6e00700 1 --2- 172.21.15.90:0/1211665286 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5d0108840 0x7fb5d01a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:15.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:17:15.652+0000 7fb5d6e00700 1 --2- 172.21.15.90:0/1211665286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5d01035e0 0x7fb5d0198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:15.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:17:15.652+0000 7fb5d6e00700 1 --2- 172.21.15.90:0/1211665286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb5d0105f10 0x7fb5d0199290 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:15.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:17:15.652+0000 7fb5d6e00700 1 -- 172.21.15.90:0/1211665286 >> 172.21.15.90:0/1211665286 conn(0x7fb5d00fcff0 msgr2=0x7fb5d0103f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:15.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:17:15.652+0000 7fb5d6e00700 1 -- 172.21.15.90:0/1211665286 shutdown_connections 2024-04-24T09:17:15.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:17:15.652+0000 7fb5d6e00700 1 -- 172.21.15.90:0/1211665286 wait complete. 2024-04-24T09:17:15.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:15 smithi090 ceph-mon[27825]: from='osd.9 [v2:172.21.15.90:6826/1307318076,v1:172.21.15.90:6827/1307318076]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-04-24T09:17:15.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:15 smithi090 ceph-mon[27825]: osdmap e67: 10 total, 9 up, 10 in 2024-04-24T09:17:15.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:15 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T09:17:15.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:15 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T09:17:15.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:15 smithi090 ceph-mon[27825]: from='osd.9 [v2:172.21.15.90:6826/1307318076,v1:172.21.15.90:6827/1307318076]' entity='osd.9' 2024-04-24T09:17:15.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:15 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:17:15.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:15 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:17:15.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:15 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:15.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:15 smithi149 ceph-mon[32592]: from='osd.9 [v2:172.21.15.90:6826/1307318076,v1:172.21.15.90:6827/1307318076]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-04-24T09:17:15.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:15 smithi149 ceph-mon[32592]: osdmap e67: 10 total, 9 up, 10 in 2024-04-24T09:17:15.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:15 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T09:17:15.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:15 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T09:17:15.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:15 smithi149 ceph-mon[32592]: from='osd.9 [v2:172.21.15.90:6826/1307318076,v1:172.21.15.90:6827/1307318076]' entity='osd.9' 2024-04-24T09:17:15.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:15 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:17:15.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:15 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:17:15.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:15 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:15.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:15 smithi114 ceph-mon[32501]: from='osd.9 [v2:172.21.15.90:6826/1307318076,v1:172.21.15.90:6827/1307318076]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-04-24T09:17:15.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:15 smithi114 ceph-mon[32501]: osdmap e67: 10 total, 9 up, 10 in 2024-04-24T09:17:15.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:15 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T09:17:15.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:15 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T09:17:15.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:15 smithi114 ceph-mon[32501]: from='osd.9 [v2:172.21.15.90:6826/1307318076,v1:172.21.15.90:6827/1307318076]' entity='osd.9' 2024-04-24T09:17:15.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:15 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:17:15.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:15 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:17:15.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:15 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:16.502 DEBUG:teuthology.orchestra.run.smithi090:osd.9> sudo journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@osd.9.service 2024-04-24T09:17:16.507 INFO:tasks.cephadm:Deploying osd.10 on smithi114 with /dev/vg_nvme/lv_1... 2024-04-24T09:17:16.507 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-24T09:17:16.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:16 smithi090 ceph-mon[27825]: purged_snaps scrub starts 2024-04-24T09:17:16.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:16 smithi090 ceph-mon[27825]: purged_snaps scrub ok 2024-04-24T09:17:16.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:16 smithi090 ceph-mon[27825]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:17:16.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:16 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T09:17:16.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:16 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:16.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:16 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:16.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:16 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:17:16.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:16 smithi090 ceph-mon[27825]: osd.9 [v2:172.21.15.90:6826/1307318076,v1:172.21.15.90:6827/1307318076] boot 2024-04-24T09:17:16.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:16 smithi090 ceph-mon[27825]: osdmap e68: 10 total, 10 up, 10 in 2024-04-24T09:17:16.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:16 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T09:17:16.752 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T09:17:16.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:16 smithi149 ceph-mon[32592]: purged_snaps scrub starts 2024-04-24T09:17:16.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:16 smithi149 ceph-mon[32592]: purged_snaps scrub ok 2024-04-24T09:17:16.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:16 smithi149 ceph-mon[32592]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:17:16.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:16 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T09:17:16.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:16 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:16.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:16 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:16.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:16 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:17:16.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:16 smithi149 ceph-mon[32592]: osd.9 [v2:172.21.15.90:6826/1307318076,v1:172.21.15.90:6827/1307318076] boot 2024-04-24T09:17:16.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:16 smithi149 ceph-mon[32592]: osdmap e68: 10 total, 10 up, 10 in 2024-04-24T09:17:16.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:16 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T09:17:16.873 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:16 smithi114 ceph-mon[32501]: purged_snaps scrub starts 2024-04-24T09:17:16.873 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:16 smithi114 ceph-mon[32501]: purged_snaps scrub ok 2024-04-24T09:17:16.873 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:16 smithi114 ceph-mon[32501]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-24T09:17:16.873 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:16 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T09:17:16.873 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:16 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:16.874 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:16 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:16.874 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:16 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:17:16.874 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:16 smithi114 ceph-mon[32501]: osd.9 [v2:172.21.15.90:6826/1307318076,v1:172.21.15.90:6827/1307318076] boot 2024-04-24T09:17:16.874 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:16 smithi114 ceph-mon[32501]: osdmap e68: 10 total, 10 up, 10 in 2024-04-24T09:17:16.874 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:16 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T09:17:18.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:18 smithi090 ceph-mon[27825]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:17:18.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:18 smithi090 ceph-mon[27825]: osdmap e69: 10 total, 10 up, 10 in 2024-04-24T09:17:18.861 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:18 smithi114 ceph-mon[32501]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:17:18.861 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:18 smithi114 ceph-mon[32501]: osdmap e69: 10 total, 10 up, 10 in 2024-04-24T09:17:18.861 INFO:teuthology.orchestra.run.smithi114.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-24T09:17:18.861 INFO:teuthology.orchestra.run.smithi114.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-24T09:17:18.862 INFO:teuthology.orchestra.run.smithi114.stderr: stderr: 10+0 records in 2024-04-24T09:17:18.862 INFO:teuthology.orchestra.run.smithi114.stderr:10+0 records out 2024-04-24T09:17:18.862 INFO:teuthology.orchestra.run.smithi114.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0116062 s, 903 MB/s 2024-04-24T09:17:18.862 INFO:teuthology.orchestra.run.smithi114.stderr: stderr: 2024-04-24T09:17:18.862 INFO:teuthology.orchestra.run.smithi114.stderr:--> Zapping successful for: 2024-04-24T09:17:18.863 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:18 smithi149 ceph-mon[32592]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:17:18.863 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:18 smithi149 ceph-mon[32592]: osdmap e69: 10 total, 10 up, 10 in 2024-04-24T09:17:19.520 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi114:vg_nvme/lv_1 2024-04-24T09:17:19.764 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T09:17:20.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:20 smithi090 ceph-mon[27825]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:17:20.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:20 smithi149 ceph-mon[32592]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:17:20.878 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:20 smithi114 ceph-mon[32501]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:17:21.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:21.411+0000 7f063a19e700 1 -- 172.21.15.114:0/1060744168 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06341046f0 msgr2=0x7f063410afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:21.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:21.411+0000 7f063a19e700 1 --2- 172.21.15.114:0/1060744168 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06341046f0 0x7f063410afa0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f062c00b0d0 tx=0x7f062c00b3e0 comp rx=0 tx=0).stop 2024-04-24T09:17:21.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:21.411+0000 7f063a19e700 1 -- 172.21.15.114:0/1060744168 shutdown_connections 2024-04-24T09:17:21.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:21.411+0000 7f063a19e700 1 --2- 172.21.15.114:0/1060744168 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06341046f0 0x7f063410afa0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:21.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:21.411+0000 7f063a19e700 1 --2- 172.21.15.114:0/1060744168 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0634102b50 0x7f0634102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:21.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:21.411+0000 7f063a19e700 1 --2- 172.21.15.114:0/1060744168 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0634103d50 0x7f06341041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:21.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:21.411+0000 7f063a19e700 1 -- 172.21.15.114:0/1060744168 >> 172.21.15.114:0/1060744168 conn(0x7f06340fe0f0 msgr2=0x7f0634100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:21.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:21.411+0000 7f063a19e700 1 -- 172.21.15.114:0/1060744168 shutdown_connections 2024-04-24T09:17:21.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:21.411+0000 7f063a19e700 1 -- 172.21.15.114:0/1060744168 wait complete. 2024-04-24T09:17:21.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:21.412+0000 7f063a19e700 1 Processor -- start 2024-04-24T09:17:21.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:21.412+0000 7f063a19e700 1 -- start start 2024-04-24T09:17:21.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:21.412+0000 7f063a19e700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0634102b50 0x7f0634076890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:21.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:21.412+0000 7f063a19e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0634103d50 0x7f0634074ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:21.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:21.412+0000 7f063a19e700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f06341046f0 0x7f0634075420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:21.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:21.412+0000 7f063a19e700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f063410e7b0 con 0x7f0634103d50 2024-04-24T09:17:21.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:21.412+0000 7f063a19e700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f063410e630 con 0x7f0634102b50 2024-04-24T09:17:21.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:21.412+0000 7f063a19e700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f063410e930 con 0x7f06341046f0 2024-04-24T09:17:21.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:21.412+0000 7f06337fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0634102b50 0x7f0634076890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:21.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:21.412+0000 7f06337fe700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0634102b50 0x7f0634076890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38808/0 (socket says 172.21.15.114:38808) 2024-04-24T09:17:21.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:21.412+0000 7f06337fe700 1 -- 172.21.15.114:0/2683641475 learned_addr learned my addr 172.21.15.114:0/2683641475 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-04-24T09:17:21.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:21.412+0000 7f0633fff700 1 --2- 172.21.15.114:0/2683641475 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f06341046f0 0x7f0634075420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:21.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:21.412+0000 7f06337fe700 1 -- 172.21.15.114:0/2683641475 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f06341046f0 msgr2=0x7f0634075420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:21.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:21.412+0000 7f06337fe700 1 --2- 172.21.15.114:0/2683641475 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f06341046f0 0x7f0634075420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:21.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:21.412+0000 7f06337fe700 1 -- 172.21.15.114:0/2683641475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0634103d50 msgr2=0x7f0634074ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:21.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:21.412+0000 7f06337fe700 1 --2- 172.21.15.114:0/2683641475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0634103d50 0x7f0634074ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:21.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:21.412+0000 7f06337fe700 1 -- 172.21.15.114:0/2683641475 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0634075cd0 con 0x7f0634102b50 2024-04-24T09:17:21.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:21.413+0000 7f06337fe700 1 --2- 172.21.15.114:0/2683641475 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0634102b50 0x7f0634076890 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f062400b500 tx=0x7f062400b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:21.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:21.413+0000 7f06317fa700 1 -- 172.21.15.114:0/2683641475 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0624015020 con 0x7f0634102b50 2024-04-24T09:17:21.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:21.413+0000 7f063a19e700 1 -- 172.21.15.114:0/2683641475 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06341a2f00 con 0x7f0634102b50 2024-04-24T09:17:21.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:21.413+0000 7f06317fa700 1 -- 172.21.15.114:0/2683641475 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0624011d10 con 0x7f0634102b50 2024-04-24T09:17:21.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:21.413+0000 7f06317fa700 1 -- 172.21.15.114:0/2683641475 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062400f2e0 con 0x7f0634102b50 2024-04-24T09:17:21.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:21.413+0000 7f063a19e700 1 -- 172.21.15.114:0/2683641475 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06341a33d0 con 0x7f0634102b50 2024-04-24T09:17:21.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:21.414+0000 7f063a19e700 1 -- 172.21.15.114:0/2683641475 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f063402dfe0 con 0x7f0634102b50 2024-04-24T09:17:21.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:21.415+0000 7f06317fa700 1 -- 172.21.15.114:0/2683641475 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0624018020 con 0x7f0634102b50 2024-04-24T09:17:21.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:21.415+0000 7f06317fa700 1 --2- 172.21.15.114:0/2683641475 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f06200a3c40 0x7f06200a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:21.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:21.416+0000 7f06317fa700 1 -- 172.21.15.114:0/2683641475 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7f06240cf870 con 0x7f0634102b50 2024-04-24T09:17:21.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:21.416+0000 7f062affd700 1 --2- 172.21.15.114:0/2683641475 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f06200a3c40 0x7f06200a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:21.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:21.417+0000 7f062affd700 1 --2- 172.21.15.114:0/2683641475 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f06200a3c40 0x7f06200a6100 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f061c005950 tx=0x7f061c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:21.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:21.421+0000 7f06317fa700 1 -- 172.21.15.114:0/2683641475 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f062401f020 con 0x7f0634102b50 2024-04-24T09:17:21.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:21.628+0000 7f063a19e700 1 -- 172.21.15.114:0/2683641475 --> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f0634062f30 con 0x7f06200a3c40 2024-04-24T09:17:22.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:22 smithi090 ceph-mon[27825]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:17:22.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:22 smithi090 ceph-mon[27825]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:17:22.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:22 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:17:22.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:22 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:17:22.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:22 smithi090 ceph-mon[27825]: Detected new or changed devices on smithi090 2024-04-24T09:17:22.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:22 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:17:22.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:22 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:22.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:22 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:22.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:22 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:22.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:22 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:22.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:22 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:22.854 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:22 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:22.854 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:22 smithi090 ceph-mon[27825]: Adjusting osd_memory_target on smithi090 to 4272M 2024-04-24T09:17:22.854 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:22 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:22.854 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:22 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:17:22.854 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:22 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:17:22.854 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:22 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:22.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:22 smithi149 ceph-mon[32592]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:17:22.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:22 smithi149 ceph-mon[32592]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:17:22.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:22 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:17:22.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:22 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:17:22.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:22 smithi149 ceph-mon[32592]: Detected new or changed devices on smithi090 2024-04-24T09:17:22.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:22 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:17:22.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:22 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:22.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:22 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:22.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:22 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:22.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:22 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:22.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:22 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:22.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:22 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:22.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:22 smithi149 ceph-mon[32592]: Adjusting osd_memory_target on smithi090 to 4272M 2024-04-24T09:17:22.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:22 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:22.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:22 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:17:22.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:22 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:17:22.863 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:22 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:22.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:22 smithi114 ceph-mon[32501]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:17:22.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:22 smithi114 ceph-mon[32501]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:17:22.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:22 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:17:22.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:22 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:17:22.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:22 smithi114 ceph-mon[32501]: Detected new or changed devices on smithi090 2024-04-24T09:17:22.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:22 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:17:22.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:22 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:22.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:22 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:22.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:22 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:22.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:22 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:22.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:22 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:22.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:22 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:22.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:22 smithi114 ceph-mon[32501]: Adjusting osd_memory_target on smithi090 to 4272M 2024-04-24T09:17:22.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:22 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:22.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:22 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:17:22.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:22 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:17:22.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:22 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:24.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:24 smithi090 ceph-mon[27825]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:17:24.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:24 smithi090 ceph-mon[27825]: from='client.? 172.21.15.114:0/253440660' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46a3b2a6-e6e5-4578-b4bb-f3e998b9bc95"}]: dispatch 2024-04-24T09:17:24.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:24 smithi090 ceph-mon[27825]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46a3b2a6-e6e5-4578-b4bb-f3e998b9bc95"}]: dispatch 2024-04-24T09:17:24.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:24 smithi090 ceph-mon[27825]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46a3b2a6-e6e5-4578-b4bb-f3e998b9bc95"}]': finished 2024-04-24T09:17:24.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:24 smithi090 ceph-mon[27825]: osdmap e70: 11 total, 10 up, 11 in 2024-04-24T09:17:24.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:24 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T09:17:24.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:24 smithi149 ceph-mon[32592]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:17:24.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:24 smithi149 ceph-mon[32592]: from='client.? 172.21.15.114:0/253440660' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46a3b2a6-e6e5-4578-b4bb-f3e998b9bc95"}]: dispatch 2024-04-24T09:17:24.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:24 smithi149 ceph-mon[32592]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46a3b2a6-e6e5-4578-b4bb-f3e998b9bc95"}]: dispatch 2024-04-24T09:17:24.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:24 smithi149 ceph-mon[32592]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46a3b2a6-e6e5-4578-b4bb-f3e998b9bc95"}]': finished 2024-04-24T09:17:24.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:24 smithi149 ceph-mon[32592]: osdmap e70: 11 total, 10 up, 11 in 2024-04-24T09:17:24.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:24 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T09:17:24.871 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:24 smithi114 ceph-mon[32501]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:17:24.871 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:24 smithi114 ceph-mon[32501]: from='client.? 172.21.15.114:0/253440660' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46a3b2a6-e6e5-4578-b4bb-f3e998b9bc95"}]: dispatch 2024-04-24T09:17:24.871 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:24 smithi114 ceph-mon[32501]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46a3b2a6-e6e5-4578-b4bb-f3e998b9bc95"}]: dispatch 2024-04-24T09:17:24.871 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:24 smithi114 ceph-mon[32501]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46a3b2a6-e6e5-4578-b4bb-f3e998b9bc95"}]': finished 2024-04-24T09:17:24.871 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:24 smithi114 ceph-mon[32501]: osdmap e70: 11 total, 10 up, 11 in 2024-04-24T09:17:24.871 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:24 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T09:17:25.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:25 smithi090 ceph-mon[27825]: from='client.? 172.21.15.114:0/2963728598' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:17:25.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:25 smithi149 ceph-mon[32592]: from='client.? 172.21.15.114:0/2963728598' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:17:25.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:25 smithi114 ceph-mon[32501]: from='client.? 172.21.15.114:0/2963728598' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:17:26.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:26 smithi090 ceph-mon[27825]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:17:26.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:26 smithi149 ceph-mon[32592]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:17:26.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:26 smithi114 ceph-mon[32501]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:17:28.818 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:28 smithi114 ceph-mon[32501]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:17:28.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:28 smithi149 ceph-mon[32592]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:17:29.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:28 smithi090 ceph-mon[27825]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:17:30.809 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:30 smithi114 ceph-mon[32501]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:17:31.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:30 smithi090 ceph-mon[27825]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:17:31.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:30 smithi149 ceph-mon[32592]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:17:32.863 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:32 smithi114 ceph-mon[32501]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:17:32.863 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:32 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-24T09:17:32.863 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:32 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:17:33.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:32 smithi090 ceph-mon[27825]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:17:33.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:32 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-24T09:17:33.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:32 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:17:33.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:32 smithi149 ceph-mon[32592]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:17:33.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:32 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-24T09:17:33.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:32 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:17:33.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:33 smithi114 ceph-mon[32501]: Deploying daemon osd.10 on smithi114 2024-04-24T09:17:34.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:33 smithi090 ceph-mon[27825]: Deploying daemon osd.10 on smithi114 2024-04-24T09:17:34.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:33 smithi149 ceph-mon[32592]: Deploying daemon osd.10 on smithi114 2024-04-24T09:17:34.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:34 smithi114 ceph-mon[32501]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:17:35.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:34 smithi090 ceph-mon[27825]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:17:35.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:34 smithi149 ceph-mon[32592]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:17:36.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:36 smithi114 ceph-mon[32501]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:17:37.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:36 smithi090 ceph-mon[27825]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:17:37.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:36 smithi149 ceph-mon[32592]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:17:38.824 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:38 smithi114 ceph-mon[32501]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:17:38.824 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:38 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:38.824 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:38 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:38.824 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:38 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:17:38.905 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:38 smithi090 ceph-mon[27825]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:17:38.905 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:38 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:38.905 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:38 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:38.905 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:38 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:17:39.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:38 smithi149 ceph-mon[32592]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:17:39.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:38 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:39.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:38 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:39.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:38 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:17:40.778 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:40 smithi114 ceph-mon[32501]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:17:40.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:40 smithi090 ceph-mon[27825]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:17:41.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:40 smithi149 ceph-mon[32592]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:17:42.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:42.015+0000 7f06317fa700 1 -- 172.21.15.114:0/2683641475 <== mgr.14152 v2:172.21.15.90:6800/4261662956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f0634062f30 con 0x7f06200a3c40 2024-04-24T09:17:42.015 INFO:teuthology.orchestra.run.smithi114.stdout:Created osd(s) 10 on host 'smithi114' 2024-04-24T09:17:42.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:42.018+0000 7f063a19e700 1 -- 172.21.15.114:0/2683641475 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f06200a3c40 msgr2=0x7f06200a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:42.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:42.018+0000 7f063a19e700 1 --2- 172.21.15.114:0/2683641475 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f06200a3c40 0x7f06200a6100 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f061c005950 tx=0x7f061c0058e0 comp rx=0 tx=0).stop 2024-04-24T09:17:42.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:42.018+0000 7f063a19e700 1 -- 172.21.15.114:0/2683641475 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0634102b50 msgr2=0x7f0634076890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:42.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:42.018+0000 7f063a19e700 1 --2- 172.21.15.114:0/2683641475 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0634102b50 0x7f0634076890 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f062400b500 tx=0x7f062400b8c0 comp rx=0 tx=0).stop 2024-04-24T09:17:42.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:42.018+0000 7f063a19e700 1 -- 172.21.15.114:0/2683641475 shutdown_connections 2024-04-24T09:17:42.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:42.018+0000 7f063a19e700 1 --2- 172.21.15.114:0/2683641475 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f06341046f0 0x7f0634075420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:42.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:42.018+0000 7f063a19e700 1 --2- 172.21.15.114:0/2683641475 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0634102b50 0x7f0634076890 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:42.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:42.018+0000 7f063a19e700 1 --2- 172.21.15.114:0/2683641475 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f06200a3c40 0x7f06200a6100 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:42.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:42.018+0000 7f063a19e700 1 --2- 172.21.15.114:0/2683641475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0634103d50 0x7f0634074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:42.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:42.018+0000 7f063a19e700 1 -- 172.21.15.114:0/2683641475 >> 172.21.15.114:0/2683641475 conn(0x7f06340fe0f0 msgr2=0x7f0634107050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:42.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:42.018+0000 7f063a19e700 1 -- 172.21.15.114:0/2683641475 shutdown_connections 2024-04-24T09:17:42.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-04-24T09:17:42.018+0000 7f063a19e700 1 -- 172.21.15.114:0/2683641475 wait complete. 2024-04-24T09:17:42.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:42 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:42.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:42 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:42.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:42 smithi114 ceph-mon[32501]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:17:42.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:42 smithi114 ceph-mon[32501]: from='osd.10 [v2:172.21.15.114:6824/1719802140,v1:172.21.15.114:6825/1719802140]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T09:17:42.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:42 smithi114 ceph-mon[32501]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T09:17:42.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:42 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:42.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:42 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:42.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:42 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:42.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:42 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:42.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:42 smithi090 ceph-mon[27825]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:17:42.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:42 smithi090 ceph-mon[27825]: from='osd.10 [v2:172.21.15.114:6824/1719802140,v1:172.21.15.114:6825/1719802140]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T09:17:42.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:42 smithi090 ceph-mon[27825]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T09:17:42.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:42 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:42.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:42 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:42.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:42 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:42.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:42 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:42.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:42 smithi149 ceph-mon[32592]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:17:42.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:42 smithi149 ceph-mon[32592]: from='osd.10 [v2:172.21.15.114:6824/1719802140,v1:172.21.15.114:6825/1719802140]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T09:17:42.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:42 smithi149 ceph-mon[32592]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T09:17:42.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:42 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:42.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:42 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:42.643 DEBUG:teuthology.orchestra.run.smithi114:osd.10> sudo journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@osd.10.service 2024-04-24T09:17:42.645 INFO:tasks.cephadm:Deploying osd.11 on smithi149 with /dev/vg_nvme/lv_1... 2024-04-24T09:17:42.645 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-24T09:17:42.889 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T09:17:43.288 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:43 smithi149 ceph-mon[32592]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-24T09:17:43.288 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:43 smithi149 ceph-mon[32592]: from='osd.10 [v2:172.21.15.114:6824/1719802140,v1:172.21.15.114:6825/1719802140]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-04-24T09:17:43.288 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:43 smithi149 ceph-mon[32592]: osdmap e71: 11 total, 10 up, 11 in 2024-04-24T09:17:43.288 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:43 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T09:17:43.288 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:43 smithi149 ceph-mon[32592]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-04-24T09:17:43.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:43 smithi114 ceph-mon[32501]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-24T09:17:43.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:43 smithi114 ceph-mon[32501]: from='osd.10 [v2:172.21.15.114:6824/1719802140,v1:172.21.15.114:6825/1719802140]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-04-24T09:17:43.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:43 smithi114 ceph-mon[32501]: osdmap e71: 11 total, 10 up, 11 in 2024-04-24T09:17:43.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:43 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T09:17:43.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:43 smithi114 ceph-mon[32501]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-04-24T09:17:43.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:43 smithi090 ceph-mon[27825]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-24T09:17:43.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:43 smithi090 ceph-mon[27825]: from='osd.10 [v2:172.21.15.114:6824/1719802140,v1:172.21.15.114:6825/1719802140]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-04-24T09:17:43.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:43 smithi090 ceph-mon[27825]: osdmap e71: 11 total, 10 up, 11 in 2024-04-24T09:17:43.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:43 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T09:17:43.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:43 smithi090 ceph-mon[27825]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]: dispatch 2024-04-24T09:17:43.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:17:43 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:17:43.589+0000 7f4992c2e700 -1 osd.10 0 waiting for initial osdmap 2024-04-24T09:17:43.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:17:43 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:17:43.602+0000 7f498aa19700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T09:17:44.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:44 smithi149 ceph-mon[32592]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-04-24T09:17:44.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:44 smithi149 ceph-mon[32592]: osdmap e72: 11 total, 10 up, 11 in 2024-04-24T09:17:44.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:44 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T09:17:44.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:44 smithi149 ceph-mon[32592]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:17:44.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:44 smithi149 ceph-mon[32592]: from='osd.10 ' entity='osd.10' 2024-04-24T09:17:44.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:44 smithi114 ceph-mon[32501]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-04-24T09:17:44.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:44 smithi114 ceph-mon[32501]: osdmap e72: 11 total, 10 up, 11 in 2024-04-24T09:17:44.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:44 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T09:17:44.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:44 smithi114 ceph-mon[32501]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:17:44.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:44 smithi114 ceph-mon[32501]: from='osd.10 ' entity='osd.10' 2024-04-24T09:17:44.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:44 smithi090 ceph-mon[27825]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-04-24T09:17:44.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:44 smithi090 ceph-mon[27825]: osdmap e72: 11 total, 10 up, 11 in 2024-04-24T09:17:44.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:44 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T09:17:44.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:44 smithi090 ceph-mon[27825]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-24T09:17:44.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:44 smithi090 ceph-mon[27825]: from='osd.10 ' entity='osd.10' 2024-04-24T09:17:44.909 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-24T09:17:44.909 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-24T09:17:44.909 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2024-04-24T09:17:44.909 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2024-04-24T09:17:44.909 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112569 s, 931 MB/s 2024-04-24T09:17:44.909 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2024-04-24T09:17:45.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:45 smithi114 ceph-mon[32501]: purged_snaps scrub starts 2024-04-24T09:17:45.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:45 smithi114 ceph-mon[32501]: purged_snaps scrub ok 2024-04-24T09:17:45.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:45 smithi114 ceph-mon[32501]: osd.10 [v2:172.21.15.114:6824/1719802140,v1:172.21.15.114:6825/1719802140] boot 2024-04-24T09:17:45.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:45 smithi114 ceph-mon[32501]: osdmap e73: 11 total, 11 up, 11 in 2024-04-24T09:17:45.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:45 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T09:17:45.559 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:45 smithi149 ceph-mon[32592]: purged_snaps scrub starts 2024-04-24T09:17:45.560 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:45 smithi149 ceph-mon[32592]: purged_snaps scrub ok 2024-04-24T09:17:45.560 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:45 smithi149 ceph-mon[32592]: osd.10 [v2:172.21.15.114:6824/1719802140,v1:172.21.15.114:6825/1719802140] boot 2024-04-24T09:17:45.560 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:45 smithi149 ceph-mon[32592]: osdmap e73: 11 total, 11 up, 11 in 2024-04-24T09:17:45.560 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:45 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T09:17:45.586 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi149:vg_nvme/lv_1 2024-04-24T09:17:45.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:45 smithi090 ceph-mon[27825]: purged_snaps scrub starts 2024-04-24T09:17:45.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:45 smithi090 ceph-mon[27825]: purged_snaps scrub ok 2024-04-24T09:17:45.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:45 smithi090 ceph-mon[27825]: osd.10 [v2:172.21.15.114:6824/1719802140,v1:172.21.15.114:6825/1719802140] boot 2024-04-24T09:17:45.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:45 smithi090 ceph-mon[27825]: osdmap e73: 11 total, 11 up, 11 in 2024-04-24T09:17:45.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:45 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T09:17:45.846 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T09:17:46.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:46 smithi114 ceph-mon[32501]: osdmap e74: 11 total, 11 up, 11 in 2024-04-24T09:17:46.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:46 smithi114 ceph-mon[32501]: pgmap v268: 1 pgs: 1 peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:17:46.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:46 smithi090 ceph-mon[27825]: osdmap e74: 11 total, 11 up, 11 in 2024-04-24T09:17:46.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:46 smithi090 ceph-mon[27825]: pgmap v268: 1 pgs: 1 peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:17:46.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:46 smithi149 ceph-mon[32592]: osdmap e74: 11 total, 11 up, 11 in 2024-04-24T09:17:46.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:46 smithi149 ceph-mon[32592]: pgmap v268: 1 pgs: 1 peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:17:47.299 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:47 smithi149 ceph-mon[32592]: osdmap e75: 11 total, 11 up, 11 in 2024-04-24T09:17:47.300 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:47 smithi149 ceph-mon[32592]: Detected new or changed devices on smithi114 2024-04-24T09:17:47.300 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:47 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:47.300 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:47 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:47.300 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:47 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:47.300 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:47 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:47.300 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:47 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:47.300 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:47 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:47.300 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:47 smithi149 ceph-mon[32592]: Adjusting osd_memory_target on smithi114 to 4272M 2024-04-24T09:17:47.300 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:47 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:47.300 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:47 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:17:47.300 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:47 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:17:47.300 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:47 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:47.300 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:47 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:17:47.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:47 smithi114 ceph-mon[32501]: osdmap e75: 11 total, 11 up, 11 in 2024-04-24T09:17:47.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:47 smithi114 ceph-mon[32501]: Detected new or changed devices on smithi114 2024-04-24T09:17:47.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:47 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:47.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:47 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:47.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:47 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:47.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:47 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:47.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:47 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:47.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:47 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:47.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:47 smithi114 ceph-mon[32501]: Adjusting osd_memory_target on smithi114 to 4272M 2024-04-24T09:17:47.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:47 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:47.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:47 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:17:47.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:47 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:17:47.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:47 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:47.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:47 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:17:47.545 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:17:47.519+0000 7f1b253aa700 1 -- 172.21.15.149:0/966269524 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b20101a80 msgr2=0x7f1b20101ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:47.545 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:17:47.519+0000 7f1b253aa700 1 --2- 172.21.15.149:0/966269524 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b20101a80 0x7f1b20101ea0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1b10009990 tx=0x7f1b10009ca0 comp rx=0 tx=0).stop 2024-04-24T09:17:47.545 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:17:47.519+0000 7f1b253aa700 1 -- 172.21.15.149:0/966269524 shutdown_connections 2024-04-24T09:17:47.546 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:17:47.519+0000 7f1b253aa700 1 --2- 172.21.15.149:0/966269524 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b20103620 0x7f1b20109ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:47.546 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:17:47.519+0000 7f1b253aa700 1 --2- 172.21.15.149:0/966269524 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b20101a80 0x7f1b20101ea0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:47.546 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:17:47.519+0000 7f1b253aa700 1 --2- 172.21.15.149:0/966269524 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b20102c80 0x7f1b201030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:47.546 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:17:47.519+0000 7f1b253aa700 1 -- 172.21.15.149:0/966269524 >> 172.21.15.149:0/966269524 conn(0x7f1b200fd040 msgr2=0x7f1b200ff460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:47.546 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:17:47.519+0000 7f1b253aa700 1 -- 172.21.15.149:0/966269524 shutdown_connections 2024-04-24T09:17:47.546 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:17:47.519+0000 7f1b253aa700 1 -- 172.21.15.149:0/966269524 wait complete. 2024-04-24T09:17:47.546 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:17:47.519+0000 7f1b253aa700 1 Processor -- start 2024-04-24T09:17:47.546 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:17:47.520+0000 7f1b253aa700 1 -- start start 2024-04-24T09:17:47.546 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:17:47.520+0000 7f1b253aa700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b20101a80 0x7f1b2019b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:47.546 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:17:47.520+0000 7f1b253aa700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b20102c80 0x7f1b2019b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:47.546 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:17:47.520+0000 7f1b253aa700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b20103620 0x7f1b201a25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:47.546 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:17:47.520+0000 7f1b253aa700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2010d6e0 con 0x7f1b20102c80 2024-04-24T09:17:47.547 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:17:47.520+0000 7f1b253aa700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2010d560 con 0x7f1b20103620 2024-04-24T09:17:47.547 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:17:47.520+0000 7f1b253aa700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2010d860 con 0x7f1b20101a80 2024-04-24T09:17:47.547 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:17:47.520+0000 7f1b1effd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b20101a80 0x7f1b2019b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:47.547 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:17:47.520+0000 7f1b1effd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b20101a80 0x7f1b2019b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:39458/0 (socket says 172.21.15.149:39458) 2024-04-24T09:17:47.547 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:17:47.520+0000 7f1b1effd700 1 -- 172.21.15.149:0/448423569 learned_addr learned my addr 172.21.15.149:0/448423569 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-04-24T09:17:47.547 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:17:47.520+0000 7f1b1effd700 1 -- 172.21.15.149:0/448423569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b20103620 msgr2=0x7f1b201a25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:47.547 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:17:47.520+0000 7f1b1effd700 1 --2- 172.21.15.149:0/448423569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b20103620 0x7f1b201a25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:47.547 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:17:47.520+0000 7f1b1effd700 1 -- 172.21.15.149:0/448423569 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b20102c80 msgr2=0x7f1b2019b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:47.547 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:17:47.520+0000 7f1b1effd700 1 --2- 172.21.15.149:0/448423569 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b20102c80 0x7f1b2019b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:47.547 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:17:47.520+0000 7f1b1effd700 1 -- 172.21.15.149:0/448423569 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b201a2c60 con 0x7f1b20101a80 2024-04-24T09:17:47.547 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:17:47.520+0000 7f1b1effd700 1 --2- 172.21.15.149:0/448423569 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b20101a80 0x7f1b2019b000 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1b100047b0 tx=0x7f1b10009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:47.547 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:17:47.520+0000 7f1b07fff700 1 -- 172.21.15.149:0/448423569 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b10027070 con 0x7f1b20101a80 2024-04-24T09:17:47.547 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:17:47.521+0000 7f1b07fff700 1 -- 172.21.15.149:0/448423569 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b10022470 con 0x7f1b20101a80 2024-04-24T09:17:47.548 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:17:47.521+0000 7f1b253aa700 1 -- 172.21.15.149:0/448423569 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b201a2ef0 con 0x7f1b20101a80 2024-04-24T09:17:47.548 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:17:47.521+0000 7f1b07fff700 1 -- 172.21.15.149:0/448423569 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b1000f6c0 con 0x7f1b20101a80 2024-04-24T09:17:47.548 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:17:47.521+0000 7f1b253aa700 1 -- 172.21.15.149:0/448423569 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b201a3360 con 0x7f1b20101a80 2024-04-24T09:17:47.548 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:17:47.522+0000 7f1b253aa700 1 -- 172.21.15.149:0/448423569 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b200697e0 con 0x7f1b20101a80 2024-04-24T09:17:47.548 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:17:47.523+0000 7f1b07fff700 1 -- 172.21.15.149:0/448423569 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1b1002a070 con 0x7f1b20101a80 2024-04-24T09:17:47.548 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:17:47.523+0000 7f1b07fff700 1 --2- 172.21.15.149:0/448423569 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f1b080a3c40 0x7f1b080a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:47.548 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:17:47.523+0000 7f1b07fff700 1 -- 172.21.15.149:0/448423569 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6694+0+0 (secure 0 0 0) 0x7f1b100d0eb0 con 0x7f1b20101a80 2024-04-24T09:17:47.548 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:17:47.524+0000 7f1b1e7fc700 1 --2- 172.21.15.149:0/448423569 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f1b080a3c40 0x7f1b080a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:47.548 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:17:47.525+0000 7f1b1e7fc700 1 --2- 172.21.15.149:0/448423569 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f1b080a3c40 0x7f1b080a6100 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1b140097b0 tx=0x7f1b14006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:47.548 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:17:47.528+0000 7f1b07fff700 1 -- 172.21.15.149:0/448423569 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1b1005eff0 con 0x7f1b20101a80 2024-04-24T09:17:47.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:47 smithi090 ceph-mon[27825]: osdmap e75: 11 total, 11 up, 11 in 2024-04-24T09:17:47.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:47 smithi090 ceph-mon[27825]: Detected new or changed devices on smithi114 2024-04-24T09:17:47.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:47 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:47.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:47 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:47.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:47 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:47.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:47 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:47.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:47 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:47.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:47 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:17:47.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:47 smithi090 ceph-mon[27825]: Adjusting osd_memory_target on smithi114 to 4272M 2024-04-24T09:17:47.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:47 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:47.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:47 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:17:47.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:47 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:17:47.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:47 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:47.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:47 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:17:47.739 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:17:47.738+0000 7f1b253aa700 1 -- 172.21.15.149:0/448423569 --> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f1b200ff1b0 con 0x7f1b080a3c40 2024-04-24T09:17:48.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:48 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:17:48.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:48 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:17:48.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:48 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:48.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:48 smithi114 ceph-mon[32501]: osdmap e76: 11 total, 11 up, 11 in 2024-04-24T09:17:48.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:48 smithi114 ceph-mon[32501]: pgmap v271: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:17:48.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:48 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:17:48.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:48 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:17:48.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:48 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:17:48.479 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:48 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:17:48.479 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:48 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:17:48.480 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:48 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:48.480 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:48 smithi149 ceph-mon[32592]: osdmap e76: 11 total, 11 up, 11 in 2024-04-24T09:17:48.480 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:48 smithi149 ceph-mon[32592]: pgmap v271: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:17:48.480 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:48 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:17:48.480 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:48 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:17:48.480 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:48 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:17:48.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:48 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:17:48.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:48 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:17:48.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:48 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:17:48.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:48 smithi090 ceph-mon[27825]: osdmap e76: 11 total, 11 up, 11 in 2024-04-24T09:17:48.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:48 smithi090 ceph-mon[27825]: pgmap v271: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:17:48.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:48 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T09:17:48.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:48 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T09:17:48.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:48 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:17:49.553 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:49 smithi149 ceph-mon[32592]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:17:49.553 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:49 smithi149 ceph-mon[32592]: osdmap e77: 11 total, 11 up, 11 in 2024-04-24T09:17:49.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:49 smithi090 ceph-mon[27825]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:17:49.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:49 smithi090 ceph-mon[27825]: osdmap e77: 11 total, 11 up, 11 in 2024-04-24T09:17:49.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:49 smithi114 ceph-mon[32501]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:17:49.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:49 smithi114 ceph-mon[32501]: osdmap e77: 11 total, 11 up, 11 in 2024-04-24T09:17:50.598 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:50 smithi090 ceph-mon[27825]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 133 KiB/s, 0 objects/s recovering 2024-04-24T09:17:50.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:50 smithi149 ceph-mon[32592]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 133 KiB/s, 0 objects/s recovering 2024-04-24T09:17:50.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:50 smithi114 ceph-mon[32501]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 133 KiB/s, 0 objects/s recovering 2024-04-24T09:17:51.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:51 smithi090 ceph-mon[27825]: from='client.? 172.21.15.149:0/2786804200' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ccd4456b-e947-46cf-a779-148ffa289b58"}]: dispatch 2024-04-24T09:17:51.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:51 smithi090 ceph-mon[27825]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ccd4456b-e947-46cf-a779-148ffa289b58"}]: dispatch 2024-04-24T09:17:51.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:51 smithi090 ceph-mon[27825]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ccd4456b-e947-46cf-a779-148ffa289b58"}]': finished 2024-04-24T09:17:51.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:51 smithi090 ceph-mon[27825]: osdmap e78: 12 total, 11 up, 12 in 2024-04-24T09:17:51.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:51 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T09:17:51.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:51 smithi090 ceph-mon[27825]: from='client.? 172.21.15.149:0/2416396355' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:17:51.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:51 smithi149 ceph-mon[32592]: from='client.? 172.21.15.149:0/2786804200' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ccd4456b-e947-46cf-a779-148ffa289b58"}]: dispatch 2024-04-24T09:17:51.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:51 smithi149 ceph-mon[32592]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ccd4456b-e947-46cf-a779-148ffa289b58"}]: dispatch 2024-04-24T09:17:51.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:51 smithi149 ceph-mon[32592]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ccd4456b-e947-46cf-a779-148ffa289b58"}]': finished 2024-04-24T09:17:51.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:51 smithi149 ceph-mon[32592]: osdmap e78: 12 total, 11 up, 12 in 2024-04-24T09:17:51.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:51 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T09:17:51.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:51 smithi149 ceph-mon[32592]: from='client.? 172.21.15.149:0/2416396355' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:17:51.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:51 smithi114 ceph-mon[32501]: from='client.? 172.21.15.149:0/2786804200' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ccd4456b-e947-46cf-a779-148ffa289b58"}]: dispatch 2024-04-24T09:17:51.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:51 smithi114 ceph-mon[32501]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ccd4456b-e947-46cf-a779-148ffa289b58"}]: dispatch 2024-04-24T09:17:51.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:51 smithi114 ceph-mon[32501]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ccd4456b-e947-46cf-a779-148ffa289b58"}]': finished 2024-04-24T09:17:51.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:51 smithi114 ceph-mon[32501]: osdmap e78: 12 total, 11 up, 12 in 2024-04-24T09:17:51.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:51 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T09:17:51.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:51 smithi114 ceph-mon[32501]: from='client.? 172.21.15.149:0/2416396355' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T09:17:52.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:52 smithi090 ceph-mon[27825]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-04-24T09:17:52.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:52 smithi149 ceph-mon[32592]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-04-24T09:17:52.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:52 smithi114 ceph-mon[32501]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-04-24T09:17:54.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:54 smithi090 ceph-mon[27825]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-04-24T09:17:54.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:54 smithi149 ceph-mon[32592]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-04-24T09:17:54.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:54 smithi114 ceph-mon[32501]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-04-24T09:17:56.597 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:56 smithi149 ceph-mon[32592]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T09:17:56.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:56 smithi090 ceph-mon[27825]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T09:17:56.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:56 smithi114 ceph-mon[32501]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T09:17:58.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:58 smithi090 ceph-mon[27825]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-24T09:17:58.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:58 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-24T09:17:58.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:58 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:17:58.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:58 smithi149 ceph-mon[32592]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-24T09:17:58.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:58 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-24T09:17:58.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:58 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:17:58.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:58 smithi114 ceph-mon[32501]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-24T09:17:58.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:58 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-24T09:17:58.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:58 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:17:59.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:17:59 smithi090 ceph-mon[27825]: Deploying daemon osd.11 on smithi149 2024-04-24T09:17:59.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:17:59 smithi149 ceph-mon[32592]: Deploying daemon osd.11 on smithi149 2024-04-24T09:17:59.925 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:17:59 smithi114 ceph-mon[32501]: Deploying daemon osd.11 on smithi149 2024-04-24T09:18:00.809 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:00 smithi149 ceph-mon[32592]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:18:00.827 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:00 smithi114 ceph-mon[32501]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:18:00.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:00 smithi090 ceph-mon[27825]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:18:02.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:02 smithi090 ceph-mon[27825]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:18:02.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:02 smithi149 ceph-mon[32592]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:18:02.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:02 smithi114 ceph-mon[32501]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:18:04.817 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:04 smithi149 ceph-mon[32592]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:18:04.817 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:04 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:04.817 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:04 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:04.817 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:04 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:18:04.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:04 smithi090 ceph-mon[27825]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:18:04.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:04 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:04.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:04 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:04.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:04 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:18:04.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:04 smithi114 ceph-mon[32501]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:18:04.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:04 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:04.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:04 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:04.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:04 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:18:06.687 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:06 smithi149 ceph-mon[32592]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:18:06.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:06 smithi090 ceph-mon[27825]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:18:06.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:06 smithi114 ceph-mon[32501]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:18:07.833 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:07.833+0000 7f1b07fff700 1 -- 172.21.15.149:0/448423569 <== mgr.14152 v2:172.21.15.90:6800/4261662956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f1b200ff1b0 con 0x7f1b080a3c40 2024-04-24T09:18:07.833 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 11 on host 'smithi149' 2024-04-24T09:18:07.836 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:07.836+0000 7f1b253aa700 1 -- 172.21.15.149:0/448423569 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f1b080a3c40 msgr2=0x7f1b080a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:07.836 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:07.836+0000 7f1b253aa700 1 --2- 172.21.15.149:0/448423569 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f1b080a3c40 0x7f1b080a6100 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1b140097b0 tx=0x7f1b14006d20 comp rx=0 tx=0).stop 2024-04-24T09:18:07.836 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:07.836+0000 7f1b253aa700 1 -- 172.21.15.149:0/448423569 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b20101a80 msgr2=0x7f1b2019b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:07.836 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:07.836+0000 7f1b253aa700 1 --2- 172.21.15.149:0/448423569 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b20101a80 0x7f1b2019b000 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1b100047b0 tx=0x7f1b10009f90 comp rx=0 tx=0).stop 2024-04-24T09:18:07.836 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:07.836+0000 7f1b253aa700 1 -- 172.21.15.149:0/448423569 shutdown_connections 2024-04-24T09:18:07.836 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:07.836+0000 7f1b253aa700 1 --2- 172.21.15.149:0/448423569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1b20103620 0x7f1b201a25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:07.836 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:07.836+0000 7f1b253aa700 1 --2- 172.21.15.149:0/448423569 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b20101a80 0x7f1b2019b000 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:07.836 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:07.836+0000 7f1b253aa700 1 --2- 172.21.15.149:0/448423569 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f1b080a3c40 0x7f1b080a6100 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:07.837 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:07.836+0000 7f1b253aa700 1 --2- 172.21.15.149:0/448423569 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b20102c80 0x7f1b2019b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:07.837 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:07.836+0000 7f1b253aa700 1 -- 172.21.15.149:0/448423569 >> 172.21.15.149:0/448423569 conn(0x7f1b200fd040 msgr2=0x7f1b200feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:07.837 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:07.836+0000 7f1b253aa700 1 -- 172.21.15.149:0/448423569 shutdown_connections 2024-04-24T09:18:07.837 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:07.836+0000 7f1b253aa700 1 -- 172.21.15.149:0/448423569 wait complete. 2024-04-24T09:18:08.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:07 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:08.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:07 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:08.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:07 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:18:08.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:07 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:18:08.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:07 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:08.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:07 smithi090 ceph-mon[27825]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:18:08.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:07 smithi090 ceph-mon[27825]: from='osd.11 [v2:172.21.15.149:6824/1737275166,v1:172.21.15.149:6825/1737275166]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T09:18:08.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:07 smithi090 ceph-mon[27825]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T09:18:08.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:07 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:08.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:07 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:08.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:07 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:18:08.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:07 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:08.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:07 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:08.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:07 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:18:08.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:07 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:18:08.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:07 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:08.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:07 smithi149 ceph-mon[32592]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:18:08.363 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:07 smithi149 ceph-mon[32592]: from='osd.11 [v2:172.21.15.149:6824/1737275166,v1:172.21.15.149:6825/1737275166]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T09:18:08.363 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:07 smithi149 ceph-mon[32592]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T09:18:08.363 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:07 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:08.363 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:07 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:08.363 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:07 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:18:08.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:07 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:08.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:07 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:08.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:07 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:18:08.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:07 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:18:08.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:07 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:08.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:07 smithi114 ceph-mon[32501]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:18:08.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:07 smithi114 ceph-mon[32501]: from='osd.11 [v2:172.21.15.149:6824/1737275166,v1:172.21.15.149:6825/1737275166]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T09:18:08.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:07 smithi114 ceph-mon[32501]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T09:18:08.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:07 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:08.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:07 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:08.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:07 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:18:08.513 DEBUG:teuthology.orchestra.run.smithi149:osd.11> sudo journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@osd.11.service 2024-04-24T09:18:08.516 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-24T09:18:08.516 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-24T09:18:08.970 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:18:09.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:09 smithi149 ceph-mon[32592]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-24T09:18:09.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:09 smithi149 ceph-mon[32592]: from='osd.11 [v2:172.21.15.149:6824/1737275166,v1:172.21.15.149:6825/1737275166]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-04-24T09:18:09.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:09 smithi149 ceph-mon[32592]: osdmap e79: 12 total, 11 up, 12 in 2024-04-24T09:18:09.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:09 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T09:18:09.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:09 smithi149 ceph-mon[32592]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-04-24T09:18:09.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:09 smithi090 ceph-mon[27825]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-24T09:18:09.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:09 smithi090 ceph-mon[27825]: from='osd.11 [v2:172.21.15.149:6824/1737275166,v1:172.21.15.149:6825/1737275166]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-04-24T09:18:09.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:09 smithi090 ceph-mon[27825]: osdmap e79: 12 total, 11 up, 12 in 2024-04-24T09:18:09.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:09 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T09:18:09.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:09 smithi090 ceph-mon[27825]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-04-24T09:18:09.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:09 smithi114 ceph-mon[32501]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-24T09:18:09.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:09 smithi114 ceph-mon[32501]: from='osd.11 [v2:172.21.15.149:6824/1737275166,v1:172.21.15.149:6825/1737275166]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-04-24T09:18:09.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:09 smithi114 ceph-mon[32501]: osdmap e79: 12 total, 11 up, 12 in 2024-04-24T09:18:09.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:09 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T09:18:09.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:09 smithi114 ceph-mon[32501]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-04-24T09:18:09.671 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:18:09 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:18:09.450+0000 7f5ea03f4700 -1 osd.11 0 waiting for initial osdmap 2024-04-24T09:18:09.672 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:18:09 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:18:09.465+0000 7f5e981df700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T09:18:10.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:10 smithi090 ceph-mon[27825]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-04-24T09:18:10.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:10 smithi090 ceph-mon[27825]: osdmap e80: 12 total, 11 up, 12 in 2024-04-24T09:18:10.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:10 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T09:18:10.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:10 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T09:18:10.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:10 smithi090 ceph-mon[27825]: from='osd.11 ' entity='osd.11' 2024-04-24T09:18:10.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:10 smithi090 ceph-mon[27825]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:18:10.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:10 smithi149 ceph-mon[32592]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-04-24T09:18:10.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:10 smithi149 ceph-mon[32592]: osdmap e80: 12 total, 11 up, 12 in 2024-04-24T09:18:10.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:10 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T09:18:10.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:10 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T09:18:10.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:10 smithi149 ceph-mon[32592]: from='osd.11 ' entity='osd.11' 2024-04-24T09:18:10.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:10 smithi149 ceph-mon[32592]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:18:10.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:10 smithi114 ceph-mon[32501]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-04-24T09:18:10.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:10 smithi114 ceph-mon[32501]: osdmap e80: 12 total, 11 up, 12 in 2024-04-24T09:18:10.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:10 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T09:18:10.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:10 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T09:18:10.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:10 smithi114 ceph-mon[32501]: from='osd.11 ' entity='osd.11' 2024-04-24T09:18:10.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:10 smithi114 ceph-mon[32501]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-24T09:18:11.177 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:11 smithi090 ceph-mon[27825]: purged_snaps scrub starts 2024-04-24T09:18:11.178 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:11 smithi090 ceph-mon[27825]: purged_snaps scrub ok 2024-04-24T09:18:11.178 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:11 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T09:18:11.178 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:11 smithi090 ceph-mon[27825]: osd.11 [v2:172.21.15.149:6824/1737275166,v1:172.21.15.149:6825/1737275166] boot 2024-04-24T09:18:11.178 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:11 smithi090 ceph-mon[27825]: osdmap e81: 12 total, 12 up, 12 in 2024-04-24T09:18:11.178 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:11 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T09:18:11.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:11 smithi149 ceph-mon[32592]: purged_snaps scrub starts 2024-04-24T09:18:11.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:11 smithi149 ceph-mon[32592]: purged_snaps scrub ok 2024-04-24T09:18:11.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:11 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T09:18:11.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:11 smithi149 ceph-mon[32592]: osd.11 [v2:172.21.15.149:6824/1737275166,v1:172.21.15.149:6825/1737275166] boot 2024-04-24T09:18:11.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:11 smithi149 ceph-mon[32592]: osdmap e81: 12 total, 12 up, 12 in 2024-04-24T09:18:11.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:11 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T09:18:11.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:11 smithi114 ceph-mon[32501]: purged_snaps scrub starts 2024-04-24T09:18:11.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:11 smithi114 ceph-mon[32501]: purged_snaps scrub ok 2024-04-24T09:18:11.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:11 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T09:18:11.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:11 smithi114 ceph-mon[32501]: osd.11 [v2:172.21.15.149:6824/1737275166,v1:172.21.15.149:6825/1737275166] boot 2024-04-24T09:18:11.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:11 smithi114 ceph-mon[32501]: osdmap e81: 12 total, 12 up, 12 in 2024-04-24T09:18:11.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:11 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T09:18:11.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.461+0000 7ff6ee800700 1 -- 172.21.15.90:0/1070175728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6e8102b60 msgr2=0x7ff6e8102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:11.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.461+0000 7ff6ee800700 1 --2- 172.21.15.90:0/1070175728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6e8102b60 0x7ff6e8102f80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff6d80099c0 tx=0x7ff6d8009cd0 comp rx=0 tx=0).stop 2024-04-24T09:18:11.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.461+0000 7ff6ee800700 1 -- 172.21.15.90:0/1070175728 shutdown_connections 2024-04-24T09:18:11.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.461+0000 7ff6ee800700 1 --2- 172.21.15.90:0/1070175728 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff6e8104700 0x7ff6e810afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:11.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.461+0000 7ff6ee800700 1 --2- 172.21.15.90:0/1070175728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6e8102b60 0x7ff6e8102f80 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:11.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.461+0000 7ff6ee800700 1 --2- 172.21.15.90:0/1070175728 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff6e8103d60 0x7ff6e81041c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:11.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.461+0000 7ff6ee800700 1 -- 172.21.15.90:0/1070175728 >> 172.21.15.90:0/1070175728 conn(0x7ff6e80fe0c0 msgr2=0x7ff6e8100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:11.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.461+0000 7ff6ee800700 1 -- 172.21.15.90:0/1070175728 shutdown_connections 2024-04-24T09:18:11.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.461+0000 7ff6ee800700 1 -- 172.21.15.90:0/1070175728 wait complete. 2024-04-24T09:18:11.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.463+0000 7ff6ee800700 1 Processor -- start 2024-04-24T09:18:11.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.463+0000 7ff6ee800700 1 -- start start 2024-04-24T09:18:11.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.463+0000 7ff6ee800700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff6e8102b60 0x7ff6e8076890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:11.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.463+0000 7ff6ee800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6e8103d60 0x7ff6e8074ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:11.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.463+0000 7ff6ee800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff6e8104700 0x7ff6e8075420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:11.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.463+0000 7ff6ee800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e810e720 con 0x7ff6e8103d60 2024-04-24T09:18:11.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.463+0000 7ff6ee800700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e810e5a0 con 0x7ff6e8102b60 2024-04-24T09:18:11.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.463+0000 7ff6ee800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e810e8a0 con 0x7ff6e8104700 2024-04-24T09:18:11.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.463+0000 7ff6e6a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6e8103d60 0x7ff6e8074ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:11.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.464+0000 7ff6e7400700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff6e8102b60 0x7ff6e8076890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:11.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.464+0000 7ff6e7e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff6e8104700 0x7ff6e8075420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:11.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.464+0000 7ff6e6a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6e8103d60 0x7ff6e8074ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36868/0 (socket says 172.21.15.90:36868) 2024-04-24T09:18:11.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.464+0000 7ff6e6a00700 1 -- 172.21.15.90:0/1012553545 learned_addr learned my addr 172.21.15.90:0/1012553545 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:18:11.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.464+0000 7ff6e7400700 1 -- 172.21.15.90:0/1012553545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff6e8104700 msgr2=0x7ff6e8075420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:11.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.464+0000 7ff6e7400700 1 --2- 172.21.15.90:0/1012553545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff6e8104700 0x7ff6e8075420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:11.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.464+0000 7ff6e7400700 1 -- 172.21.15.90:0/1012553545 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6e8103d60 msgr2=0x7ff6e8074ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:11.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.464+0000 7ff6e7400700 1 --2- 172.21.15.90:0/1012553545 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6e8103d60 0x7ff6e8074ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:11.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.464+0000 7ff6e7400700 1 -- 172.21.15.90:0/1012553545 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6e8075cd0 con 0x7ff6e8102b60 2024-04-24T09:18:11.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.464+0000 7ff6e6a00700 1 --2- 172.21.15.90:0/1012553545 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6e8103d60 0x7ff6e8074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:18:11.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.464+0000 7ff6e7400700 1 --2- 172.21.15.90:0/1012553545 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff6e8102b60 0x7ff6e8076890 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8003790 tx=0x7ff6d8009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:11.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.465+0000 7ff6dfe00700 1 -- 172.21.15.90:0/1012553545 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d8009e60 con 0x7ff6e8102b60 2024-04-24T09:18:11.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.465+0000 7ff6dfe00700 1 -- 172.21.15.90:0/1012553545 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6d8003960 con 0x7ff6e8102b60 2024-04-24T09:18:11.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.465+0000 7ff6ee800700 1 -- 172.21.15.90:0/1012553545 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6e81a2e60 con 0x7ff6e8102b60 2024-04-24T09:18:11.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.465+0000 7ff6dfe00700 1 -- 172.21.15.90:0/1012553545 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d8005560 con 0x7ff6e8102b60 2024-04-24T09:18:11.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.465+0000 7ff6ee800700 1 -- 172.21.15.90:0/1012553545 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6e81a3250 con 0x7ff6e8102b60 2024-04-24T09:18:11.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.466+0000 7ff6ee800700 1 -- 172.21.15.90:0/1012553545 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6e80697e0 con 0x7ff6e8102b60 2024-04-24T09:18:11.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.470+0000 7ff6dfe00700 1 -- 172.21.15.90:0/1012553545 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff6d8003b10 con 0x7ff6e8102b60 2024-04-24T09:18:11.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.471+0000 7ff6dfe00700 1 --2- 172.21.15.90:0/1012553545 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7ff6cc0a3c40 0x7ff6cc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:11.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.471+0000 7ff6dfe00700 1 -- 172.21.15.90:0/1012553545 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff6d80d0890 con 0x7ff6e8102b60 2024-04-24T09:18:11.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.471+0000 7ff6e6a00700 1 --2- 172.21.15.90:0/1012553545 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7ff6cc0a3c40 0x7ff6cc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:11.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.472+0000 7ff6e6a00700 1 --2- 172.21.15.90:0/1012553545 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7ff6cc0a3c40 0x7ff6cc0a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4011550 tx=0x7ff6d4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:11.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.476+0000 7ff6dfe00700 1 -- 172.21.15.90:0/1012553545 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff6d8013070 con 0x7ff6e8102b60 2024-04-24T09:18:11.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.709+0000 7ff6ee800700 1 -- 172.21.15.90:0/1012553545 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7ff6e8063700 con 0x7ff6e8102b60 2024-04-24T09:18:11.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.710+0000 7ff6dfe00700 1 -- 172.21.15.90:0/1012553545 <== mon.2 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v82) ==== 74+0+133 (secure 0 0 0) 0x7ff6d809d700 con 0x7ff6e8102b60 2024-04-24T09:18:11.710 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:18:11.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.714+0000 7ff6ee800700 1 -- 172.21.15.90:0/1012553545 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7ff6cc0a3c40 msgr2=0x7ff6cc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:11.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.714+0000 7ff6ee800700 1 --2- 172.21.15.90:0/1012553545 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7ff6cc0a3c40 0x7ff6cc0a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4011550 tx=0x7ff6d4009260 comp rx=0 tx=0).stop 2024-04-24T09:18:11.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.714+0000 7ff6ee800700 1 -- 172.21.15.90:0/1012553545 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff6e8102b60 msgr2=0x7ff6e8076890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:11.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.714+0000 7ff6ee800700 1 --2- 172.21.15.90:0/1012553545 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff6e8102b60 0x7ff6e8076890 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8003790 tx=0x7ff6d8009580 comp rx=0 tx=0).stop 2024-04-24T09:18:11.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.714+0000 7ff6ee800700 1 -- 172.21.15.90:0/1012553545 shutdown_connections 2024-04-24T09:18:11.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.714+0000 7ff6ee800700 1 --2- 172.21.15.90:0/1012553545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff6e8104700 0x7ff6e8075420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:11.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.714+0000 7ff6ee800700 1 --2- 172.21.15.90:0/1012553545 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff6e8102b60 0x7ff6e8076890 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:11.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.714+0000 7ff6ee800700 1 --2- 172.21.15.90:0/1012553545 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7ff6cc0a3c40 0x7ff6cc0a6100 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:11.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.714+0000 7ff6ee800700 1 --2- 172.21.15.90:0/1012553545 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6e8103d60 0x7ff6e8074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:11.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.714+0000 7ff6ee800700 1 -- 172.21.15.90:0/1012553545 >> 172.21.15.90:0/1012553545 conn(0x7ff6e80fe0c0 msgr2=0x7ff6e8107060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:11.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.714+0000 7ff6ee800700 1 -- 172.21.15.90:0/1012553545 shutdown_connections 2024-04-24T09:18:11.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:11.714+0000 7ff6ee800700 1 -- 172.21.15.90:0/1012553545 wait complete. 2024-04-24T09:18:12.653 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":82,"num_osds":12,"num_up_osds":12,"osd_up_since":1713950290,"num_in_osds":12,"osd_in_since":1713950270,"num_remapped_pgs":0} 2024-04-24T09:18:12.653 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-24T09:18:12.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:12 smithi090 ceph-mon[27825]: osdmap e82: 12 total, 12 up, 12 in 2024-04-24T09:18:12.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:12 smithi090 ceph-mon[27825]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:12.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:12 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/1012553545' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-24T09:18:12.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:12 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:12.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:12 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:12.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:12 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:18:12.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:12 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:18:12.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:12 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:18:12.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:12 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:18:12.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:12 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:12.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:12 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:18:12.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:12 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:18:12.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:12 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:12.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:12 smithi149 ceph-mon[32592]: osdmap e82: 12 total, 12 up, 12 in 2024-04-24T09:18:12.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:12 smithi149 ceph-mon[32592]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:12.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:12 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/1012553545' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-24T09:18:12.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:12 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:12.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:12 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:12.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:12 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:18:12.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:12 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:18:12.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:12 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:18:12.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:12 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:18:12.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:12 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:12.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:12 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:18:12.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:12 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:18:12.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:12 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:12.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:12 smithi114 ceph-mon[32501]: osdmap e82: 12 total, 12 up, 12 in 2024-04-24T09:18:12.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:12 smithi114 ceph-mon[32501]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:12.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:12 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/1012553545' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-24T09:18:12.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:12 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:12.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:12 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:12.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:12 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:18:12.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:12 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:18:12.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:12 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:18:12.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:12 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:18:12.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:12 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:12.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:12 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:18:12.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:12 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:18:12.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:12 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:13.069 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:18:13.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:13 smithi149 ceph-mon[32592]: Detected new or changed devices on smithi149 2024-04-24T09:18:13.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:13 smithi149 ceph-mon[32592]: Adjusting osd_memory_target on smithi149 to 1454M 2024-04-24T09:18:13.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:13 smithi090 ceph-mon[27825]: Detected new or changed devices on smithi149 2024-04-24T09:18:13.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:13 smithi090 ceph-mon[27825]: Adjusting osd_memory_target on smithi149 to 1454M 2024-04-24T09:18:13.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:13 smithi114 ceph-mon[32501]: Detected new or changed devices on smithi149 2024-04-24T09:18:13.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:13 smithi114 ceph-mon[32501]: Adjusting osd_memory_target on smithi149 to 1454M 2024-04-24T09:18:14.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:14 smithi090 ceph-mon[27825]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:14.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:14 smithi149 ceph-mon[32592]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:14.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:14 smithi114 ceph-mon[32501]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:15.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.448+0000 7f1230600700 1 -- 172.21.15.90:0/220004329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1228075640 msgr2=0x7f122810efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:15.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.448+0000 7f1230600700 1 --2- 172.21.15.90:0/220004329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1228075640 0x7f122810efb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1214009990 tx=0x7f1214009ca0 comp rx=0 tx=0).stop 2024-04-24T09:18:15.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.448+0000 7f1230600700 1 -- 172.21.15.90:0/220004329 shutdown_connections 2024-04-24T09:18:15.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.448+0000 7f1230600700 1 --2- 172.21.15.90:0/220004329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f122810f4f0 0x7f12281118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:15.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.448+0000 7f1230600700 1 --2- 172.21.15.90:0/220004329 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1228074d20 0x7f1228075100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:15.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.448+0000 7f1230600700 1 --2- 172.21.15.90:0/220004329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1228075640 0x7f122810efb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:15.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.448+0000 7f1230600700 1 -- 172.21.15.90:0/220004329 >> 172.21.15.90:0/220004329 conn(0x7f1228079c50 msgr2=0x7f122807a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:15.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.448+0000 7f1230600700 1 -- 172.21.15.90:0/220004329 shutdown_connections 2024-04-24T09:18:15.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.448+0000 7f1230600700 1 -- 172.21.15.90:0/220004329 wait complete. 2024-04-24T09:18:15.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.449+0000 7f1230600700 1 Processor -- start 2024-04-24T09:18:15.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.449+0000 7f1230600700 1 -- start start 2024-04-24T09:18:15.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.450+0000 7f1230600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1228074d20 0x7f122819b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:15.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.450+0000 7f1230600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1228075640 0x7f122819bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:15.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.450+0000 7f1230600700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f122810f4f0 0x7f122819fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:15.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.450+0000 7f1230600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1228115000 con 0x7f1228075640 2024-04-24T09:18:15.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.450+0000 7f1230600700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1228114e80 con 0x7f122810f4f0 2024-04-24T09:18:15.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.450+0000 7f1230600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1228115180 con 0x7f1228074d20 2024-04-24T09:18:15.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.450+0000 7f122d200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1228075640 0x7f122819bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:15.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.450+0000 7f122dc00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1228074d20 0x7f122819b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:15.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.450+0000 7f122dc00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1228074d20 0x7f122819b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:43380/0 (socket says 172.21.15.90:43380) 2024-04-24T09:18:15.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.450+0000 7f122dc00700 1 -- 172.21.15.90:0/2930703490 learned_addr learned my addr 172.21.15.90:0/2930703490 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:18:15.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.450+0000 7f122d200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1228075640 0x7f122819bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36896/0 (socket says 172.21.15.90:36896) 2024-04-24T09:18:15.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.450+0000 7f122e600700 1 --2- 172.21.15.90:0/2930703490 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f122810f4f0 0x7f122819fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:15.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.451+0000 7f122dc00700 1 -- 172.21.15.90:0/2930703490 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f122810f4f0 msgr2=0x7f122819fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:15.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.451+0000 7f122dc00700 1 --2- 172.21.15.90:0/2930703490 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f122810f4f0 0x7f122819fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:15.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.451+0000 7f122dc00700 1 -- 172.21.15.90:0/2930703490 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1228075640 msgr2=0x7f122819bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:15.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.451+0000 7f122dc00700 1 --2- 172.21.15.90:0/2930703490 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1228075640 0x7f122819bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:15.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.451+0000 7f122dc00700 1 -- 172.21.15.90:0/2930703490 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12281a0530 con 0x7f1228074d20 2024-04-24T09:18:15.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.451+0000 7f122d200700 1 --2- 172.21.15.90:0/2930703490 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1228075640 0x7f122819bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:18:15.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.451+0000 7f122dc00700 1 --2- 172.21.15.90:0/2930703490 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1228074d20 0x7f122819b570 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f121c00e9a0 tx=0x7f121c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:15.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.451+0000 7f1226000700 1 -- 172.21.15.90:0/2930703490 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121c00ca10 con 0x7f1228074d20 2024-04-24T09:18:15.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.451+0000 7f1230600700 1 -- 172.21.15.90:0/2930703490 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12281a0820 con 0x7f1228074d20 2024-04-24T09:18:15.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.451+0000 7f1226000700 1 -- 172.21.15.90:0/2930703490 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f121c00cbb0 con 0x7f1228074d20 2024-04-24T09:18:15.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.451+0000 7f1226000700 1 -- 172.21.15.90:0/2930703490 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121c018680 con 0x7f1228074d20 2024-04-24T09:18:15.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.452+0000 7f1230600700 1 -- 172.21.15.90:0/2930703490 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12281a80a0 con 0x7f1228074d20 2024-04-24T09:18:15.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.452+0000 7f1230600700 1 -- 172.21.15.90:0/2930703490 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12280697e0 con 0x7f1228074d20 2024-04-24T09:18:15.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.455+0000 7f1226000700 1 -- 172.21.15.90:0/2930703490 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f121c010430 con 0x7f1228074d20 2024-04-24T09:18:15.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.456+0000 7f1226000700 1 --2- 172.21.15.90:0/2930703490 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f12100a3c40 0x7f12100a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:15.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.456+0000 7f1226000700 1 -- 172.21.15.90:0/2930703490 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f121c0d0200 con 0x7f1228074d20 2024-04-24T09:18:15.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.456+0000 7f122d200700 1 --2- 172.21.15.90:0/2930703490 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f12100a3c40 0x7f12100a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:15.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.457+0000 7f122d200700 1 --2- 172.21.15.90:0/2930703490 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f12100a3c40 0x7f12100a6100 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f121400b2e0 tx=0x7f1214005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:15.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.460+0000 7f1226000700 1 -- 172.21.15.90:0/2930703490 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f121c09cfe0 con 0x7f1228074d20 2024-04-24T09:18:15.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.693+0000 7f1230600700 1 -- 172.21.15.90:0/2930703490 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f122802dfe0 con 0x7f1228074d20 2024-04-24T09:18:15.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.695+0000 7f1226000700 1 -- 172.21.15.90:0/2930703490 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18468 (secure 0 0 0) 0x7f121c09c8e0 con 0x7f1228074d20 2024-04-24T09:18:15.695 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:18:15.695 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":82,"fsid":"30ee9be2-021a-11ef-bc93-c7b262605968","created":"2024-04-24T09:09:49.496019+0000","modified":"2024-04-24T09:18:11.472103+0000","last_up_change":"2024-04-24T09:18:10.455632+0000","last_in_change":"2024-04-24T09:17:50.600366+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T09:13:45.548774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13a59b9f-b46e-4ec4-90af-bb7a3d5c753b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6803","nonce":2864288066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6805","nonce":2864288066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6809","nonce":2864288066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6807","nonce":2864288066}]},"public_addr":"172.21.15.90:6803/2864288066","cluster_addr":"172.21.15.90:6805/2864288066","heartbeat_back_addr":"172.21.15.90:6809/2864288066","heartbeat_front_addr":"172.21.15.90:6807/2864288066","state":["exists","up"]},{"osd":1,"uuid":"1567ce8f-24e6-42cf-bf64-f6f0c534a9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6801","nonce":344222414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6803","nonce":344222414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6807","nonce":344222414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6805","nonce":344222414}]},"public_addr":"172.21.15.114:6801/344222414","cluster_addr":"172.21.15.114:6803/344222414","heartbeat_back_addr":"172.21.15.114:6807/344222414","heartbeat_front_addr":"172.21.15.114:6805/344222414","state":["exists","up"]},{"osd":2,"uuid":"f861b304-9c7d-4629-8522-37f33a747945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6801","nonce":3257067419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6803","nonce":3257067419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6807","nonce":3257067419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6805","nonce":3257067419}]},"public_addr":"172.21.15.149:6801/3257067419","cluster_addr":"172.21.15.149:6803/3257067419","heartbeat_back_addr":"172.21.15.149:6807/3257067419","heartbeat_front_addr":"172.21.15.149:6805/3257067419","state":["exists","up"]},{"osd":3,"uuid":"9377ab6e-5996-45fb-93a9-475a2e5f17a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6811","nonce":3068524970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6813","nonce":3068524970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6817","nonce":3068524970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6815","nonce":3068524970}]},"public_addr":"172.21.15.90:6811/3068524970","cluster_addr":"172.21.15.90:6813/3068524970","heartbeat_back_addr":"172.21.15.90:6817/3068524970","heartbeat_front_addr":"172.21.15.90:6815/3068524970","state":["exists","up"]},{"osd":4,"uuid":"e2ddd19e-e641-4d61-b342-02b5f12b1c4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6809","nonce":2782015522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6811","nonce":2782015522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6815","nonce":2782015522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6813","nonce":2782015522}]},"public_addr":"172.21.15.114:6809/2782015522","cluster_addr":"172.21.15.114:6811/2782015522","heartbeat_back_addr":"172.21.15.114:6815/2782015522","heartbeat_front_addr":"172.21.15.114:6813/2782015522","state":["exists","up"]},{"osd":5,"uuid":"b6ff9d10-5210-4b46-a192-6dcbf9d1f156","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6809","nonce":4266460479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6811","nonce":4266460479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6815","nonce":4266460479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6813","nonce":4266460479}]},"public_addr":"172.21.15.149:6809/4266460479","cluster_addr":"172.21.15.149:6811/4266460479","heartbeat_back_addr":"172.21.15.149:6815/4266460479","heartbeat_front_addr":"172.21.15.149:6813/4266460479","state":["exists","up"]},{"osd":6,"uuid":"4287b5a1-f953-421c-bf62-5e390bee834d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6819","nonce":1121461276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6821","nonce":1121461276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6825","nonce":1121461276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6823","nonce":1121461276}]},"public_addr":"172.21.15.90:6819/1121461276","cluster_addr":"172.21.15.90:6821/1121461276","heartbeat_back_addr":"172.21.15.90:6825/1121461276","heartbeat_front_addr":"172.21.15.90:6823/1121461276","state":["exists","up"]},{"osd":7,"uuid":"a54e507d-d6ba-4bf2-9a7f-a7b95c3859bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6817","nonce":3222546280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6819","nonce":3222546280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6823","nonce":3222546280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6821","nonce":3222546280}]},"public_addr":"172.21.15.114:6817/3222546280","cluster_addr":"172.21.15.114:6819/3222546280","heartbeat_back_addr":"172.21.15.114:6823/3222546280","heartbeat_front_addr":"172.21.15.114:6821/3222546280","state":["exists","up"]},{"osd":8,"uuid":"9950b010-5865-4109-ae37-582b1dbd2f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6817","nonce":157615149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6819","nonce":157615149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6823","nonce":157615149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6821","nonce":157615149}]},"public_addr":"172.21.15.149:6817/157615149","cluster_addr":"172.21.15.149:6819/157615149","heartbeat_back_addr":"172.21.15.149:6823/157615149","heartbeat_front_addr":"172.21.15.149:6821/157615149","state":["exists","up"]},{"osd":9,"uuid":"31e2635c-e234-421f-9757-592fd673c936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6827","nonce":1307318076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6829","nonce":1307318076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6833","nonce":1307318076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6831","nonce":1307318076}]},"public_addr":"172.21.15.90:6827/1307318076","cluster_addr":"172.21.15.90:6829/1307318076","heartbeat_back_addr":"172.21.15.90:6833/1307318076","heartbeat_front_addr":"172.21.15.90:6831/1307318076","state":["exists","up"]},{"osd":10,"uuid":"46a3b2a6-e6e5-4578-b4bb-f3e998b9bc95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6825","nonce":1719802140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6827","nonce":1719802140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6831","nonce":1719802140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6829","nonce":1719802140}]},"public_addr":"172.21.15.114:6825/1719802140","cluster_addr":"172.21.15.114:6827/1719802140","heartbeat_back_addr":"172.21.15.114:6831/1719802140","heartbeat_front_addr":"172.21.15.114:6829/1719802140","state":["exists","up"]},{"osd":11,"uuid":"ccd4456b-e947-46cf-a779-148ffa289b58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6825","nonce":1737275166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6827","nonce":1737275166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6831","nonce":1737275166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6829","nonce":1737275166}]},"public_addr":"172.21.15.149:6825/1737275166","cluster_addr":"172.21.15.149:6827/1737275166","heartbeat_back_addr":"172.21.15.149:6831/1737275166","heartbeat_front_addr":"172.21.15.149:6829/1737275166","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:12:53.015078+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:13:18.047053+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:13:41.935891+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:14:15.095388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:14:42.512355+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:15:08.113775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:15:43.828550+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:16:10.901909+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:16:36.422962+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:17:13.924446+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:17:42.814401+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:18:08.672793+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/3090852580":"2024-04-25T09:10:57.377655+0000","172.21.15.90:0/687197394":"2024-04-25T09:10:57.377655+0000","172.21.15.90:6801/1809038230":"2024-04-25T09:10:57.377655+0000","172.21.15.90:6800/1809038230":"2024-04-25T09:10:57.377655+0000","172.21.15.90:0/2964149071":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/3617388113":"2024-04-25T09:10:22.599674+0000","172.21.15.90:6801/3089140842":"2024-04-25T09:10:22.599674+0000","172.21.15.90:6800/3089140842":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/2840416149":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/305432710":"2024-04-25T09:10:57.377655+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T09:18:15.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.698+0000 7f1230600700 1 -- 172.21.15.90:0/2930703490 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f12100a3c40 msgr2=0x7f12100a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:15.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.698+0000 7f1230600700 1 --2- 172.21.15.90:0/2930703490 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f12100a3c40 0x7f12100a6100 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f121400b2e0 tx=0x7f1214005960 comp rx=0 tx=0).stop 2024-04-24T09:18:15.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.698+0000 7f1230600700 1 -- 172.21.15.90:0/2930703490 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1228074d20 msgr2=0x7f122819b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:15.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.698+0000 7f1230600700 1 --2- 172.21.15.90:0/2930703490 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1228074d20 0x7f122819b570 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f121c00e9a0 tx=0x7f121c00ecb0 comp rx=0 tx=0).stop 2024-04-24T09:18:15.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.698+0000 7f1230600700 1 -- 172.21.15.90:0/2930703490 shutdown_connections 2024-04-24T09:18:15.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.698+0000 7f1230600700 1 --2- 172.21.15.90:0/2930703490 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f122810f4f0 0x7f122819fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:15.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.698+0000 7f1230600700 1 --2- 172.21.15.90:0/2930703490 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1228074d20 0x7f122819b570 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:15.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.698+0000 7f1230600700 1 --2- 172.21.15.90:0/2930703490 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f12100a3c40 0x7f12100a6100 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:15.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.698+0000 7f1230600700 1 --2- 172.21.15.90:0/2930703490 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1228075640 0x7f122819bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:15.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.698+0000 7f1230600700 1 -- 172.21.15.90:0/2930703490 >> 172.21.15.90:0/2930703490 conn(0x7f1228079c50 msgr2=0x7f12280ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:15.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.698+0000 7f1230600700 1 -- 172.21.15.90:0/2930703490 shutdown_connections 2024-04-24T09:18:15.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:15.699+0000 7f1230600700 1 -- 172.21.15.90:0/2930703490 wait complete. 2024-04-24T09:18:16.628 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T09:13:45.548774+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T09:18:16.629 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-24T09:18:16.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:16 smithi090 ceph-mon[27825]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:16.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:16 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/2930703490' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:18:16.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:16 smithi149 ceph-mon[32592]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:16.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:16 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/2930703490' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:18:16.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:16 smithi114 ceph-mon[32501]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:16.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:16 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/2930703490' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:18:17.067 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:18:18.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:18 smithi090 ceph-mon[27825]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:18.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:18 smithi149 ceph-mon[32592]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:18.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:18 smithi114 ceph-mon[32501]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:19.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.357+0000 7fbc59200700 1 -- 172.21.15.90:0/2568266521 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc5406b090 msgr2=0x7fbc5406b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:19.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.357+0000 7fbc59200700 1 --2- 172.21.15.90:0/2568266521 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc5406b090 0x7fbc5406b4f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c009990 tx=0x7fbc3c009ca0 comp rx=0 tx=0).stop 2024-04-24T09:18:19.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.358+0000 7fbc59200700 1 -- 172.21.15.90:0/2568266521 shutdown_connections 2024-04-24T09:18:19.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.358+0000 7fbc59200700 1 --2- 172.21.15.90:0/2568266521 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc5406ba30 0x7fbc5410f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:19.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.358+0000 7fbc59200700 1 --2- 172.21.15.90:0/2568266521 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc54108b60 0x7fbc54108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:19.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.358+0000 7fbc59200700 1 --2- 172.21.15.90:0/2568266521 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc5406b090 0x7fbc5406b4f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:19.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.358+0000 7fbc59200700 1 -- 172.21.15.90:0/2568266521 >> 172.21.15.90:0/2568266521 conn(0x7fbc54078960 msgr2=0x7fbc54078d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:19.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.358+0000 7fbc59200700 1 -- 172.21.15.90:0/2568266521 shutdown_connections 2024-04-24T09:18:19.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.358+0000 7fbc59200700 1 -- 172.21.15.90:0/2568266521 wait complete. 2024-04-24T09:18:19.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.359+0000 7fbc59200700 1 Processor -- start 2024-04-24T09:18:19.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.359+0000 7fbc59200700 1 -- start start 2024-04-24T09:18:19.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.359+0000 7fbc59200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc5406b090 0x7fbc5419b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:19.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.359+0000 7fbc59200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc5406ba30 0x7fbc5419b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:19.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.359+0000 7fbc59200700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc54108b60 0x7fbc5419fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:19.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.359+0000 7fbc59200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fbc54112e10 con 0x7fbc5406ba30 2024-04-24T09:18:19.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.359+0000 7fbc59200700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fbc54112c90 con 0x7fbc54108b60 2024-04-24T09:18:19.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.359+0000 7fbc59200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbc54112f90 con 0x7fbc5406b090 2024-04-24T09:18:19.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.360+0000 7fbc4be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc5406ba30 0x7fbc5419b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:19.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.360+0000 7fbc52800700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc54108b60 0x7fbc5419fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:19.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.360+0000 7fbc51e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc5406b090 0x7fbc5419b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:19.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.360+0000 7fbc4be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc5406ba30 0x7fbc5419b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42448/0 (socket says 172.21.15.90:42448) 2024-04-24T09:18:19.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.360+0000 7fbc4be00700 1 -- 172.21.15.90:0/3803759468 learned_addr learned my addr 172.21.15.90:0/3803759468 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:18:19.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.360+0000 7fbc52800700 1 -- 172.21.15.90:0/3803759468 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc5406b090 msgr2=0x7fbc5419b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:19.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.360+0000 7fbc52800700 1 --2- 172.21.15.90:0/3803759468 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc5406b090 0x7fbc5419b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:19.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.360+0000 7fbc52800700 1 -- 172.21.15.90:0/3803759468 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc5406ba30 msgr2=0x7fbc5419b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:19.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.360+0000 7fbc52800700 1 --2- 172.21.15.90:0/3803759468 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc5406ba30 0x7fbc5419b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:19.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.360+0000 7fbc52800700 1 -- 172.21.15.90:0/3803759468 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc541a04b0 con 0x7fbc54108b60 2024-04-24T09:18:19.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.360+0000 7fbc4be00700 1 --2- 172.21.15.90:0/3803759468 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc5406ba30 0x7fbc5419b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:18:19.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.361+0000 7fbc52800700 1 --2- 172.21.15.90:0/3803759468 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc54108b60 0x7fbc5419fd70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c00efc0 tx=0x7fbc4c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:19.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.361+0000 7fbc4aa00700 1 -- 172.21.15.90:0/3803759468 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c0092e0 con 0x7fbc54108b60 2024-04-24T09:18:19.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.361+0000 7fbc4aa00700 1 -- 172.21.15.90:0/3803759468 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc4c004bb0 con 0x7fbc54108b60 2024-04-24T09:18:19.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.361+0000 7fbc59200700 1 -- 172.21.15.90:0/3803759468 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc541a0740 con 0x7fbc54108b60 2024-04-24T09:18:19.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.361+0000 7fbc4aa00700 1 -- 172.21.15.90:0/3803759468 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c004280 con 0x7fbc54108b60 2024-04-24T09:18:19.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.361+0000 7fbc59200700 1 -- 172.21.15.90:0/3803759468 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc541a8020 con 0x7fbc54108b60 2024-04-24T09:18:19.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.362+0000 7fbc59200700 1 -- 172.21.15.90:0/3803759468 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc540697e0 con 0x7fbc54108b60 2024-04-24T09:18:19.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.364+0000 7fbc4aa00700 1 -- 172.21.15.90:0/3803759468 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbc4c01b030 con 0x7fbc54108b60 2024-04-24T09:18:19.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.365+0000 7fbc4aa00700 1 --2- 172.21.15.90:0/3803759468 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fbc380a3c40 0x7fbc380a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:19.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.366+0000 7fbc4aa00700 1 -- 172.21.15.90:0/3803759468 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fbc4c0cfaf0 con 0x7fbc54108b60 2024-04-24T09:18:19.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.366+0000 7fbc51e00700 1 --2- 172.21.15.90:0/3803759468 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fbc380a3c40 0x7fbc380a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:19.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.366+0000 7fbc51e00700 1 --2- 172.21.15.90:0/3803759468 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fbc380a3c40 0x7fbc380a6100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbc44009be0 tx=0x7fbc44009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:19.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.370+0000 7fbc4aa00700 1 -- 172.21.15.90:0/3803759468 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbc4c022020 con 0x7fbc54108b60 2024-04-24T09:18:19.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.595+0000 7fbc59200700 1 -- 172.21.15.90:0/3803759468 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbc541a81f0 con 0x7fbc54108b60 2024-04-24T09:18:19.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.595+0000 7fbc4aa00700 1 -- 172.21.15.90:0/3803759468 <== mon.2 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fbc4c010040 con 0x7fbc54108b60 2024-04-24T09:18:19.596 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 1 2024-04-24T09:18:19.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.599+0000 7fbc59200700 1 -- 172.21.15.90:0/3803759468 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fbc380a3c40 msgr2=0x7fbc380a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:19.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.599+0000 7fbc59200700 1 --2- 172.21.15.90:0/3803759468 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fbc380a3c40 0x7fbc380a6100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbc44009be0 tx=0x7fbc44009260 comp rx=0 tx=0).stop 2024-04-24T09:18:19.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.599+0000 7fbc59200700 1 -- 172.21.15.90:0/3803759468 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc54108b60 msgr2=0x7fbc5419fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:19.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.599+0000 7fbc59200700 1 --2- 172.21.15.90:0/3803759468 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc54108b60 0x7fbc5419fd70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c00efc0 tx=0x7fbc4c00c3a0 comp rx=0 tx=0).stop 2024-04-24T09:18:19.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.599+0000 7fbc59200700 1 -- 172.21.15.90:0/3803759468 shutdown_connections 2024-04-24T09:18:19.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.599+0000 7fbc59200700 1 --2- 172.21.15.90:0/3803759468 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc54108b60 0x7fbc5419fd70 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:19.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.599+0000 7fbc59200700 1 --2- 172.21.15.90:0/3803759468 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc5406b090 0x7fbc5419b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:19.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.599+0000 7fbc59200700 1 --2- 172.21.15.90:0/3803759468 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fbc380a3c40 0x7fbc380a6100 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:19.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.599+0000 7fbc59200700 1 --2- 172.21.15.90:0/3803759468 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc5406ba30 0x7fbc5419b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:19.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.599+0000 7fbc59200700 1 -- 172.21.15.90:0/3803759468 >> 172.21.15.90:0/3803759468 conn(0x7fbc54078960 msgr2=0x7fbc540ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:19.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.599+0000 7fbc59200700 1 -- 172.21.15.90:0/3803759468 shutdown_connections 2024-04-24T09:18:19.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:19.599+0000 7fbc59200700 1 -- 172.21.15.90:0/3803759468 wait complete. 2024-04-24T09:18:20.489 INFO:tasks.cephadm:Adding mds.a on smithi090 2024-04-24T09:18:20.489 INFO:tasks.cephadm:Adding mds.d on smithi090 2024-04-24T09:18:20.489 INFO:tasks.cephadm:Adding mds.g on smithi090 2024-04-24T09:18:20.489 INFO:tasks.cephadm:Adding mds.j on smithi090 2024-04-24T09:18:20.489 INFO:tasks.cephadm:Adding mds.b on smithi114 2024-04-24T09:18:20.489 INFO:tasks.cephadm:Adding mds.e on smithi114 2024-04-24T09:18:20.489 INFO:tasks.cephadm:Adding mds.h on smithi114 2024-04-24T09:18:20.489 INFO:tasks.cephadm:Adding mds.k on smithi114 2024-04-24T09:18:20.489 INFO:tasks.cephadm:Adding mds.c on smithi149 2024-04-24T09:18:20.489 INFO:tasks.cephadm:Adding mds.f on smithi149 2024-04-24T09:18:20.490 INFO:tasks.cephadm:Adding mds.i on smithi149 2024-04-24T09:18:20.490 INFO:tasks.cephadm:Adding mds.l on smithi149 2024-04-24T09:18:20.490 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi090=a;smithi090=d;smithi090=g;smithi090=j;smithi114=b;smithi114=e;smithi114=h;smithi114=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l' 2024-04-24T09:18:20.740 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T09:18:20.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:20 smithi090 ceph-mon[27825]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:20.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:20 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/3803759468' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:18:20.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:20 smithi149 ceph-mon[32592]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:20.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:20 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/3803759468' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:18:20.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:20 smithi114 ceph-mon[32501]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:20.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:20 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/3803759468' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:18:22.625 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.623+0000 7f313c28b700 1 -- 172.21.15.149:0/3468589792 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f31341035f0 msgr2=0x7f3134109ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:22.625 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.623+0000 7f313c28b700 1 --2- 172.21.15.149:0/3468589792 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f31341035f0 0x7f3134109ea0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f312c00b0a0 tx=0x7f312c00b3b0 comp rx=0 tx=0).stop 2024-04-24T09:18:22.625 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.623+0000 7f313c28b700 1 -- 172.21.15.149:0/3468589792 shutdown_connections 2024-04-24T09:18:22.625 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.623+0000 7f313c28b700 1 --2- 172.21.15.149:0/3468589792 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f31341035f0 0x7f3134109ea0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:22.625 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.623+0000 7f313c28b700 1 --2- 172.21.15.149:0/3468589792 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3134101a50 0x7f3134101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:22.625 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.623+0000 7f313c28b700 1 --2- 172.21.15.149:0/3468589792 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3134102c50 0x7f31341030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:22.625 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.623+0000 7f313c28b700 1 -- 172.21.15.149:0/3468589792 >> 172.21.15.149:0/3468589792 conn(0x7f31340fcff0 msgr2=0x7f31340ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:22.625 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.623+0000 7f313c28b700 1 -- 172.21.15.149:0/3468589792 shutdown_connections 2024-04-24T09:18:22.625 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.623+0000 7f313c28b700 1 -- 172.21.15.149:0/3468589792 wait complete. 2024-04-24T09:18:22.625 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.623+0000 7f313c28b700 1 Processor -- start 2024-04-24T09:18:22.626 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.623+0000 7f313c28b700 1 -- start start 2024-04-24T09:18:22.626 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.624+0000 7f313c28b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3134101a50 0x7f3134198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:22.626 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.624+0000 7f313c28b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3134102c50 0x7f31341992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:22.626 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.624+0000 7f313c28b700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31341035f0 0x7f31341a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:22.626 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.624+0000 7f313c28b700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f313410d690 con 0x7f3134102c50 2024-04-24T09:18:22.626 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.624+0000 7f313c28b700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f313410d510 con 0x7f31341035f0 2024-04-24T09:18:22.626 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.624+0000 7f313c28b700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f313410d810 con 0x7f3134101a50 2024-04-24T09:18:22.626 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.624+0000 7f313a027700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3134101a50 0x7f3134198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:22.626 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.624+0000 7f313a027700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3134101a50 0x7f3134198d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:39842/0 (socket says 172.21.15.149:39842) 2024-04-24T09:18:22.626 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.624+0000 7f313a027700 1 -- 172.21.15.149:0/1856403017 learned_addr learned my addr 172.21.15.149:0/1856403017 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-04-24T09:18:22.626 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.624+0000 7f313a027700 1 -- 172.21.15.149:0/1856403017 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31341035f0 msgr2=0x7f31341a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:22.626 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.624+0000 7f313a027700 1 --2- 172.21.15.149:0/1856403017 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31341035f0 0x7f31341a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:22.627 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.624+0000 7f313a027700 1 -- 172.21.15.149:0/1856403017 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3134102c50 msgr2=0x7f31341992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:22.627 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.624+0000 7f313a027700 1 --2- 172.21.15.149:0/1856403017 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3134102c50 0x7f31341992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:22.627 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.624+0000 7f313a027700 1 -- 172.21.15.149:0/1856403017 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31341a09c0 con 0x7f3134101a50 2024-04-24T09:18:22.627 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.624+0000 7f313a027700 1 --2- 172.21.15.149:0/1856403017 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3134101a50 0x7f3134198d60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f313000c9c0 tx=0x7f313000cd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:22.627 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.624+0000 7f312b7fe700 1 -- 172.21.15.149:0/1856403017 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3130007ac0 con 0x7f3134101a50 2024-04-24T09:18:22.627 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.624+0000 7f312b7fe700 1 -- 172.21.15.149:0/1856403017 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3130007c60 con 0x7f3134101a50 2024-04-24T09:18:22.627 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.624+0000 7f312b7fe700 1 -- 172.21.15.149:0/1856403017 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3130018780 con 0x7f3134101a50 2024-04-24T09:18:22.627 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.624+0000 7f313c28b700 1 -- 172.21.15.149:0/1856403017 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31341a0cb0 con 0x7f3134101a50 2024-04-24T09:18:22.627 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.624+0000 7f313c28b700 1 -- 172.21.15.149:0/1856403017 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31341a1260 con 0x7f3134101a50 2024-04-24T09:18:22.627 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.625+0000 7f313c28b700 1 -- 172.21.15.149:0/1856403017 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31341a14c0 con 0x7f3134101a50 2024-04-24T09:18:22.628 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.626+0000 7f312b7fe700 1 -- 172.21.15.149:0/1856403017 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f313000f450 con 0x7f3134101a50 2024-04-24T09:18:22.628 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.627+0000 7f312b7fe700 1 --2- 172.21.15.149:0/1856403017 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f31200a3cc0 0x7f31200a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:22.628 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.627+0000 7f312b7fe700 1 -- 172.21.15.149:0/1856403017 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f31300d0050 con 0x7f3134101a50 2024-04-24T09:18:22.628 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.628+0000 7f3139826700 1 --2- 172.21.15.149:0/1856403017 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f31200a3cc0 0x7f31200a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:22.628 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.629+0000 7f3139826700 1 --2- 172.21.15.149:0/1856403017 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f31200a3cc0 0x7f31200a6180 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f31240097b0 tx=0x7f3124006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:22.630 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.631+0000 7f312b7fe700 1 -- 172.21.15.149:0/1856403017 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3130014030 con 0x7f3134101a50 2024-04-24T09:18:22.808 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:22 smithi149 ceph-mon[32592]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:22.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:22 smithi090 ceph-mon[27825]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:22.866 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.866+0000 7f313c28b700 1 -- 172.21.15.149:0/1856403017 --> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi090=a;smithi090=d;smithi090=g;smithi090=j;smithi114=b;smithi114=e;smithi114=h;smithi114=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l", "target": ["mon-mgr", ""]}) -- 0x7f313406afe0 con 0x7f31200a3cc0 2024-04-24T09:18:22.884 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.885+0000 7f312b7fe700 1 -- 172.21.15.149:0/1856403017 <== mgr.14152 v2:172.21.15.90:6800/4261662956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f313406afe0 con 0x7f31200a3cc0 2024-04-24T09:18:22.885 INFO:teuthology.orchestra.run.smithi149.stdout:Scheduled mds.all update... 2024-04-24T09:18:22.887 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.887+0000 7f313c28b700 1 -- 172.21.15.149:0/1856403017 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f31200a3cc0 msgr2=0x7f31200a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:22.887 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.887+0000 7f313c28b700 1 --2- 172.21.15.149:0/1856403017 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f31200a3cc0 0x7f31200a6180 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f31240097b0 tx=0x7f3124006d20 comp rx=0 tx=0).stop 2024-04-24T09:18:22.887 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.887+0000 7f313c28b700 1 -- 172.21.15.149:0/1856403017 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3134101a50 msgr2=0x7f3134198d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:22.887 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.887+0000 7f313c28b700 1 --2- 172.21.15.149:0/1856403017 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3134101a50 0x7f3134198d60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f313000c9c0 tx=0x7f313000cd80 comp rx=0 tx=0).stop 2024-04-24T09:18:22.887 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.887+0000 7f313c28b700 1 -- 172.21.15.149:0/1856403017 shutdown_connections 2024-04-24T09:18:22.887 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.887+0000 7f313c28b700 1 --2- 172.21.15.149:0/1856403017 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31341035f0 0x7f31341a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:22.887 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.887+0000 7f313c28b700 1 --2- 172.21.15.149:0/1856403017 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3134101a50 0x7f3134198d60 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:22.887 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.887+0000 7f313c28b700 1 --2- 172.21.15.149:0/1856403017 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f31200a3cc0 0x7f31200a6180 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:22.887 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.887+0000 7f313c28b700 1 --2- 172.21.15.149:0/1856403017 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3134102c50 0x7f31341992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:22.888 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.887+0000 7f313c28b700 1 -- 172.21.15.149:0/1856403017 >> 172.21.15.149:0/1856403017 conn(0x7f31340fcff0 msgr2=0x7f313406a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:22.888 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.888+0000 7f313c28b700 1 -- 172.21.15.149:0/1856403017 shutdown_connections 2024-04-24T09:18:22.888 INFO:teuthology.orchestra.run.smithi149.stderr:2024-04-24T09:18:22.888+0000 7f313c28b700 1 -- 172.21.15.149:0/1856403017 wait complete. 2024-04-24T09:18:22.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:22 smithi114 ceph-mon[32501]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:23.692 DEBUG:teuthology.orchestra.run.smithi090:mds.a> sudo journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mds.a.service 2024-04-24T09:18:23.700 DEBUG:teuthology.orchestra.run.smithi090:mds.d> sudo journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mds.d.service 2024-04-24T09:18:23.745 DEBUG:teuthology.orchestra.run.smithi090:mds.g> sudo journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mds.g.service 2024-04-24T09:18:23.791 DEBUG:teuthology.orchestra.run.smithi090:mds.j> sudo journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mds.j.service 2024-04-24T09:18:23.836 DEBUG:teuthology.orchestra.run.smithi114:mds.b> sudo journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mds.b.service 2024-04-24T09:18:23.839 DEBUG:teuthology.orchestra.run.smithi114:mds.e> sudo journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mds.e.service 2024-04-24T09:18:23.882 DEBUG:teuthology.orchestra.run.smithi114:mds.h> sudo journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mds.h.service 2024-04-24T09:18:23.925 DEBUG:teuthology.orchestra.run.smithi114:mds.k> sudo journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mds.k.service 2024-04-24T09:18:23.968 DEBUG:teuthology.orchestra.run.smithi149:mds.c> sudo journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mds.c.service 2024-04-24T09:18:23.971 DEBUG:teuthology.orchestra.run.smithi149:mds.f> sudo journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mds.f.service 2024-04-24T09:18:23.991 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:23 smithi149 ceph-mon[32592]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi090=a;smithi090=d;smithi090=g;smithi090=j;smithi114=b;smithi114=e;smithi114=h;smithi114=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:18:23.991 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:23 smithi149 ceph-mon[32592]: Saving service mds.all spec with placement smithi090=a;smithi090=d;smithi090=g;smithi090=j;smithi114=b;smithi114=e;smithi114=h;smithi114=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l;count:12 2024-04-24T09:18:23.991 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:23 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:23.991 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:23 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:18:23.991 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:23 smithi149 ceph-mon[32592]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:23.992 DEBUG:teuthology.orchestra.run.smithi149:mds.i> sudo journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mds.i.service 2024-04-24T09:18:24.035 DEBUG:teuthology.orchestra.run.smithi149:mds.l> sudo journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mds.l.service 2024-04-24T09:18:24.078 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-24T09:18:24.078 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T09:18:24.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:23 smithi090 ceph-mon[27825]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi090=a;smithi090=d;smithi090=g;smithi090=j;smithi114=b;smithi114=e;smithi114=h;smithi114=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:18:24.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:23 smithi090 ceph-mon[27825]: Saving service mds.all spec with placement smithi090=a;smithi090=d;smithi090=g;smithi090=j;smithi114=b;smithi114=e;smithi114=h;smithi114=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l;count:12 2024-04-24T09:18:24.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:23 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:24.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:23 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:18:24.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:23 smithi090 ceph-mon[27825]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:24.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:23 smithi114 ceph-mon[32501]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi090=a;smithi090=d;smithi090=g;smithi090=j;smithi114=b;smithi114=e;smithi114=h;smithi114=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:18:24.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:23 smithi114 ceph-mon[32501]: Saving service mds.all spec with placement smithi090=a;smithi090=d;smithi090=g;smithi090=j;smithi114=b;smithi114=e;smithi114=h;smithi114=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l;count:12 2024-04-24T09:18:24.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:23 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:24.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:23 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:18:24.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:23 smithi114 ceph-mon[32501]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:24.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.302+0000 7fb9a9400640 1 -- 172.21.15.90:0/615847443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9a4109670 msgr2=0x7fb9a4109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:24.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.302+0000 7fb9a9400640 1 --2- 172.21.15.90:0/615847443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9a4109670 0x7fb9a4109af0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb98c009900 tx=0x7fb98c02a400 comp rx=0 tx=0).stop 2024-04-24T09:18:24.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.303+0000 7fb9a9400640 1 -- 172.21.15.90:0/615847443 shutdown_connections 2024-04-24T09:18:24.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.303+0000 7fb9a9400640 1 --2- 172.21.15.90:0/615847443 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb9a410a030 0x7fb9a41108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:24.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.303+0000 7fb9a9400640 1 --2- 172.21.15.90:0/615847443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9a4109670 0x7fb9a4109af0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:24.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.303+0000 7fb9a9400640 1 --2- 172.21.15.90:0/615847443 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9a4108480 0x7fb9a41088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:24.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.303+0000 7fb9a9400640 1 -- 172.21.15.90:0/615847443 >> 172.21.15.90:0/615847443 conn(0x7fb9a4077440 msgr2=0x7fb9a4079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:24.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.303+0000 7fb9a9400640 1 -- 172.21.15.90:0/615847443 shutdown_connections 2024-04-24T09:18:24.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.303+0000 7fb9a9400640 1 -- 172.21.15.90:0/615847443 wait complete. 2024-04-24T09:18:24.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.304+0000 7fb9a9400640 1 Processor -- start 2024-04-24T09:18:24.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.304+0000 7fb9a9400640 1 -- start start 2024-04-24T09:18:24.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.304+0000 7fb9a9400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9a4108480 0x7fb9a419e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:24.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.304+0000 7fb9a9400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb9a4109670 0x7fb9a419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:24.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.304+0000 7fb9a9400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9a410a030 0x7fb9a41a5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:24.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.304+0000 7fb9a9400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a4114160 con 0x7fb9a4109670 2024-04-24T09:18:24.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.304+0000 7fb9a9400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a4113fe0 con 0x7fb9a4108480 2024-04-24T09:18:24.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.304+0000 7fb9a9400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a41142e0 con 0x7fb9a410a030 2024-04-24T09:18:24.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.305+0000 7fb99b400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb9a4109670 0x7fb9a419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:24.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.305+0000 7fb9a3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9a4108480 0x7fb9a419e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:24.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.305+0000 7fb9a8a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9a410a030 0x7fb9a41a5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:24.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.305+0000 7fb9a3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9a4108480 0x7fb9a419e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.90:45562/0 (socket says 172.21.15.90:45562) 2024-04-24T09:18:24.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.305+0000 7fb9a3e00640 1 -- 172.21.15.90:0/2174387002 learned_addr learned my addr 172.21.15.90:0/2174387002 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:18:24.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.305+0000 7fb9a8a00640 1 -- 172.21.15.90:0/2174387002 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9a4108480 msgr2=0x7fb9a419e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:24.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.305+0000 7fb9a8a00640 1 --2- 172.21.15.90:0/2174387002 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9a4108480 0x7fb9a419e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:24.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.305+0000 7fb9a8a00640 1 -- 172.21.15.90:0/2174387002 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb9a4109670 msgr2=0x7fb9a419ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:24.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.305+0000 7fb9a8a00640 1 --2- 172.21.15.90:0/2174387002 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb9a4109670 0x7fb9a419ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:24.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.305+0000 7fb9a8a00640 1 -- 172.21.15.90:0/2174387002 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a41a6370 con 0x7fb9a410a030 2024-04-24T09:18:24.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.305+0000 7fb9a3e00640 1 --2- 172.21.15.90:0/2174387002 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9a4108480 0x7fb9a419e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:18:24.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.305+0000 7fb99b400640 1 --2- 172.21.15.90:0/2174387002 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb9a4109670 0x7fb9a419ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:18:24.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.305+0000 7fb9a8a00640 1 --2- 172.21.15.90:0/2174387002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9a410a030 0x7fb9a41a5d00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb99c009fd0 tx=0x7fb99c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:24.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.306+0000 7fb9a1600640 1 -- 172.21.15.90:0/2174387002 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99c00ce20 con 0x7fb9a410a030 2024-04-24T09:18:24.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.306+0000 7fb9a1600640 1 -- 172.21.15.90:0/2174387002 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb99c005020 con 0x7fb9a410a030 2024-04-24T09:18:24.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.306+0000 7fb9a9400640 1 -- 172.21.15.90:0/2174387002 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9a41a6660 con 0x7fb9a410a030 2024-04-24T09:18:24.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.306+0000 7fb9a1600640 1 -- 172.21.15.90:0/2174387002 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99c010720 con 0x7fb9a410a030 2024-04-24T09:18:24.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.306+0000 7fb9a9400640 1 -- 172.21.15.90:0/2174387002 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9a41a6b30 con 0x7fb9a410a030 2024-04-24T09:18:24.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.307+0000 7fb9a9400640 1 -- 172.21.15.90:0/2174387002 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9a410f7a0 con 0x7fb9a410a030 2024-04-24T09:18:24.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.310+0000 7fb9a1600640 1 -- 172.21.15.90:0/2174387002 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb99c005540 con 0x7fb9a410a030 2024-04-24T09:18:24.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.311+0000 7fb9a1600640 1 --2- 172.21.15.90:0/2174387002 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fb9880a3bb0 0x7fb9880a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:24.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.311+0000 7fb9a1600640 1 -- 172.21.15.90:0/2174387002 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fb99c0d0690 con 0x7fb9a410a030 2024-04-24T09:18:24.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.311+0000 7fb9a3e00640 1 --2- 172.21.15.90:0/2174387002 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fb9880a3bb0 0x7fb9880a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:24.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.312+0000 7fb9a3e00640 1 --2- 172.21.15.90:0/2174387002 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fb9880a3bb0 0x7fb9880a6070 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb9940095f0 tx=0x7fb994005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:24.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.317+0000 7fb9a1600640 1 -- 172.21.15.90:0/2174387002 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb99c09d470 con 0x7fb9a410a030 2024-04-24T09:18:24.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.523+0000 7fb9a9400640 1 -- 172.21.15.90:0/2174387002 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb9a4062c00 con 0x7fb9a410a030 2024-04-24T09:18:24.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.525+0000 7fb9a1600640 1 -- 172.21.15.90:0/2174387002 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18468 (secure 0 0 0) 0x7fb99c09cd70 con 0x7fb9a410a030 2024-04-24T09:18:24.526 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:18:24.526 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":82,"fsid":"30ee9be2-021a-11ef-bc93-c7b262605968","created":"2024-04-24T09:09:49.496019+0000","modified":"2024-04-24T09:18:11.472103+0000","last_up_change":"2024-04-24T09:18:10.455632+0000","last_in_change":"2024-04-24T09:17:50.600366+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T09:13:45.548774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13a59b9f-b46e-4ec4-90af-bb7a3d5c753b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6803","nonce":2864288066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6805","nonce":2864288066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6809","nonce":2864288066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6807","nonce":2864288066}]},"public_addr":"172.21.15.90:6803/2864288066","cluster_addr":"172.21.15.90:6805/2864288066","heartbeat_back_addr":"172.21.15.90:6809/2864288066","heartbeat_front_addr":"172.21.15.90:6807/2864288066","state":["exists","up"]},{"osd":1,"uuid":"1567ce8f-24e6-42cf-bf64-f6f0c534a9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6801","nonce":344222414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6803","nonce":344222414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6807","nonce":344222414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6805","nonce":344222414}]},"public_addr":"172.21.15.114:6801/344222414","cluster_addr":"172.21.15.114:6803/344222414","heartbeat_back_addr":"172.21.15.114:6807/344222414","heartbeat_front_addr":"172.21.15.114:6805/344222414","state":["exists","up"]},{"osd":2,"uuid":"f861b304-9c7d-4629-8522-37f33a747945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6801","nonce":3257067419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6803","nonce":3257067419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6807","nonce":3257067419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6805","nonce":3257067419}]},"public_addr":"172.21.15.149:6801/3257067419","cluster_addr":"172.21.15.149:6803/3257067419","heartbeat_back_addr":"172.21.15.149:6807/3257067419","heartbeat_front_addr":"172.21.15.149:6805/3257067419","state":["exists","up"]},{"osd":3,"uuid":"9377ab6e-5996-45fb-93a9-475a2e5f17a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6811","nonce":3068524970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6813","nonce":3068524970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6817","nonce":3068524970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6815","nonce":3068524970}]},"public_addr":"172.21.15.90:6811/3068524970","cluster_addr":"172.21.15.90:6813/3068524970","heartbeat_back_addr":"172.21.15.90:6817/3068524970","heartbeat_front_addr":"172.21.15.90:6815/3068524970","state":["exists","up"]},{"osd":4,"uuid":"e2ddd19e-e641-4d61-b342-02b5f12b1c4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6809","nonce":2782015522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6811","nonce":2782015522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6815","nonce":2782015522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6813","nonce":2782015522}]},"public_addr":"172.21.15.114:6809/2782015522","cluster_addr":"172.21.15.114:6811/2782015522","heartbeat_back_addr":"172.21.15.114:6815/2782015522","heartbeat_front_addr":"172.21.15.114:6813/2782015522","state":["exists","up"]},{"osd":5,"uuid":"b6ff9d10-5210-4b46-a192-6dcbf9d1f156","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6809","nonce":4266460479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6811","nonce":4266460479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6815","nonce":4266460479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6813","nonce":4266460479}]},"public_addr":"172.21.15.149:6809/4266460479","cluster_addr":"172.21.15.149:6811/4266460479","heartbeat_back_addr":"172.21.15.149:6815/4266460479","heartbeat_front_addr":"172.21.15.149:6813/4266460479","state":["exists","up"]},{"osd":6,"uuid":"4287b5a1-f953-421c-bf62-5e390bee834d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6819","nonce":1121461276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6821","nonce":1121461276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6825","nonce":1121461276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6823","nonce":1121461276}]},"public_addr":"172.21.15.90:6819/1121461276","cluster_addr":"172.21.15.90:6821/1121461276","heartbeat_back_addr":"172.21.15.90:6825/1121461276","heartbeat_front_addr":"172.21.15.90:6823/1121461276","state":["exists","up"]},{"osd":7,"uuid":"a54e507d-d6ba-4bf2-9a7f-a7b95c3859bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6817","nonce":3222546280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6819","nonce":3222546280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6823","nonce":3222546280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6821","nonce":3222546280}]},"public_addr":"172.21.15.114:6817/3222546280","cluster_addr":"172.21.15.114:6819/3222546280","heartbeat_back_addr":"172.21.15.114:6823/3222546280","heartbeat_front_addr":"172.21.15.114:6821/3222546280","state":["exists","up"]},{"osd":8,"uuid":"9950b010-5865-4109-ae37-582b1dbd2f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6817","nonce":157615149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6819","nonce":157615149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6823","nonce":157615149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6821","nonce":157615149}]},"public_addr":"172.21.15.149:6817/157615149","cluster_addr":"172.21.15.149:6819/157615149","heartbeat_back_addr":"172.21.15.149:6823/157615149","heartbeat_front_addr":"172.21.15.149:6821/157615149","state":["exists","up"]},{"osd":9,"uuid":"31e2635c-e234-421f-9757-592fd673c936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6827","nonce":1307318076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6829","nonce":1307318076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6833","nonce":1307318076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6831","nonce":1307318076}]},"public_addr":"172.21.15.90:6827/1307318076","cluster_addr":"172.21.15.90:6829/1307318076","heartbeat_back_addr":"172.21.15.90:6833/1307318076","heartbeat_front_addr":"172.21.15.90:6831/1307318076","state":["exists","up"]},{"osd":10,"uuid":"46a3b2a6-e6e5-4578-b4bb-f3e998b9bc95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6825","nonce":1719802140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6827","nonce":1719802140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6831","nonce":1719802140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6829","nonce":1719802140}]},"public_addr":"172.21.15.114:6825/1719802140","cluster_addr":"172.21.15.114:6827/1719802140","heartbeat_back_addr":"172.21.15.114:6831/1719802140","heartbeat_front_addr":"172.21.15.114:6829/1719802140","state":["exists","up"]},{"osd":11,"uuid":"ccd4456b-e947-46cf-a779-148ffa289b58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6825","nonce":1737275166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6827","nonce":1737275166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6831","nonce":1737275166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6829","nonce":1737275166}]},"public_addr":"172.21.15.149:6825/1737275166","cluster_addr":"172.21.15.149:6827/1737275166","heartbeat_back_addr":"172.21.15.149:6831/1737275166","heartbeat_front_addr":"172.21.15.149:6829/1737275166","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:12:53.015078+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:13:18.047053+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:13:41.935891+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:14:15.095388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:14:42.512355+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:15:08.113775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:15:43.828550+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:16:10.901909+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:16:36.422962+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:17:13.924446+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:17:42.814401+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:18:08.672793+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/3090852580":"2024-04-25T09:10:57.377655+0000","172.21.15.90:0/687197394":"2024-04-25T09:10:57.377655+0000","172.21.15.90:6801/1809038230":"2024-04-25T09:10:57.377655+0000","172.21.15.90:6800/1809038230":"2024-04-25T09:10:57.377655+0000","172.21.15.90:0/2964149071":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/3617388113":"2024-04-25T09:10:22.599674+0000","172.21.15.90:6801/3089140842":"2024-04-25T09:10:22.599674+0000","172.21.15.90:6800/3089140842":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/2840416149":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/305432710":"2024-04-25T09:10:57.377655+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T09:18:24.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.528+0000 7fb9a9400640 1 -- 172.21.15.90:0/2174387002 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fb9880a3bb0 msgr2=0x7fb9880a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:24.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.528+0000 7fb9a9400640 1 --2- 172.21.15.90:0/2174387002 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fb9880a3bb0 0x7fb9880a6070 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb9940095f0 tx=0x7fb994005f90 comp rx=0 tx=0).stop 2024-04-24T09:18:24.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.528+0000 7fb9a9400640 1 -- 172.21.15.90:0/2174387002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9a410a030 msgr2=0x7fb9a41a5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:24.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.528+0000 7fb9a9400640 1 --2- 172.21.15.90:0/2174387002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9a410a030 0x7fb9a41a5d00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb99c009fd0 tx=0x7fb99c00ef10 comp rx=0 tx=0).stop 2024-04-24T09:18:24.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.528+0000 7fb9a9400640 1 -- 172.21.15.90:0/2174387002 shutdown_connections 2024-04-24T09:18:24.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.528+0000 7fb9a9400640 1 --2- 172.21.15.90:0/2174387002 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fb9880a3bb0 0x7fb9880a6070 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:24.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.528+0000 7fb9a9400640 1 --2- 172.21.15.90:0/2174387002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9a410a030 0x7fb9a41a5d00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:24.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.528+0000 7fb9a9400640 1 --2- 172.21.15.90:0/2174387002 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb9a4109670 0x7fb9a419ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:24.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.528+0000 7fb9a9400640 1 --2- 172.21.15.90:0/2174387002 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9a4108480 0x7fb9a419e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:24.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.528+0000 7fb9a9400640 1 -- 172.21.15.90:0/2174387002 >> 172.21.15.90:0/2174387002 conn(0x7fb9a4077440 msgr2=0x7fb9a410c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:24.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.528+0000 7fb9a9400640 1 -- 172.21.15.90:0/2174387002 shutdown_connections 2024-04-24T09:18:24.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.529+0000 7fb9a9400640 1 -- 172.21.15.90:0/2174387002 wait complete. 2024-04-24T09:18:24.549 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T09:13:45.548774+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T09:18:24.549 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T09:18:24.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.803+0000 7efc17e00640 1 -- 172.21.15.90:0/3723077770 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc181052c0 msgr2=0x7efc181076b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:24.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.803+0000 7efc17e00640 1 --2- 172.21.15.90:0/3723077770 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc181052c0 0x7efc181076b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7efc04009900 tx=0x7efc0402a410 comp rx=0 tx=0).stop 2024-04-24T09:18:24.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.803+0000 7efc17e00640 1 -- 172.21.15.90:0/3723077770 shutdown_connections 2024-04-24T09:18:24.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.803+0000 7efc17e00640 1 --2- 172.21.15.90:0/3723077770 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc18107bf0 0x7efc1810a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:24.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.803+0000 7efc17e00640 1 --2- 172.21.15.90:0/3723077770 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc181052c0 0x7efc181076b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:24.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.803+0000 7efc17e00640 1 --2- 172.21.15.90:0/3723077770 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc18102990 0x7efc18104d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:24.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.803+0000 7efc17e00640 1 -- 172.21.15.90:0/3723077770 >> 172.21.15.90:0/3723077770 conn(0x7efc180fc3a0 msgr2=0x7efc180fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:24.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.803+0000 7efc17e00640 1 -- 172.21.15.90:0/3723077770 shutdown_connections 2024-04-24T09:18:24.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.803+0000 7efc17e00640 1 -- 172.21.15.90:0/3723077770 wait complete. 2024-04-24T09:18:24.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.804+0000 7efc17e00640 1 Processor -- start 2024-04-24T09:18:24.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.804+0000 7efc17e00640 1 -- start start 2024-04-24T09:18:24.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.805+0000 7efc17e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc18102990 0x7efc18102300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:24.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.805+0000 7efc17e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc181052c0 0x7efc18100940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:24.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.805+0000 7efc17e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc18107bf0 0x7efc18100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:24.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.805+0000 7efc17e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efc1810db90 con 0x7efc181052c0 2024-04-24T09:18:24.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.805+0000 7efc17e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7efc1810da10 con 0x7efc18107bf0 2024-04-24T09:18:24.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.805+0000 7efc17e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7efc1810dd10 con 0x7efc18102990 2024-04-24T09:18:24.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.805+0000 7efc16a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc18102990 0x7efc18102300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:24.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.805+0000 7efc16000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc181052c0 0x7efc18100940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:24.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.805+0000 7efc16a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc18102990 0x7efc18102300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:51856/0 (socket says 172.21.15.90:51856) 2024-04-24T09:18:24.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.805+0000 7efc16a00640 1 -- 172.21.15.90:0/863208093 learned_addr learned my addr 172.21.15.90:0/863208093 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:18:24.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.805+0000 7efc17400640 1 --2- 172.21.15.90:0/863208093 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc18107bf0 0x7efc18100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:24.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.806+0000 7efc16a00640 1 -- 172.21.15.90:0/863208093 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc18107bf0 msgr2=0x7efc18100e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:24.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.806+0000 7efc16a00640 1 --2- 172.21.15.90:0/863208093 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc18107bf0 0x7efc18100e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:24.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.806+0000 7efc16a00640 1 -- 172.21.15.90:0/863208093 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc181052c0 msgr2=0x7efc18100940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:24.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.806+0000 7efc16a00640 1 --2- 172.21.15.90:0/863208093 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc181052c0 0x7efc18100940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:24.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.806+0000 7efc16a00640 1 -- 172.21.15.90:0/863208093 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc18101710 con 0x7efc18102990 2024-04-24T09:18:24.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.806+0000 7efc17400640 1 --2- 172.21.15.90:0/863208093 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc18107bf0 0x7efc18100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:18:24.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.806+0000 7efc16a00640 1 --2- 172.21.15.90:0/863208093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc18102990 0x7efc18102300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7efc0800e9f0 tx=0x7efc0800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:24.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.806+0000 7efc0f400640 1 -- 172.21.15.90:0/863208093 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc0800cd80 con 0x7efc18102990 2024-04-24T09:18:24.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.806+0000 7efc0f400640 1 -- 172.21.15.90:0/863208093 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc08005500 con 0x7efc18102990 2024-04-24T09:18:24.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.806+0000 7efc17e00640 1 -- 172.21.15.90:0/863208093 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc181a2380 con 0x7efc18102990 2024-04-24T09:18:24.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.806+0000 7efc0f400640 1 -- 172.21.15.90:0/863208093 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc08010690 con 0x7efc18102990 2024-04-24T09:18:24.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.807+0000 7efc17e00640 1 -- 172.21.15.90:0/863208093 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc181a28a0 con 0x7efc18102990 2024-04-24T09:18:24.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.808+0000 7efc17e00640 1 -- 172.21.15.90:0/863208093 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc18068d40 con 0x7efc18102990 2024-04-24T09:18:24.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.809+0000 7efc0f400640 1 -- 172.21.15.90:0/863208093 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7efc080046c0 con 0x7efc18102990 2024-04-24T09:18:24.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.811+0000 7efc0f400640 1 --2- 172.21.15.90:0/863208093 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7efbfc0a3c30 0x7efbfc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:24.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.811+0000 7efc0f400640 1 -- 172.21.15.90:0/863208093 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7efc080d0460 con 0x7efc18102990 2024-04-24T09:18:24.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.811+0000 7efc16000640 1 --2- 172.21.15.90:0/863208093 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7efbfc0a3c30 0x7efbfc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:24.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.812+0000 7efc16000640 1 --2- 172.21.15.90:0/863208093 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7efbfc0a3c30 0x7efbfc0a60f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7efc04005ab0 tx=0x7efc04000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:24.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:24.815+0000 7efc0f400640 1 -- 172.21.15.90:0/863208093 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efc0809d240 con 0x7efc18102990 2024-04-24T09:18:25.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.019+0000 7efc17e00640 1 -- 172.21.15.90:0/863208093 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efc1802d9b0 con 0x7efc18102990 2024-04-24T09:18:25.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.020+0000 7efc0f400640 1 -- 172.21.15.90:0/863208093 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7efc0809cb40 con 0x7efc18102990 2024-04-24T09:18:25.020 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 1 2024-04-24T09:18:25.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.022+0000 7efc17e00640 1 -- 172.21.15.90:0/863208093 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7efbfc0a3c30 msgr2=0x7efbfc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:25.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.022+0000 7efc17e00640 1 --2- 172.21.15.90:0/863208093 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7efbfc0a3c30 0x7efbfc0a60f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7efc04005ab0 tx=0x7efc04000bc0 comp rx=0 tx=0).stop 2024-04-24T09:18:25.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.022+0000 7efc17e00640 1 -- 172.21.15.90:0/863208093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc18102990 msgr2=0x7efc18102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:25.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.022+0000 7efc17e00640 1 --2- 172.21.15.90:0/863208093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc18102990 0x7efc18102300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7efc0800e9f0 tx=0x7efc0800edf0 comp rx=0 tx=0).stop 2024-04-24T09:18:25.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.023+0000 7efc17e00640 1 -- 172.21.15.90:0/863208093 shutdown_connections 2024-04-24T09:18:25.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.023+0000 7efc17e00640 1 --2- 172.21.15.90:0/863208093 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7efbfc0a3c30 0x7efbfc0a60f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:25.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.023+0000 7efc17e00640 1 --2- 172.21.15.90:0/863208093 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc18107bf0 0x7efc18100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:25.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.023+0000 7efc17e00640 1 --2- 172.21.15.90:0/863208093 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc181052c0 0x7efc18100940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:25.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.023+0000 7efc17e00640 1 --2- 172.21.15.90:0/863208093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc18102990 0x7efc18102300 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:25.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.023+0000 7efc17e00640 1 -- 172.21.15.90:0/863208093 >> 172.21.15.90:0/863208093 conn(0x7efc180fc3a0 msgr2=0x7efc181032f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:25.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.023+0000 7efc17e00640 1 -- 172.21.15.90:0/863208093 shutdown_connections 2024-04-24T09:18:25.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.023+0000 7efc17e00640 1 -- 172.21.15.90:0/863208093 wait complete. 2024-04-24T09:18:25.043 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:18:25.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:24 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:18:25.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:24 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:18:25.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:24 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:25.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:24 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:18:25.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:24 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:18:25.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:24 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:18:25.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:24 smithi114 ceph-mon[32501]: Deploying daemon mds.l on smithi149 2024-04-24T09:18:25.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:24 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/2174387002' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:18:25.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.241+0000 7fe03a000640 1 -- 172.21.15.90:0/2723578353 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe034101ea0 msgr2=0x7fe0341022c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:25.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.241+0000 7fe03a000640 1 --2- 172.21.15.90:0/2723578353 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe034101ea0 0x7fe0341022c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe01c009900 tx=0x7fe01c02a410 comp rx=0 tx=0).stop 2024-04-24T09:18:25.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.241+0000 7fe03a000640 1 -- 172.21.15.90:0/2723578353 shutdown_connections 2024-04-24T09:18:25.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.241+0000 7fe03a000640 1 --2- 172.21.15.90:0/2723578353 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0341039b0 0x7fe03410a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:25.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.241+0000 7fe03a000640 1 --2- 172.21.15.90:0/2723578353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe034102ff0 0x7fe034103470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:25.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.241+0000 7fe03a000640 1 --2- 172.21.15.90:0/2723578353 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe034101ea0 0x7fe0341022c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:25.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.241+0000 7fe03a000640 1 -- 172.21.15.90:0/2723578353 >> 172.21.15.90:0/2723578353 conn(0x7fe0340fd410 msgr2=0x7fe0340ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:25.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.241+0000 7fe03a000640 1 -- 172.21.15.90:0/2723578353 shutdown_connections 2024-04-24T09:18:25.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.241+0000 7fe03a000640 1 -- 172.21.15.90:0/2723578353 wait complete. 2024-04-24T09:18:25.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.242+0000 7fe03a000640 1 Processor -- start 2024-04-24T09:18:25.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.242+0000 7fe03a000640 1 -- start start 2024-04-24T09:18:25.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.243+0000 7fe03a000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe034101ea0 0x7fe034075db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:25.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.243+0000 7fe03a000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe034102ff0 0x7fe0340743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:25.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.243+0000 7fe03a000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0341039b0 0x7fe034074930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:25.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.243+0000 7fe03a000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe03410db90 con 0x7fe0341039b0 2024-04-24T09:18:25.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.243+0000 7fe03a000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe03410da10 con 0x7fe034101ea0 2024-04-24T09:18:25.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.243+0000 7fe03a000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe03410dd10 con 0x7fe034102ff0 2024-04-24T09:18:25.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.243+0000 7fe033e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe034102ff0 0x7fe0340743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:25.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.243+0000 7fe038c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe034101ea0 0x7fe034075db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:25.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.243+0000 7fe033e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe034102ff0 0x7fe0340743f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:51872/0 (socket says 172.21.15.90:51872) 2024-04-24T09:18:25.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.243+0000 7fe033e00640 1 -- 172.21.15.90:0/1353611347 learned_addr learned my addr 172.21.15.90:0/1353611347 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:18:25.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.243+0000 7fe039600640 1 --2- 172.21.15.90:0/1353611347 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0341039b0 0x7fe034074930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:25.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.243+0000 7fe033e00640 1 -- 172.21.15.90:0/1353611347 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe034101ea0 msgr2=0x7fe034075db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:25.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.243+0000 7fe033e00640 1 --2- 172.21.15.90:0/1353611347 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe034101ea0 0x7fe034075db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:25.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.243+0000 7fe033e00640 1 -- 172.21.15.90:0/1353611347 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0341039b0 msgr2=0x7fe034074930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:25.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.243+0000 7fe033e00640 1 --2- 172.21.15.90:0/1353611347 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0341039b0 0x7fe034074930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:25.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.243+0000 7fe033e00640 1 -- 172.21.15.90:0/1353611347 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0340751c0 con 0x7fe034102ff0 2024-04-24T09:18:25.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.244+0000 7fe033e00640 1 --2- 172.21.15.90:0/1353611347 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe034102ff0 0x7fe0340743f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe02400eb70 tx=0x7fe02400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:25.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.244+0000 7fe031600640 1 -- 172.21.15.90:0/1353611347 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe024003b70 con 0x7fe034102ff0 2024-04-24T09:18:25.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.244+0000 7fe031600640 1 -- 172.21.15.90:0/1353611347 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe02400ee70 con 0x7fe034102ff0 2024-04-24T09:18:25.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.244+0000 7fe03a000640 1 -- 172.21.15.90:0/1353611347 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0341a22f0 con 0x7fe034102ff0 2024-04-24T09:18:25.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.244+0000 7fe031600640 1 -- 172.21.15.90:0/1353611347 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe024010830 con 0x7fe034102ff0 2024-04-24T09:18:25.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.244+0000 7fe03a000640 1 -- 172.21.15.90:0/1353611347 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0341a2840 con 0x7fe034102ff0 2024-04-24T09:18:25.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.245+0000 7fe03a000640 1 -- 172.21.15.90:0/1353611347 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0341091d0 con 0x7fe034102ff0 2024-04-24T09:18:25.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.247+0000 7fe031600640 1 -- 172.21.15.90:0/1353611347 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe024004020 con 0x7fe034102ff0 2024-04-24T09:18:25.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.248+0000 7fe031600640 1 --2- 172.21.15.90:0/1353611347 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fe0180a3c30 0x7fe0180a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:25.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.249+0000 7fe031600640 1 -- 172.21.15.90:0/1353611347 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fe0240d0630 con 0x7fe034102ff0 2024-04-24T09:18:25.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.249+0000 7fe038c00640 1 --2- 172.21.15.90:0/1353611347 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fe0180a3c30 0x7fe0180a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:25.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.249+0000 7fe038c00640 1 --2- 172.21.15.90:0/1353611347 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fe0180a3c30 0x7fe0180a60f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe01c031440 tx=0x7fe01c0315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:25.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.253+0000 7fe031600640 1 -- 172.21.15.90:0/1353611347 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe02409d410 con 0x7fe034102ff0 2024-04-24T09:18:25.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:24 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:18:25.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:24 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:18:25.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:24 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:25.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:24 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:18:25.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:24 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:18:25.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:24 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:18:25.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:24 smithi090 ceph-mon[27825]: Deploying daemon mds.l on smithi149 2024-04-24T09:18:25.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:24 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/2174387002' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:18:25.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:24 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:18:25.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:24 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:18:25.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:24 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:25.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:24 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:18:25.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:24 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:18:25.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:24 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:18:25.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:24 smithi149 ceph-mon[32592]: Deploying daemon mds.l on smithi149 2024-04-24T09:18:25.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:24 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/2174387002' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:18:25.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.496+0000 7fe03a000640 1 -- 172.21.15.90:0/1353611347 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe03402d620 con 0x7fe034102ff0 2024-04-24T09:18:25.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.497+0000 7fe031600640 1 -- 172.21.15.90:0/1353611347 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7fe02409cd10 con 0x7fe034102ff0 2024-04-24T09:18:25.497 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:18:25.497 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-24T09:18:25.497 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 1 2024-04-24T09:18:25.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.500+0000 7fe03a000640 1 -- 172.21.15.90:0/1353611347 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fe0180a3c30 msgr2=0x7fe0180a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:25.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.500+0000 7fe03a000640 1 --2- 172.21.15.90:0/1353611347 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fe0180a3c30 0x7fe0180a60f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe01c031440 tx=0x7fe01c0315b0 comp rx=0 tx=0).stop 2024-04-24T09:18:25.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.500+0000 7fe03a000640 1 -- 172.21.15.90:0/1353611347 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe034102ff0 msgr2=0x7fe0340743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:25.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.500+0000 7fe03a000640 1 --2- 172.21.15.90:0/1353611347 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe034102ff0 0x7fe0340743f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe02400eb70 tx=0x7fe02400c6a0 comp rx=0 tx=0).stop 2024-04-24T09:18:25.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.500+0000 7fe03a000640 1 -- 172.21.15.90:0/1353611347 shutdown_connections 2024-04-24T09:18:25.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.500+0000 7fe03a000640 1 --2- 172.21.15.90:0/1353611347 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fe0180a3c30 0x7fe0180a60f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:25.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.500+0000 7fe03a000640 1 --2- 172.21.15.90:0/1353611347 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0341039b0 0x7fe034074930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:25.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.500+0000 7fe03a000640 1 --2- 172.21.15.90:0/1353611347 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe034102ff0 0x7fe0340743f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:25.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.500+0000 7fe03a000640 1 --2- 172.21.15.90:0/1353611347 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe034101ea0 0x7fe034075db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:25.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.500+0000 7fe03a000640 1 -- 172.21.15.90:0/1353611347 >> 172.21.15.90:0/1353611347 conn(0x7fe0340fd410 msgr2=0x7fe0341063a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:25.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.500+0000 7fe03a000640 1 -- 172.21.15.90:0/1353611347 shutdown_connections 2024-04-24T09:18:25.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:25.500+0000 7fe03a000640 1 -- 172.21.15.90:0/1353611347 wait complete. 2024-04-24T09:18:26.058 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:25 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/863208093' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:18:26.058 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:25 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/1353611347' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:18:26.058 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:25 smithi149 ceph-mon[32592]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:26.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:25 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/863208093' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:18:26.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:25 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/1353611347' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:18:26.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:25 smithi114 ceph-mon[32501]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:26.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:25 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/863208093' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:18:26.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:25 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/1353611347' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:18:26.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:25 smithi090 ceph-mon[27825]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:26.361 INFO:journalctl@ceph.mds.l.smithi149.stdout:Apr 24 09:18:26 smithi149 systemd[1]: Starting Ceph mds.l for 30ee9be2-021a-11ef-bc93-c7b262605968... 2024-04-24T09:18:26.744 INFO:journalctl@ceph.mds.l.smithi149.stdout:Apr 24 09:18:26 smithi149 podman[63401]: 2024-04-24 09:18:26.363268472 +0000 UTC m=+0.018720571 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:18:26.744 INFO:journalctl@ceph.mds.l.smithi149.stdout:Apr 24 09:18:26 smithi149 podman[63401]: 2024-04-24 09:18:26.491749089 +0000 UTC m=+0.147201170 container create 454120a5fc359ba6d9b8fb1954f7b994e76bfd7d1eba6c109c514261cef3838b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-l, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424) 2024-04-24T09:18:26.744 INFO:journalctl@ceph.mds.l.smithi149.stdout:Apr 24 09:18:26 smithi149 podman[63401]: 2024-04-24 09:18:26.744756307 +0000 UTC m=+0.400208396 container init 454120a5fc359ba6d9b8fb1954f7b994e76bfd7d1eba6c109c514261cef3838b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, ceph=True) 2024-04-24T09:18:27.111 INFO:journalctl@ceph.mds.l.smithi149.stdout:Apr 24 09:18:26 smithi149 podman[63401]: 2024-04-24 09:18:26.747521814 +0000 UTC m=+0.402973889 container start 454120a5fc359ba6d9b8fb1954f7b994e76bfd7d1eba6c109c514261cef3838b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-24T09:18:27.112 INFO:journalctl@ceph.mds.l.smithi149.stdout:Apr 24 09:18:26 smithi149 ceph-mds[63420]: -- 172.21.15.149:0/3974336336 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b949f0ba40 con 0x55b949edec00 2024-04-24T09:18:27.112 INFO:journalctl@ceph.mds.l.smithi149.stdout:Apr 24 09:18:26 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-l[63416]: starting mds.l at 2024-04-24T09:18:27.112 INFO:journalctl@ceph.mds.l.smithi149.stdout:Apr 24 09:18:26 smithi149 bash[63401]: 454120a5fc359ba6d9b8fb1954f7b994e76bfd7d1eba6c109c514261cef3838b 2024-04-24T09:18:27.112 INFO:journalctl@ceph.mds.l.smithi149.stdout:Apr 24 09:18:26 smithi149 systemd[1]: Started Ceph mds.l for 30ee9be2-021a-11ef-bc93-c7b262605968. 2024-04-24T09:18:27.522 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:18:27.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.735+0000 7f5dac400640 1 -- 172.21.15.90:0/1576791077 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5da41008c0 msgr2=0x7f5da4100ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:27.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.735+0000 7f5dac400640 1 --2- 172.21.15.90:0/1576791077 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5da41008c0 0x7f5da4100ce0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c009ca0 tx=0x7f5d9c02a8c0 comp rx=0 tx=0).stop 2024-04-24T09:18:27.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.735+0000 7f5dac400640 1 -- 172.21.15.90:0/1576791077 shutdown_connections 2024-04-24T09:18:27.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.735+0000 7f5dac400640 1 --2- 172.21.15.90:0/1576791077 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5da4101bc0 0x7f5da4102040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:27.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.735+0000 7f5dac400640 1 --2- 172.21.15.90:0/1576791077 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5da4101220 0x7f5da4101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:27.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.735+0000 7f5dac400640 1 --2- 172.21.15.90:0/1576791077 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5da41008c0 0x7f5da4100ce0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:27.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.735+0000 7f5dac400640 1 -- 172.21.15.90:0/1576791077 >> 172.21.15.90:0/1576791077 conn(0x7f5da40fc4e0 msgr2=0x7f5da40fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:27.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.735+0000 7f5dac400640 1 -- 172.21.15.90:0/1576791077 shutdown_connections 2024-04-24T09:18:27.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.735+0000 7f5dac400640 1 -- 172.21.15.90:0/1576791077 wait complete. 2024-04-24T09:18:27.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.736+0000 7f5dac400640 1 Processor -- start 2024-04-24T09:18:27.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.736+0000 7f5dac400640 1 -- start start 2024-04-24T09:18:27.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.737+0000 7f5dac400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5da41008c0 0x7f5da419a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:27.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.737+0000 7f5dac400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5da4101220 0x7f5da419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:27.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.737+0000 7f5dac400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5da4101bc0 0x7f5da41a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:27.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.737+0000 7f5dac400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5da410ff40 con 0x7f5da4101220 2024-04-24T09:18:27.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.737+0000 7f5dac400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5da410fdc0 con 0x7f5da4101bc0 2024-04-24T09:18:27.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.737+0000 7f5dac400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5da41100c0 con 0x7f5da41008c0 2024-04-24T09:18:27.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.737+0000 7f5da9000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5da4101220 0x7f5da419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:27.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.737+0000 7f5da9a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5da41008c0 0x7f5da419a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:27.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.737+0000 7f5daa400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5da4101bc0 0x7f5da41a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:27.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.737+0000 7f5da9a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5da41008c0 0x7f5da419a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:56782/0 (socket says 172.21.15.90:56782) 2024-04-24T09:18:27.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.737+0000 7f5da9000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5da4101220 0x7f5da419aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45464/0 (socket says 172.21.15.90:45464) 2024-04-24T09:18:27.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.737+0000 7f5da9a00640 1 -- 172.21.15.90:0/1592953184 learned_addr learned my addr 172.21.15.90:0/1592953184 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:18:27.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.737+0000 7f5da9a00640 1 -- 172.21.15.90:0/1592953184 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5da4101bc0 msgr2=0x7f5da41a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:27.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.737+0000 7f5da9a00640 1 --2- 172.21.15.90:0/1592953184 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5da4101bc0 0x7f5da41a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:27.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.737+0000 7f5da9a00640 1 -- 172.21.15.90:0/1592953184 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5da4101220 msgr2=0x7f5da419aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:27.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.737+0000 7f5da9a00640 1 --2- 172.21.15.90:0/1592953184 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5da4101220 0x7f5da419aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:27.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.738+0000 7f5da9a00640 1 -- 172.21.15.90:0/1592953184 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da41a2160 con 0x7f5da41008c0 2024-04-24T09:18:27.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.738+0000 7f5da9a00640 1 --2- 172.21.15.90:0/1592953184 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5da41008c0 0x7f5da419a540 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c000f80 tx=0x7f5d9c0319f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:27.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.738+0000 7f5da2000640 1 -- 172.21.15.90:0/1592953184 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c003b70 con 0x7f5da41008c0 2024-04-24T09:18:27.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.738+0000 7f5dac400640 1 -- 172.21.15.90:0/1592953184 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5da41a23f0 con 0x7f5da41008c0 2024-04-24T09:18:27.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.738+0000 7f5da2000640 1 -- 172.21.15.90:0/1592953184 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d9c003d10 con 0x7f5da41008c0 2024-04-24T09:18:27.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.738+0000 7f5da2000640 1 -- 172.21.15.90:0/1592953184 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c02fc00 con 0x7f5da41008c0 2024-04-24T09:18:27.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.738+0000 7f5dac400640 1 -- 172.21.15.90:0/1592953184 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5da41a2990 con 0x7f5da41008c0 2024-04-24T09:18:27.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.739+0000 7f5dac400640 1 -- 172.21.15.90:0/1592953184 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5da410b770 con 0x7f5da41008c0 2024-04-24T09:18:27.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.741+0000 7f5da2000640 1 -- 172.21.15.90:0/1592953184 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5d9c004040 con 0x7f5da41008c0 2024-04-24T09:18:27.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.742+0000 7f5da2000640 1 --2- 172.21.15.90:0/1592953184 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f5d880a3bb0 0x7f5d880a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:27.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.743+0000 7f5da2000640 1 -- 172.21.15.90:0/1592953184 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f5d9c0f14c0 con 0x7f5da41008c0 2024-04-24T09:18:27.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.743+0000 7f5da9000640 1 --2- 172.21.15.90:0/1592953184 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f5d880a3bb0 0x7f5d880a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:27.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.743+0000 7f5da9000640 1 --2- 172.21.15.90:0/1592953184 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f5d880a3bb0 0x7f5d880a6070 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5d900095f0 tx=0x7f5d90005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:27.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.747+0000 7f5da2000640 1 -- 172.21.15.90:0/1592953184 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5d9c0be2a0 con 0x7f5da41008c0 2024-04-24T09:18:27.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:27 smithi090 ceph-mon[27825]: mds.? [v2:172.21.15.149:6832/779642417,v1:172.21.15.149:6833/779642417] up:boot 2024-04-24T09:18:27.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:27 smithi090 ceph-mon[27825]: fsmap 1 up:standby 2024-04-24T09:18:27.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:27 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T09:18:27.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:27 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T09:18:27.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:27 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:27.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:27 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:27.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:27 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:27.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:27 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:18:27.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:27 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:18:27.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:27 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:18:27.854 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:27 smithi090 ceph-mon[27825]: Deploying daemon mds.k on smithi114 2024-04-24T09:18:27.854 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:27 smithi090 ceph-mon[27825]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:27.855 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:27 smithi114 ceph-mon[32501]: mds.? [v2:172.21.15.149:6832/779642417,v1:172.21.15.149:6833/779642417] up:boot 2024-04-24T09:18:27.855 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:27 smithi114 ceph-mon[32501]: fsmap 1 up:standby 2024-04-24T09:18:27.855 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:27 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T09:18:27.855 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:27 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T09:18:27.855 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:27 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:27.855 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:27 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:27.855 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:27 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:27.855 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:27 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:18:27.855 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:27 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:18:27.855 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:27 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:18:27.855 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:27 smithi114 ceph-mon[32501]: Deploying daemon mds.k on smithi114 2024-04-24T09:18:27.855 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:27 smithi114 ceph-mon[32501]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:27.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.989+0000 7f5dac400640 1 -- 172.21.15.90:0/1592953184 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5da402d620 con 0x7f5da41008c0 2024-04-24T09:18:27.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.990+0000 7f5da2000640 1 -- 172.21.15.90:0/1592953184 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1393 (secure 0 0 0) 0x7f5d9c0bdba0 con 0x7f5da41008c0 2024-04-24T09:18:27.990 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:18:27.990 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/779642417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":779642417},{"type":"v1","addr":"172.21.15.149:6833","nonce":779642417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-24T09:18:27.990 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 2 2024-04-24T09:18:27.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.993+0000 7f5dac400640 1 -- 172.21.15.90:0/1592953184 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f5d880a3bb0 msgr2=0x7f5d880a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:27.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.993+0000 7f5dac400640 1 --2- 172.21.15.90:0/1592953184 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f5d880a3bb0 0x7f5d880a6070 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5d900095f0 tx=0x7f5d90005f90 comp rx=0 tx=0).stop 2024-04-24T09:18:27.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.993+0000 7f5dac400640 1 -- 172.21.15.90:0/1592953184 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5da41008c0 msgr2=0x7f5da419a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:27.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.993+0000 7f5dac400640 1 --2- 172.21.15.90:0/1592953184 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5da41008c0 0x7f5da419a540 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c000f80 tx=0x7f5d9c0319f0 comp rx=0 tx=0).stop 2024-04-24T09:18:27.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.993+0000 7f5dac400640 1 -- 172.21.15.90:0/1592953184 shutdown_connections 2024-04-24T09:18:27.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.993+0000 7f5dac400640 1 --2- 172.21.15.90:0/1592953184 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f5d880a3bb0 0x7f5d880a6070 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:27.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.993+0000 7f5dac400640 1 --2- 172.21.15.90:0/1592953184 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5da4101bc0 0x7f5da41a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:27.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.993+0000 7f5dac400640 1 --2- 172.21.15.90:0/1592953184 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5da4101220 0x7f5da419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:27.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.993+0000 7f5dac400640 1 --2- 172.21.15.90:0/1592953184 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5da41008c0 0x7f5da419a540 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:27.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.993+0000 7f5dac400640 1 -- 172.21.15.90:0/1592953184 >> 172.21.15.90:0/1592953184 conn(0x7f5da40fc4e0 msgr2=0x7f5da40fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:27.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.993+0000 7f5dac400640 1 -- 172.21.15.90:0/1592953184 shutdown_connections 2024-04-24T09:18:27.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:27.993+0000 7f5dac400640 1 -- 172.21.15.90:0/1592953184 wait complete. 2024-04-24T09:18:28.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:27 smithi149 ceph-mon[32592]: mds.? [v2:172.21.15.149:6832/779642417,v1:172.21.15.149:6833/779642417] up:boot 2024-04-24T09:18:28.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:27 smithi149 ceph-mon[32592]: fsmap 1 up:standby 2024-04-24T09:18:28.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:27 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T09:18:28.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:27 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T09:18:28.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:27 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:28.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:27 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:28.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:27 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:28.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:27 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:18:28.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:27 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:18:28.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:27 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:18:28.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:27 smithi149 ceph-mon[32592]: Deploying daemon mds.k on smithi114 2024-04-24T09:18:28.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:27 smithi149 ceph-mon[32592]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:29.211 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:28 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/1592953184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:18:29.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:28 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/1592953184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:18:29.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:28 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/1592953184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:18:29.926 INFO:journalctl@ceph.mds.k.smithi114.stdout:Apr 24 09:18:29 smithi114 systemd[1]: Started Ceph mds.k for 30ee9be2-021a-11ef-bc93-c7b262605968. 2024-04-24T09:18:30.013 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:18:30.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.198+0000 7fa586400640 1 -- 172.21.15.90:0/1060221878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa580102050 msgr2=0x7fa580102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:30.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.198+0000 7fa586400640 1 --2- 172.21.15.90:0/1060221878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa580102050 0x7fa580102470 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa570009900 tx=0x7fa57002a410 comp rx=0 tx=0).stop 2024-04-24T09:18:30.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.198+0000 7fa586400640 1 -- 172.21.15.90:0/1060221878 shutdown_connections 2024-04-24T09:18:30.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.198+0000 7fa586400640 1 --2- 172.21.15.90:0/1060221878 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa580103c00 0x7fa58010a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:30.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.198+0000 7fa586400640 1 --2- 172.21.15.90:0/1060221878 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa580103240 0x7fa5801036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:30.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.198+0000 7fa586400640 1 --2- 172.21.15.90:0/1060221878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa580102050 0x7fa580102470 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:30.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.198+0000 7fa586400640 1 -- 172.21.15.90:0/1060221878 >> 172.21.15.90:0/1060221878 conn(0x7fa5800fd5e0 msgr2=0x7fa5800ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:30.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.198+0000 7fa586400640 1 -- 172.21.15.90:0/1060221878 shutdown_connections 2024-04-24T09:18:30.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.198+0000 7fa586400640 1 -- 172.21.15.90:0/1060221878 wait complete. 2024-04-24T09:18:30.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.199+0000 7fa586400640 1 Processor -- start 2024-04-24T09:18:30.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.200+0000 7fa586400640 1 -- start start 2024-04-24T09:18:30.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.200+0000 7fa586400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa580102050 0x7fa580198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:30.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.200+0000 7fa586400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa580103240 0x7fa5801988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:30.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.200+0000 7fa586400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa580103c00 0x7fa58019f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:30.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.200+0000 7fa586400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa58010dcf0 con 0x7fa580102050 2024-04-24T09:18:30.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.200+0000 7fa586400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa58010db70 con 0x7fa580103c00 2024-04-24T09:18:30.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.200+0000 7fa586400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa58010de70 con 0x7fa580103240 2024-04-24T09:18:30.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.200+0000 7fa57f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa580102050 0x7fa580198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:30.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.200+0000 7fa57ea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa580103240 0x7fa5801988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:30.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.200+0000 7fa57ea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa580103240 0x7fa5801988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:56794/0 (socket says 172.21.15.90:56794) 2024-04-24T09:18:30.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.200+0000 7fa57ea00640 1 -- 172.21.15.90:0/2985718198 learned_addr learned my addr 172.21.15.90:0/2985718198 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:18:30.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.201+0000 7fa57fe00640 1 --2- 172.21.15.90:0/2985718198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa580103c00 0x7fa58019f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:30.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.201+0000 7fa57ea00640 1 -- 172.21.15.90:0/2985718198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa580103c00 msgr2=0x7fa58019f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:30.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.201+0000 7fa57ea00640 1 --2- 172.21.15.90:0/2985718198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa580103c00 0x7fa58019f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:30.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.201+0000 7fa57ea00640 1 -- 172.21.15.90:0/2985718198 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa580102050 msgr2=0x7fa580198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:30.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.201+0000 7fa57ea00640 1 --2- 172.21.15.90:0/2985718198 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa580102050 0x7fa580198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:30.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.201+0000 7fa57ea00640 1 -- 172.21.15.90:0/2985718198 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa58019ff80 con 0x7fa580103240 2024-04-24T09:18:30.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.201+0000 7fa57ea00640 1 --2- 172.21.15.90:0/2985718198 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa580103240 0x7fa5801988a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa56c00e9f0 tx=0x7fa56c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:30.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.202+0000 7fa577e00640 1 -- 172.21.15.90:0/2985718198 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56c00cd80 con 0x7fa580103240 2024-04-24T09:18:30.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.202+0000 7fa577e00640 1 -- 172.21.15.90:0/2985718198 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa56c010910 con 0x7fa580103240 2024-04-24T09:18:30.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.202+0000 7fa577e00640 1 -- 172.21.15.90:0/2985718198 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56c018990 con 0x7fa580103240 2024-04-24T09:18:30.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.202+0000 7fa586400640 1 -- 172.21.15.90:0/2985718198 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5801a0270 con 0x7fa580103240 2024-04-24T09:18:30.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.202+0000 7fa586400640 1 -- 172.21.15.90:0/2985718198 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5801a0770 con 0x7fa580103240 2024-04-24T09:18:30.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.203+0000 7fa586400640 1 -- 172.21.15.90:0/2985718198 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa580109370 con 0x7fa580103240 2024-04-24T09:18:30.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.206+0000 7fa577e00640 1 -- 172.21.15.90:0/2985718198 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa56c010ac0 con 0x7fa580103240 2024-04-24T09:18:30.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.207+0000 7fa577e00640 1 --2- 172.21.15.90:0/2985718198 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa5640a3c30 0x7fa5640a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:30.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.207+0000 7fa577e00640 1 -- 172.21.15.90:0/2985718198 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa56c0d0270 con 0x7fa580103240 2024-04-24T09:18:30.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.207+0000 7fa57f400640 1 --2- 172.21.15.90:0/2985718198 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa5640a3c30 0x7fa5640a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:30.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.208+0000 7fa57f400640 1 --2- 172.21.15.90:0/2985718198 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa5640a3c30 0x7fa5640a60f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa57002aa00 tx=0x7fa5700315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:30.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.212+0000 7fa577e00640 1 -- 172.21.15.90:0/2985718198 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa56c09ca00 con 0x7fa580103240 2024-04-24T09:18:30.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:29 smithi090 ceph-mon[27825]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:30.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:29 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:30.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:29 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:30.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:29 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:30.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:29 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:18:30.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:29 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:18:30.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:29 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:18:30.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:29 smithi149 ceph-mon[32592]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:30.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:29 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:30.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:29 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:30.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:29 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:30.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:29 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:18:30.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:29 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:18:30.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:29 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:18:30.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:29 smithi114 ceph-mon[32501]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:30.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:29 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:30.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:29 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:30.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:29 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:30.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:29 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:18:30.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:29 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:18:30.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:29 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:18:30.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.455+0000 7fa586400640 1 -- 172.21.15.90:0/2985718198 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa580062c00 con 0x7fa580103240 2024-04-24T09:18:30.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.456+0000 7fa577e00640 1 -- 172.21.15.90:0/2985718198 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2218 (secure 0 0 0) 0x7fa56c0a0d00 con 0x7fa580103240 2024-04-24T09:18:30.457 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:18:30.457 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/219377141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":219377141},{"type":"v1","addr":"172.21.15.114:6833","nonce":219377141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/779642417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":779642417},{"type":"v1","addr":"172.21.15.149:6833","nonce":779642417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-24T09:18:30.457 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 3 2024-04-24T09:18:30.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.460+0000 7fa586400640 1 -- 172.21.15.90:0/2985718198 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa5640a3c30 msgr2=0x7fa5640a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:30.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.460+0000 7fa586400640 1 --2- 172.21.15.90:0/2985718198 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa5640a3c30 0x7fa5640a60f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa57002aa00 tx=0x7fa5700315b0 comp rx=0 tx=0).stop 2024-04-24T09:18:30.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.460+0000 7fa586400640 1 -- 172.21.15.90:0/2985718198 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa580103240 msgr2=0x7fa5801988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:30.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.460+0000 7fa586400640 1 --2- 172.21.15.90:0/2985718198 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa580103240 0x7fa5801988a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa56c00e9f0 tx=0x7fa56c00eeb0 comp rx=0 tx=0).stop 2024-04-24T09:18:30.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.460+0000 7fa586400640 1 -- 172.21.15.90:0/2985718198 shutdown_connections 2024-04-24T09:18:30.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.460+0000 7fa586400640 1 --2- 172.21.15.90:0/2985718198 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa5640a3c30 0x7fa5640a60f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:30.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.460+0000 7fa586400640 1 --2- 172.21.15.90:0/2985718198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa580103c00 0x7fa58019f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:30.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.460+0000 7fa586400640 1 --2- 172.21.15.90:0/2985718198 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa580103240 0x7fa5801988a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:30.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.460+0000 7fa586400640 1 --2- 172.21.15.90:0/2985718198 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa580102050 0x7fa580198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:30.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.460+0000 7fa586400640 1 -- 172.21.15.90:0/2985718198 >> 172.21.15.90:0/2985718198 conn(0x7fa5800fd5e0 msgr2=0x7fa580106540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:30.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.460+0000 7fa586400640 1 -- 172.21.15.90:0/2985718198 shutdown_connections 2024-04-24T09:18:30.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:30.460+0000 7fa586400640 1 -- 172.21.15.90:0/2985718198 wait complete. 2024-04-24T09:18:31.188 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:30 smithi090 ceph-mon[27825]: Deploying daemon mds.j on smithi090 2024-04-24T09:18:31.188 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:30 smithi090 ceph-mon[27825]: mds.? [v2:172.21.15.114:6832/219377141,v1:172.21.15.114:6833/219377141] up:boot 2024-04-24T09:18:31.188 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:30 smithi090 ceph-mon[27825]: fsmap 2 up:standby 2024-04-24T09:18:31.188 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:30 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T09:18:31.188 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:30 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/2985718198' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:18:31.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:30 smithi149 ceph-mon[32592]: Deploying daemon mds.j on smithi090 2024-04-24T09:18:31.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:30 smithi149 ceph-mon[32592]: mds.? [v2:172.21.15.114:6832/219377141,v1:172.21.15.114:6833/219377141] up:boot 2024-04-24T09:18:31.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:30 smithi149 ceph-mon[32592]: fsmap 2 up:standby 2024-04-24T09:18:31.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:30 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T09:18:31.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:30 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/2985718198' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:18:31.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:30 smithi114 ceph-mon[32501]: Deploying daemon mds.j on smithi090 2024-04-24T09:18:31.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:30 smithi114 ceph-mon[32501]: mds.? [v2:172.21.15.114:6832/219377141,v1:172.21.15.114:6833/219377141] up:boot 2024-04-24T09:18:31.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:30 smithi114 ceph-mon[32501]: fsmap 2 up:standby 2024-04-24T09:18:31.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:30 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T09:18:31.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:30 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/2985718198' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:18:32.094 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:31 smithi114 ceph-mon[32501]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:32.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:31 smithi090 ceph-mon[27825]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:32.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:31 smithi149 ceph-mon[32592]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:32.487 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:18:32.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:32.736+0000 7f9cc5200640 1 -- 172.21.15.90:0/3430157557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cc01008c0 msgr2=0x7f9cc0102cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:32.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:32.736+0000 7f9cc5200640 1 --2- 172.21.15.90:0/3430157557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cc01008c0 0x7f9cc0102cf0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0009860 tx=0x7f9cb002a340 comp rx=0 tx=0).stop 2024-04-24T09:18:32.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:32.737+0000 7f9cc5200640 1 -- 172.21.15.90:0/3430157557 shutdown_connections 2024-04-24T09:18:32.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:32.737+0000 7f9cc5200640 1 --2- 172.21.15.90:0/3430157557 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9cc0105be0 0x7f9cc0106060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:32.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:32.737+0000 7f9cc5200640 1 --2- 172.21.15.90:0/3430157557 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9cc0103230 0x7f9cc01056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:32.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:32.737+0000 7f9cc5200640 1 --2- 172.21.15.90:0/3430157557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cc01008c0 0x7f9cc0102cf0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:32.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:32.737+0000 7f9cc5200640 1 -- 172.21.15.90:0/3430157557 >> 172.21.15.90:0/3430157557 conn(0x7f9cc00fc4e0 msgr2=0x7f9cc00fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:32.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:32.737+0000 7f9cc5200640 1 -- 172.21.15.90:0/3430157557 shutdown_connections 2024-04-24T09:18:32.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:32.737+0000 7f9cc5200640 1 -- 172.21.15.90:0/3430157557 wait complete. 2024-04-24T09:18:32.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:32.739+0000 7f9cc5200640 1 Processor -- start 2024-04-24T09:18:32.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:32.739+0000 7f9cc5200640 1 -- start start 2024-04-24T09:18:32.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:32.740+0000 7f9cc5200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9cc01008c0 0x7f9cc00707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:32.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:32.740+0000 7f9cc5200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9cc0103230 0x7f9cc0070d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:32.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:32.741+0000 7f9cc5200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cc0105be0 0x7f9cc01ae9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:32.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:32.741+0000 7f9cbde00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9cc01008c0 0x7f9cc00707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:32.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:32.741+0000 7f9cc5200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc01142b0 con 0x7f9cc0103230 2024-04-24T09:18:32.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:32.741+0000 7f9cc5200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc0114130 con 0x7f9cc01008c0 2024-04-24T09:18:32.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:32.741+0000 7f9cc5200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc0114430 con 0x7f9cc0105be0 2024-04-24T09:18:32.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:32.741+0000 7f9cbde00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9cc01008c0 0x7f9cc00707f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.90:57388/0 (socket says 172.21.15.90:57388) 2024-04-24T09:18:32.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:32.741+0000 7f9cbde00640 1 -- 172.21.15.90:0/1032209131 learned_addr learned my addr 172.21.15.90:0/1032209131 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:18:32.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:32.741+0000 7f9cbd400640 1 --2- 172.21.15.90:0/1032209131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9cc0103230 0x7f9cc0070d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:32.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:32.741+0000 7f9cbe800640 1 --2- 172.21.15.90:0/1032209131 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cc0105be0 0x7f9cc01ae9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:32.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:32.741+0000 7f9cbde00640 1 -- 172.21.15.90:0/1032209131 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cc0105be0 msgr2=0x7f9cc01ae9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:32.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:32.741+0000 7f9cbde00640 1 --2- 172.21.15.90:0/1032209131 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cc0105be0 0x7f9cc01ae9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:32.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:32.741+0000 7f9cbde00640 1 -- 172.21.15.90:0/1032209131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9cc0103230 msgr2=0x7f9cc0070d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:32.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:32.742+0000 7f9cbde00640 1 --2- 172.21.15.90:0/1032209131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9cc0103230 0x7f9cc0070d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:32.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:32.742+0000 7f9cbde00640 1 -- 172.21.15.90:0/1032209131 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cc0073eb0 con 0x7f9cc01008c0 2024-04-24T09:18:32.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:32.742+0000 7f9cbd400640 1 --2- 172.21.15.90:0/1032209131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9cc0103230 0x7f9cc0070d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:18:32.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:32.742+0000 7f9cbde00640 1 --2- 172.21.15.90:0/1032209131 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9cc01008c0 0x7f9cc00707f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9cb002a930 tx=0x7f9cb0033ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:32.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:32.742+0000 7f9cb6a00640 1 -- 172.21.15.90:0/1032209131 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0037040 con 0x7f9cc01008c0 2024-04-24T09:18:32.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:32.743+0000 7f9cc5200640 1 -- 172.21.15.90:0/1032209131 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cc01aef00 con 0x7f9cc01008c0 2024-04-24T09:18:32.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:32.743+0000 7f9cb6a00640 1 -- 172.21.15.90:0/1032209131 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cb0030980 con 0x7f9cc01008c0 2024-04-24T09:18:32.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:32.743+0000 7f9cb6a00640 1 -- 172.21.15.90:0/1032209131 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0037040 con 0x7f9cc01008c0 2024-04-24T09:18:32.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:32.743+0000 7f9cc5200640 1 -- 172.21.15.90:0/1032209131 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cc01af2b0 con 0x7f9cc01008c0 2024-04-24T09:18:32.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:32.744+0000 7f9cc5200640 1 -- 172.21.15.90:0/1032209131 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cc0069270 con 0x7f9cc01008c0 2024-04-24T09:18:32.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:32.747+0000 7f9cb6a00640 1 -- 172.21.15.90:0/1032209131 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9cb0005030 con 0x7f9cc01008c0 2024-04-24T09:18:32.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:32.749+0000 7f9cb6a00640 1 --2- 172.21.15.90:0/1032209131 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f9ca40a3e30 0x7f9ca40a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:32.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:32.749+0000 7f9cb6a00640 1 -- 172.21.15.90:0/1032209131 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f9cb00eff50 con 0x7f9cc01008c0 2024-04-24T09:18:32.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:32.749+0000 7f9cbd400640 1 --2- 172.21.15.90:0/1032209131 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f9ca40a3e30 0x7f9ca40a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:32.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:32.750+0000 7f9cbd400640 1 --2- 172.21.15.90:0/1032209131 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f9ca40a3e30 0x7f9ca40a62f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0071d00 tx=0x7f9cac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:32.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:32.753+0000 7f9cb6a00640 1 -- 172.21.15.90:0/1032209131 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9cb00bcd30 con 0x7f9cc01008c0 2024-04-24T09:18:32.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:32.997+0000 7f9cc5200640 1 -- 172.21.15.90:0/1032209131 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9cc0062c00 con 0x7f9cc01008c0 2024-04-24T09:18:32.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:32.997+0000 7f9cb6a00640 1 -- 172.21.15.90:0/1032209131 <== mon.2 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2218 (secure 0 0 0) 0x7f9cb00bc630 con 0x7f9cc01008c0 2024-04-24T09:18:32.998 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:18:32.998 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/219377141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":219377141},{"type":"v1","addr":"172.21.15.114:6833","nonce":219377141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/779642417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":779642417},{"type":"v1","addr":"172.21.15.149:6833","nonce":779642417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-24T09:18:32.998 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 3 2024-04-24T09:18:33.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:33.000+0000 7f9cc5200640 1 -- 172.21.15.90:0/1032209131 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f9ca40a3e30 msgr2=0x7f9ca40a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:33.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:33.000+0000 7f9cc5200640 1 --2- 172.21.15.90:0/1032209131 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f9ca40a3e30 0x7f9ca40a62f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0071d00 tx=0x7f9cac009260 comp rx=0 tx=0).stop 2024-04-24T09:18:33.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:33.000+0000 7f9cc5200640 1 -- 172.21.15.90:0/1032209131 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9cc01008c0 msgr2=0x7f9cc00707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:33.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:33.000+0000 7f9cc5200640 1 --2- 172.21.15.90:0/1032209131 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9cc01008c0 0x7f9cc00707f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9cb002a930 tx=0x7f9cb0033ed0 comp rx=0 tx=0).stop 2024-04-24T09:18:33.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:33.000+0000 7f9cc5200640 1 -- 172.21.15.90:0/1032209131 shutdown_connections 2024-04-24T09:18:33.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:33.000+0000 7f9cc5200640 1 --2- 172.21.15.90:0/1032209131 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f9ca40a3e30 0x7f9ca40a62f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:33.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:33.000+0000 7f9cc5200640 1 --2- 172.21.15.90:0/1032209131 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9cc0105be0 0x7f9cc01ae9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:33.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:33.000+0000 7f9cc5200640 1 --2- 172.21.15.90:0/1032209131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9cc0103230 0x7f9cc0070d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:33.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:33.000+0000 7f9cc5200640 1 --2- 172.21.15.90:0/1032209131 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9cc01008c0 0x7f9cc00707f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:33.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:33.000+0000 7f9cc5200640 1 -- 172.21.15.90:0/1032209131 >> 172.21.15.90:0/1032209131 conn(0x7f9cc00fc4e0 msgr2=0x7f9cc0103bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:33.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:33.000+0000 7f9cc5200640 1 -- 172.21.15.90:0/1032209131 shutdown_connections 2024-04-24T09:18:33.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:33.000+0000 7f9cc5200640 1 -- 172.21.15.90:0/1032209131 wait complete. 2024-04-24T09:18:33.316 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:33 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/1032209131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:18:33.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:33 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/1032209131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:18:33.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:33 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/1032209131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:18:33.603 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 24 09:18:33 smithi090 podman[68038]: 2024-04-24 09:18:33.346094091 +0000 UTC m=+0.160749248 container create 5902404428e682c853e8757243755db4e44d12dda9b5fa2ed9c5d176d3350c8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=) 2024-04-24T09:18:34.058 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 24 09:18:33 smithi090 podman[68038]: 2024-04-24 09:18:33.66735907 +0000 UTC m=+0.482014237 container init 5902404428e682c853e8757243755db4e44d12dda9b5fa2ed9c5d176d3350c8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_CLEAN=True) 2024-04-24T09:18:34.123 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 24 09:18:33 smithi090 podman[68038]: 2024-04-24 09:18:33.677635523 +0000 UTC m=+0.492290661 container start 5902404428e682c853e8757243755db4e44d12dda9b5fa2ed9c5d176d3350c8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-j, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T09:18:34.123 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 24 09:18:33 smithi090 ceph-mds[68056]: -- 172.21.15.90:0/695891520 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5647b9c49a40 con 0x5647b9c1d000 2024-04-24T09:18:34.123 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 24 09:18:33 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-j[68052]: starting mds.j at 2024-04-24T09:18:34.124 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 24 09:18:33 smithi090 bash[68038]: 5902404428e682c853e8757243755db4e44d12dda9b5fa2ed9c5d176d3350c8b 2024-04-24T09:18:34.124 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 24 09:18:33 smithi090 systemd[1]: Started Ceph mds.j for 30ee9be2-021a-11ef-bc93-c7b262605968. 2024-04-24T09:18:34.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:34 smithi090 ceph-mon[27825]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:34.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:34 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:34.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:34 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:34.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:34 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:34.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:34 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:18:34.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:34 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:18:34.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:34 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:18:34.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:34 smithi149 ceph-mon[32592]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:34.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:34 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:34.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:34 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:34.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:34 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:34.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:34 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:18:34.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:34 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:18:34.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:34 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:18:34.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:34 smithi114 ceph-mon[32501]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:34.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:34 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:34.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:34 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:34.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:34 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:34.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:34 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:18:34.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:34 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:18:34.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:34 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:18:35.020 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:18:35.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.231+0000 7fa025600640 1 -- 172.21.15.90:0/1977058274 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa020105470 msgr2=0x7fa0201058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:35.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.231+0000 7fa025600640 1 --2- 172.21.15.90:0/1977058274 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa020105470 0x7fa0201058f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa008009900 tx=0x7fa00802a410 comp rx=0 tx=0).stop 2024-04-24T09:18:35.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.231+0000 7fa025600640 1 -- 172.21.15.90:0/1977058274 shutdown_connections 2024-04-24T09:18:35.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.231+0000 7fa025600640 1 --2- 172.21.15.90:0/1977058274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa020105e30 0x7fa02010c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:35.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.231+0000 7fa025600640 1 --2- 172.21.15.90:0/1977058274 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa020105470 0x7fa0201058f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:35.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.231+0000 7fa025600640 1 --2- 172.21.15.90:0/1977058274 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa020104280 0x7fa0201046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:35.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.231+0000 7fa025600640 1 -- 172.21.15.90:0/1977058274 >> 172.21.15.90:0/1977058274 conn(0x7fa0200ff7f0 msgr2=0x7fa020101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:35.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.231+0000 7fa025600640 1 -- 172.21.15.90:0/1977058274 shutdown_connections 2024-04-24T09:18:35.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.231+0000 7fa025600640 1 -- 172.21.15.90:0/1977058274 wait complete. 2024-04-24T09:18:35.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.232+0000 7fa025600640 1 Processor -- start 2024-04-24T09:18:35.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.233+0000 7fa025600640 1 -- start start 2024-04-24T09:18:35.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.233+0000 7fa025600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa020104280 0x7fa02019a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:35.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.233+0000 7fa025600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa020105470 0x7fa02019ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:35.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.233+0000 7fa025600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa020105e30 0x7fa0201a1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:35.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.233+0000 7fa025600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa02010ff10 con 0x7fa020104280 2024-04-24T09:18:35.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.233+0000 7fa025600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa02010fd90 con 0x7fa020105e30 2024-04-24T09:18:35.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.233+0000 7fa025600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa020110090 con 0x7fa020105470 2024-04-24T09:18:35.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.233+0000 7fa01ea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa020104280 0x7fa02019a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:35.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.233+0000 7fa01ea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa020104280 0x7fa02019a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45528/0 (socket says 172.21.15.90:45528) 2024-04-24T09:18:35.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.233+0000 7fa01ea00640 1 -- 172.21.15.90:0/1841059893 learned_addr learned my addr 172.21.15.90:0/1841059893 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:18:35.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.233+0000 7fa017e00640 1 --2- 172.21.15.90:0/1841059893 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa020105470 0x7fa02019ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:35.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.233+0000 7fa01f400640 1 --2- 172.21.15.90:0/1841059893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa020105e30 0x7fa0201a1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:35.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.234+0000 7fa01ea00640 1 -- 172.21.15.90:0/1841059893 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa020105470 msgr2=0x7fa02019ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:35.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.234+0000 7fa01ea00640 1 --2- 172.21.15.90:0/1841059893 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa020105470 0x7fa02019ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:35.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.234+0000 7fa01ea00640 1 -- 172.21.15.90:0/1841059893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa020105e30 msgr2=0x7fa0201a1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:35.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.234+0000 7fa01ea00640 1 --2- 172.21.15.90:0/1841059893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa020105e30 0x7fa0201a1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:35.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.234+0000 7fa01ea00640 1 -- 172.21.15.90:0/1841059893 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0201a21e0 con 0x7fa020104280 2024-04-24T09:18:35.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.234+0000 7fa017e00640 1 --2- 172.21.15.90:0/1841059893 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa020105470 0x7fa02019ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:18:35.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.234+0000 7fa01ea00640 1 --2- 172.21.15.90:0/1841059893 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa020104280 0x7fa02019a5c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa01000e9c0 tx=0x7fa01000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:35.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.234+0000 7fa017400640 1 -- 172.21.15.90:0/1841059893 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa01000cd80 con 0x7fa020104280 2024-04-24T09:18:35.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.234+0000 7fa017400640 1 -- 172.21.15.90:0/1841059893 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0100035f0 con 0x7fa020104280 2024-04-24T09:18:35.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.234+0000 7fa025600640 1 -- 172.21.15.90:0/1841059893 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0201a24d0 con 0x7fa020104280 2024-04-24T09:18:35.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.234+0000 7fa017400640 1 -- 172.21.15.90:0/1841059893 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa010010690 con 0x7fa020104280 2024-04-24T09:18:35.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.234+0000 7fa025600640 1 -- 172.21.15.90:0/1841059893 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0201a29a0 con 0x7fa020104280 2024-04-24T09:18:35.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.236+0000 7fa025600640 1 -- 172.21.15.90:0/1841059893 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa02010b5f0 con 0x7fa020104280 2024-04-24T09:18:35.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.238+0000 7fa017400640 1 -- 172.21.15.90:0/1841059893 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa0100037a0 con 0x7fa020104280 2024-04-24T09:18:35.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.239+0000 7fa017400640 1 --2- 172.21.15.90:0/1841059893 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa0040a3bb0 0x7fa0040a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:35.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.239+0000 7fa017400640 1 -- 172.21.15.90:0/1841059893 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa0100d04f0 con 0x7fa020104280 2024-04-24T09:18:35.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.239+0000 7fa017e00640 1 --2- 172.21.15.90:0/1841059893 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa0040a3bb0 0x7fa0040a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:35.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.240+0000 7fa017e00640 1 --2- 172.21.15.90:0/1841059893 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa0040a3bb0 0x7fa0040a6070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa008000c00 tx=0x7fa0080095f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:35.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.242+0000 7fa017400640 1 -- 172.21.15.90:0/1841059893 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa01009d370 con 0x7fa020104280 2024-04-24T09:18:35.332 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:35 smithi090 ceph-mon[27825]: Deploying daemon mds.d on smithi090 2024-04-24T09:18:35.332 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:35 smithi090 ceph-mon[27825]: mds.? [v2:172.21.15.90:6834/2396995634,v1:172.21.15.90:6835/2396995634] up:boot 2024-04-24T09:18:35.332 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:35 smithi090 ceph-mon[27825]: fsmap 3 up:standby 2024-04-24T09:18:35.332 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:35 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-24T09:18:35.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:35 smithi149 ceph-mon[32592]: Deploying daemon mds.d on smithi090 2024-04-24T09:18:35.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:35 smithi149 ceph-mon[32592]: mds.? [v2:172.21.15.90:6834/2396995634,v1:172.21.15.90:6835/2396995634] up:boot 2024-04-24T09:18:35.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:35 smithi149 ceph-mon[32592]: fsmap 3 up:standby 2024-04-24T09:18:35.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:35 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-24T09:18:35.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:35 smithi114 ceph-mon[32501]: Deploying daemon mds.d on smithi090 2024-04-24T09:18:35.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:35 smithi114 ceph-mon[32501]: mds.? [v2:172.21.15.90:6834/2396995634,v1:172.21.15.90:6835/2396995634] up:boot 2024-04-24T09:18:35.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:35 smithi114 ceph-mon[32501]: fsmap 3 up:standby 2024-04-24T09:18:35.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:35 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-24T09:18:35.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.496+0000 7fa025600640 1 -- 172.21.15.90:0/1841059893 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa02002d620 con 0x7fa020104280 2024-04-24T09:18:35.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.497+0000 7fa017400640 1 -- 172.21.15.90:0/1841059893 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3043 (secure 0 0 0) 0x7fa01009cc70 con 0x7fa020104280 2024-04-24T09:18:35.497 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:18:35.497 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/219377141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":219377141},{"type":"v1","addr":"172.21.15.114:6833","nonce":219377141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24446,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2396995634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2396995634},{"type":"v1","addr":"172.21.15.90:6835","nonce":2396995634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/779642417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":779642417},{"type":"v1","addr":"172.21.15.149:6833","nonce":779642417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-24T09:18:35.497 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 4 2024-04-24T09:18:35.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.499+0000 7fa025600640 1 -- 172.21.15.90:0/1841059893 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa0040a3bb0 msgr2=0x7fa0040a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:35.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.499+0000 7fa025600640 1 --2- 172.21.15.90:0/1841059893 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa0040a3bb0 0x7fa0040a6070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa008000c00 tx=0x7fa0080095f0 comp rx=0 tx=0).stop 2024-04-24T09:18:35.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.499+0000 7fa025600640 1 -- 172.21.15.90:0/1841059893 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa020104280 msgr2=0x7fa02019a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:35.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.499+0000 7fa025600640 1 --2- 172.21.15.90:0/1841059893 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa020104280 0x7fa02019a5c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa01000e9c0 tx=0x7fa01000ee80 comp rx=0 tx=0).stop 2024-04-24T09:18:35.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.500+0000 7fa025600640 1 -- 172.21.15.90:0/1841059893 shutdown_connections 2024-04-24T09:18:35.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.500+0000 7fa025600640 1 --2- 172.21.15.90:0/1841059893 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa0040a3bb0 0x7fa0040a6070 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:35.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.500+0000 7fa025600640 1 --2- 172.21.15.90:0/1841059893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa020105e30 0x7fa0201a1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:35.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.500+0000 7fa025600640 1 --2- 172.21.15.90:0/1841059893 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa020105470 0x7fa02019ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:35.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.500+0000 7fa025600640 1 --2- 172.21.15.90:0/1841059893 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa020104280 0x7fa02019a5c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:35.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.500+0000 7fa025600640 1 -- 172.21.15.90:0/1841059893 >> 172.21.15.90:0/1841059893 conn(0x7fa0200ff7f0 msgr2=0x7fa0201087c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:35.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.500+0000 7fa025600640 1 -- 172.21.15.90:0/1841059893 shutdown_connections 2024-04-24T09:18:35.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:35.500+0000 7fa025600640 1 -- 172.21.15.90:0/1841059893 wait complete. 2024-04-24T09:18:36.324 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:36 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/1841059893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:18:36.325 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:36 smithi090 ceph-mon[27825]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:36.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:36 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/1841059893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:18:36.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:36 smithi149 ceph-mon[32592]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:36.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:36 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/1841059893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:18:36.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:36 smithi114 ceph-mon[32501]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:37.516 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 24 09:18:37 smithi090 systemd[1]: Starting Ceph mds.d for 30ee9be2-021a-11ef-bc93-c7b262605968... 2024-04-24T09:18:37.520 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:18:37.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.720+0000 7f0cdea00640 1 -- 172.21.15.90:0/3589243545 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cd8105400 msgr2=0x7f0cd81077f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:37.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.720+0000 7f0cdea00640 1 --2- 172.21.15.90:0/3589243545 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cd8105400 0x7f0cd81077f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4009900 tx=0x7f0cc402a3f0 comp rx=0 tx=0).stop 2024-04-24T09:18:37.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.720+0000 7f0cdea00640 1 -- 172.21.15.90:0/3589243545 shutdown_connections 2024-04-24T09:18:37.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.720+0000 7f0cdea00640 1 --2- 172.21.15.90:0/3589243545 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0cd8107d30 0x7f0cd810a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:37.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.720+0000 7f0cdea00640 1 --2- 172.21.15.90:0/3589243545 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cd8105400 0x7f0cd81077f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:37.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.720+0000 7f0cdea00640 1 --2- 172.21.15.90:0/3589243545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0cd8102ad0 0x7f0cd8104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:37.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.720+0000 7f0cdea00640 1 -- 172.21.15.90:0/3589243545 >> 172.21.15.90:0/3589243545 conn(0x7f0cd80fc4e0 msgr2=0x7f0cd80fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:37.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.721+0000 7f0cdea00640 1 -- 172.21.15.90:0/3589243545 shutdown_connections 2024-04-24T09:18:37.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.721+0000 7f0cdea00640 1 -- 172.21.15.90:0/3589243545 wait complete. 2024-04-24T09:18:37.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.721+0000 7f0cdea00640 1 Processor -- start 2024-04-24T09:18:37.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.722+0000 7f0cdea00640 1 -- start start 2024-04-24T09:18:37.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.722+0000 7f0cdea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0cd8102ad0 0x7f0cd819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:37.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.722+0000 7f0cdea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cd8105400 0x7f0cd819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:37.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.722+0000 7f0cdea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0cd8107d30 0x7f0cd81a1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:37.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.722+0000 7f0cdea00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd810dd30 con 0x7f0cd8105400 2024-04-24T09:18:37.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.722+0000 7f0cdea00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd810dbb0 con 0x7f0cd8102ad0 2024-04-24T09:18:37.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.722+0000 7f0cdea00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd810deb0 con 0x7f0cd8107d30 2024-04-24T09:18:37.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.722+0000 7f0cd7e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0cd8102ad0 0x7f0cd819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:37.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.722+0000 7f0cd7e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0cd8102ad0 0x7f0cd819a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.90:52676/0 (socket says 172.21.15.90:52676) 2024-04-24T09:18:37.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.722+0000 7f0cd7e00640 1 -- 172.21.15.90:0/3650969214 learned_addr learned my addr 172.21.15.90:0/3650969214 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:18:37.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.722+0000 7f0cd7400640 1 --2- 172.21.15.90:0/3650969214 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cd8105400 0x7f0cd819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:37.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.722+0000 7f0cdca00640 1 --2- 172.21.15.90:0/3650969214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0cd8107d30 0x7f0cd81a1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:37.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.723+0000 7f0cd7e00640 1 -- 172.21.15.90:0/3650969214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0cd8107d30 msgr2=0x7f0cd81a1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:37.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.723+0000 7f0cd7e00640 1 --2- 172.21.15.90:0/3650969214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0cd8107d30 0x7f0cd81a1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:37.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.723+0000 7f0cd7e00640 1 -- 172.21.15.90:0/3650969214 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cd8105400 msgr2=0x7f0cd819ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:37.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.723+0000 7f0cd7e00640 1 --2- 172.21.15.90:0/3650969214 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cd8105400 0x7f0cd819ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:37.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.723+0000 7f0cd7e00640 1 -- 172.21.15.90:0/3650969214 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cd81a2220 con 0x7f0cd8102ad0 2024-04-24T09:18:37.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.723+0000 7f0cd7e00640 1 --2- 172.21.15.90:0/3650969214 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0cd8102ad0 0x7f0cd819a600 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0cc800e9f0 tx=0x7f0cc800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:37.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.723+0000 7f0cd4c00640 1 -- 172.21.15.90:0/3650969214 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc800cd80 con 0x7f0cd8102ad0 2024-04-24T09:18:37.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.723+0000 7f0cd4c00640 1 -- 172.21.15.90:0/3650969214 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cc8005500 con 0x7f0cd8102ad0 2024-04-24T09:18:37.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.723+0000 7f0cdea00640 1 -- 172.21.15.90:0/3650969214 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cd81a2510 con 0x7f0cd8102ad0 2024-04-24T09:18:37.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.723+0000 7f0cd4c00640 1 -- 172.21.15.90:0/3650969214 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc8010690 con 0x7f0cd8102ad0 2024-04-24T09:18:37.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.723+0000 7f0cdea00640 1 -- 172.21.15.90:0/3650969214 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cd81a2a30 con 0x7f0cd8102ad0 2024-04-24T09:18:37.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.724+0000 7f0cdea00640 1 -- 172.21.15.90:0/3650969214 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cd8109880 con 0x7f0cd8102ad0 2024-04-24T09:18:37.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.727+0000 7f0cd4c00640 1 -- 172.21.15.90:0/3650969214 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0cc80046c0 con 0x7f0cd8102ad0 2024-04-24T09:18:37.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.728+0000 7f0cd4c00640 1 --2- 172.21.15.90:0/3650969214 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f0cbc0a3c30 0x7f0cbc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:37.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.728+0000 7f0cd4c00640 1 -- 172.21.15.90:0/3650969214 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f0cc80d0380 con 0x7f0cd8102ad0 2024-04-24T09:18:37.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.728+0000 7f0cd7400640 1 --2- 172.21.15.90:0/3650969214 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f0cbc0a3c30 0x7f0cbc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:37.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.729+0000 7f0cd7400640 1 --2- 172.21.15.90:0/3650969214 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f0cbc0a3c30 0x7f0cbc0a60f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0cc402a9e0 tx=0x7f0cc40315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:37.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.732+0000 7f0cd4c00640 1 -- 172.21.15.90:0/3650969214 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0cc809d160 con 0x7f0cd8102ad0 2024-04-24T09:18:37.852 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 24 09:18:37 smithi090 podman[68380]: 2024-04-24 09:18:37.423182833 +0000 UTC m=+0.038528513 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:18:37.852 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 24 09:18:37 smithi090 podman[68380]: 2024-04-24 09:18:37.550536457 +0000 UTC m=+0.165882121 container create 65013c07581f6edae5a6c6298d948af91f66c24da1bfdde8509f1d47d740e6c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-24T09:18:37.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.978+0000 7f0cdea00640 1 -- 172.21.15.90:0/3650969214 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0cd802d620 con 0x7f0cd8102ad0 2024-04-24T09:18:37.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.979+0000 7f0cd4c00640 1 -- 172.21.15.90:0/3650969214 <== mon.2 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3043 (secure 0 0 0) 0x7f0cc809ca60 con 0x7f0cd8102ad0 2024-04-24T09:18:37.979 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:18:37.979 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/219377141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":219377141},{"type":"v1","addr":"172.21.15.114:6833","nonce":219377141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24446,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2396995634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2396995634},{"type":"v1","addr":"172.21.15.90:6835","nonce":2396995634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/779642417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":779642417},{"type":"v1","addr":"172.21.15.149:6833","nonce":779642417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-24T09:18:37.979 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 4 2024-04-24T09:18:37.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.982+0000 7f0cdea00640 1 -- 172.21.15.90:0/3650969214 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f0cbc0a3c30 msgr2=0x7f0cbc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:37.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.982+0000 7f0cdea00640 1 --2- 172.21.15.90:0/3650969214 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f0cbc0a3c30 0x7f0cbc0a60f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0cc402a9e0 tx=0x7f0cc40315b0 comp rx=0 tx=0).stop 2024-04-24T09:18:37.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.982+0000 7f0cdea00640 1 -- 172.21.15.90:0/3650969214 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0cd8102ad0 msgr2=0x7f0cd819a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:37.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.982+0000 7f0cdea00640 1 --2- 172.21.15.90:0/3650969214 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0cd8102ad0 0x7f0cd819a600 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0cc800e9f0 tx=0x7f0cc800edf0 comp rx=0 tx=0).stop 2024-04-24T09:18:37.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.982+0000 7f0cdea00640 1 -- 172.21.15.90:0/3650969214 shutdown_connections 2024-04-24T09:18:37.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.982+0000 7f0cdea00640 1 --2- 172.21.15.90:0/3650969214 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f0cbc0a3c30 0x7f0cbc0a60f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:37.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.982+0000 7f0cdea00640 1 --2- 172.21.15.90:0/3650969214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0cd8107d30 0x7f0cd81a1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:37.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.982+0000 7f0cdea00640 1 --2- 172.21.15.90:0/3650969214 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cd8105400 0x7f0cd819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:37.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.982+0000 7f0cdea00640 1 --2- 172.21.15.90:0/3650969214 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0cd8102ad0 0x7f0cd819a600 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:37.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.982+0000 7f0cdea00640 1 -- 172.21.15.90:0/3650969214 >> 172.21.15.90:0/3650969214 conn(0x7f0cd80fc4e0 msgr2=0x7f0cd81022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:37.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.983+0000 7f0cdea00640 1 -- 172.21.15.90:0/3650969214 shutdown_connections 2024-04-24T09:18:37.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:37.983+0000 7f0cdea00640 1 -- 172.21.15.90:0/3650969214 wait complete. 2024-04-24T09:18:38.353 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 24 09:18:37 smithi090 podman[68380]: 2024-04-24 09:18:37.869140174 +0000 UTC m=+0.484485877 container init 65013c07581f6edae5a6c6298d948af91f66c24da1bfdde8509f1d47d740e6c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=) 2024-04-24T09:18:38.353 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 24 09:18:37 smithi090 podman[68380]: 2024-04-24 09:18:37.881158892 +0000 UTC m=+0.496504611 container start 65013c07581f6edae5a6c6298d948af91f66c24da1bfdde8509f1d47d740e6c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-d, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-24T09:18:38.353 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 24 09:18:37 smithi090 ceph-mds[68437]: -- 172.21.15.90:0/218653298 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5596c9647a40 con 0x5596c961a800 2024-04-24T09:18:38.353 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 24 09:18:37 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-d[68433]: starting mds.d at 2024-04-24T09:18:38.353 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 24 09:18:37 smithi090 bash[68380]: 65013c07581f6edae5a6c6298d948af91f66c24da1bfdde8509f1d47d740e6c9 2024-04-24T09:18:38.353 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 24 09:18:37 smithi090 systemd[1]: Started Ceph mds.d for 30ee9be2-021a-11ef-bc93-c7b262605968. 2024-04-24T09:18:38.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:38 smithi090 ceph-mon[27825]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:38.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:38 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/3650969214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:18:38.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:38 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:38.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:38 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:38.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:38 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:38.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:38 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:18:38.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:38 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:18:38.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:38 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:18:38.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:38 smithi149 ceph-mon[32592]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:38.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:38 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/3650969214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:18:38.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:38 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:38.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:38 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:38.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:38 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:38.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:38 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:18:38.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:38 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:18:38.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:38 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:18:38.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:38 smithi114 ceph-mon[32501]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:38.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:38 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/3650969214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:18:38.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:38 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:38.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:38 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:38.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:38 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:38.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:38 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:18:38.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:38 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:18:38.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:38 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:18:39.733 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:39 smithi090 ceph-mon[27825]: Deploying daemon mds.e on smithi114 2024-04-24T09:18:39.733 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:39 smithi090 ceph-mon[27825]: mds.? [v2:172.21.15.90:6836/227251251,v1:172.21.15.90:6837/227251251] up:boot 2024-04-24T09:18:39.733 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:39 smithi090 ceph-mon[27825]: fsmap 4 up:standby 2024-04-24T09:18:39.733 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:39 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-24T09:18:39.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:39 smithi149 ceph-mon[32592]: Deploying daemon mds.e on smithi114 2024-04-24T09:18:39.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:39 smithi149 ceph-mon[32592]: mds.? [v2:172.21.15.90:6836/227251251,v1:172.21.15.90:6837/227251251] up:boot 2024-04-24T09:18:39.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:39 smithi149 ceph-mon[32592]: fsmap 4 up:standby 2024-04-24T09:18:39.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:39 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-24T09:18:39.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:39 smithi114 ceph-mon[32501]: Deploying daemon mds.e on smithi114 2024-04-24T09:18:39.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:39 smithi114 ceph-mon[32501]: mds.? [v2:172.21.15.90:6836/227251251,v1:172.21.15.90:6837/227251251] up:boot 2024-04-24T09:18:39.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:39 smithi114 ceph-mon[32501]: fsmap 4 up:standby 2024-04-24T09:18:39.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:39 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-24T09:18:40.007 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:18:40.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.200+0000 7f25d4400640 1 -- 172.21.15.90:0/4225438646 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25cc105450 msgr2=0x7f25cc1058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:40.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.200+0000 7f25d4400640 1 --2- 172.21.15.90:0/4225438646 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25cc105450 0x7f25cc1058d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f25b8009900 tx=0x7f25b802a410 comp rx=0 tx=0).stop 2024-04-24T09:18:40.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.200+0000 7f25d4400640 1 -- 172.21.15.90:0/4225438646 shutdown_connections 2024-04-24T09:18:40.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.200+0000 7f25d4400640 1 --2- 172.21.15.90:0/4225438646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25cc105e10 0x7f25cc10c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:40.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.200+0000 7f25d4400640 1 --2- 172.21.15.90:0/4225438646 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25cc105450 0x7f25cc1058d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:40.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.200+0000 7f25d4400640 1 --2- 172.21.15.90:0/4225438646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25cc104260 0x7f25cc104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:40.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.200+0000 7f25d4400640 1 -- 172.21.15.90:0/4225438646 >> 172.21.15.90:0/4225438646 conn(0x7f25cc0ff7f0 msgr2=0x7f25cc101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:40.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.200+0000 7f25d4400640 1 -- 172.21.15.90:0/4225438646 shutdown_connections 2024-04-24T09:18:40.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.200+0000 7f25d4400640 1 -- 172.21.15.90:0/4225438646 wait complete. 2024-04-24T09:18:40.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.202+0000 7f25d4400640 1 Processor -- start 2024-04-24T09:18:40.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.202+0000 7f25d4400640 1 -- start start 2024-04-24T09:18:40.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.202+0000 7f25d4400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25cc104260 0x7f25cc19a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:40.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.202+0000 7f25d4400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25cc105450 0x7f25cc19ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:40.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.202+0000 7f25d4400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25cc105e10 0x7f25cc1a1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:40.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.202+0000 7f25d4400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f25cc10ff40 con 0x7f25cc104260 2024-04-24T09:18:40.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.202+0000 7f25d4400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f25cc10fdc0 con 0x7f25cc105e10 2024-04-24T09:18:40.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.202+0000 7f25d4400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f25cc1100c0 con 0x7f25cc105450 2024-04-24T09:18:40.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.203+0000 7f25d1a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25cc104260 0x7f25cc19a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:40.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.203+0000 7f25d1a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25cc104260 0x7f25cc19a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48420/0 (socket says 172.21.15.90:48420) 2024-04-24T09:18:40.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.203+0000 7f25d1a00640 1 -- 172.21.15.90:0/2608154126 learned_addr learned my addr 172.21.15.90:0/2608154126 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:18:40.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.203+0000 7f25d1000640 1 --2- 172.21.15.90:0/2608154126 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25cc105450 0x7f25cc19ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:40.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.203+0000 7f25d2400640 1 --2- 172.21.15.90:0/2608154126 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25cc105e10 0x7f25cc1a1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:40.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.203+0000 7f25d1a00640 1 -- 172.21.15.90:0/2608154126 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25cc105450 msgr2=0x7f25cc19ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:40.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.203+0000 7f25d1a00640 1 --2- 172.21.15.90:0/2608154126 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25cc105450 0x7f25cc19ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:40.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.203+0000 7f25d1a00640 1 -- 172.21.15.90:0/2608154126 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25cc105e10 msgr2=0x7f25cc1a1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:40.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.203+0000 7f25d1a00640 1 --2- 172.21.15.90:0/2608154126 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25cc105e10 0x7f25cc1a1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:40.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.203+0000 7f25d1a00640 1 -- 172.21.15.90:0/2608154126 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25cc1a21f0 con 0x7f25cc104260 2024-04-24T09:18:40.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.203+0000 7f25d1a00640 1 --2- 172.21.15.90:0/2608154126 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25cc104260 0x7f25cc19a5d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f25c000e9c0 tx=0x7f25c000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:40.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.204+0000 7f25ca000640 1 -- 172.21.15.90:0/2608154126 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c000cd80 con 0x7f25cc104260 2024-04-24T09:18:40.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.204+0000 7f25ca000640 1 -- 172.21.15.90:0/2608154126 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25c0010910 con 0x7f25cc104260 2024-04-24T09:18:40.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.204+0000 7f25ca000640 1 -- 172.21.15.90:0/2608154126 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c0018950 con 0x7f25cc104260 2024-04-24T09:18:40.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.204+0000 7f25d4400640 1 -- 172.21.15.90:0/2608154126 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25cc1a24e0 con 0x7f25cc104260 2024-04-24T09:18:40.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.204+0000 7f25d4400640 1 -- 172.21.15.90:0/2608154126 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25cc1a2a50 con 0x7f25cc104260 2024-04-24T09:18:40.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.205+0000 7f25d4400640 1 -- 172.21.15.90:0/2608154126 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25cc10b520 con 0x7f25cc104260 2024-04-24T09:18:40.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.207+0000 7f25ca000640 1 -- 172.21.15.90:0/2608154126 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f25c001f020 con 0x7f25cc104260 2024-04-24T09:18:40.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.209+0000 7f25ca000640 1 --2- 172.21.15.90:0/2608154126 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f25b40a3c30 0x7f25b40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:40.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.209+0000 7f25ca000640 1 -- 172.21.15.90:0/2608154126 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f25c00d0520 con 0x7f25cc104260 2024-04-24T09:18:40.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.209+0000 7f25d1000640 1 --2- 172.21.15.90:0/2608154126 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f25b40a3c30 0x7f25b40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:40.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.210+0000 7f25d1000640 1 --2- 172.21.15.90:0/2608154126 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f25b40a3c30 0x7f25b40a60f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f25b8031440 tx=0x7f25b80315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:40.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.212+0000 7f25ca000640 1 -- 172.21.15.90:0/2608154126 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f25c009d3a0 con 0x7f25cc104260 2024-04-24T09:18:40.410 INFO:journalctl@ceph.mds.e.smithi114.stdout:Apr 24 09:18:40 smithi114 systemd[1]: Starting Ceph mds.e for 30ee9be2-021a-11ef-bc93-c7b262605968... 2024-04-24T09:18:40.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.454+0000 7f25d4400640 1 -- 172.21.15.90:0/2608154126 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f25cc02d620 con 0x7f25cc104260 2024-04-24T09:18:40.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.455+0000 7f25ca000640 1 -- 172.21.15.90:0/2608154126 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3865 (secure 0 0 0) 0x7f25c009cca0 con 0x7f25cc104260 2024-04-24T09:18:40.455 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:18:40.455 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/227251251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":227251251},{"type":"v1","addr":"172.21.15.90:6837","nonce":227251251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/219377141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":219377141},{"type":"v1","addr":"172.21.15.114:6833","nonce":219377141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24446,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2396995634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2396995634},{"type":"v1","addr":"172.21.15.90:6835","nonce":2396995634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/779642417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":779642417},{"type":"v1","addr":"172.21.15.149:6833","nonce":779642417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-24T09:18:40.456 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 5 2024-04-24T09:18:40.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.457+0000 7f25d4400640 1 -- 172.21.15.90:0/2608154126 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f25b40a3c30 msgr2=0x7f25b40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:40.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.457+0000 7f25d4400640 1 --2- 172.21.15.90:0/2608154126 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f25b40a3c30 0x7f25b40a60f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f25b8031440 tx=0x7f25b80315b0 comp rx=0 tx=0).stop 2024-04-24T09:18:40.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.457+0000 7f25d4400640 1 -- 172.21.15.90:0/2608154126 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25cc104260 msgr2=0x7f25cc19a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:40.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.457+0000 7f25d4400640 1 --2- 172.21.15.90:0/2608154126 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25cc104260 0x7f25cc19a5d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f25c000e9c0 tx=0x7f25c000edc0 comp rx=0 tx=0).stop 2024-04-24T09:18:40.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.458+0000 7f25d4400640 1 -- 172.21.15.90:0/2608154126 shutdown_connections 2024-04-24T09:18:40.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.458+0000 7f25d4400640 1 --2- 172.21.15.90:0/2608154126 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f25b40a3c30 0x7f25b40a60f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:40.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.458+0000 7f25d4400640 1 --2- 172.21.15.90:0/2608154126 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25cc105e10 0x7f25cc1a1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:40.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.458+0000 7f25d4400640 1 --2- 172.21.15.90:0/2608154126 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25cc105450 0x7f25cc19ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:40.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.458+0000 7f25d4400640 1 --2- 172.21.15.90:0/2608154126 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25cc104260 0x7f25cc19a5d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:40.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.458+0000 7f25d4400640 1 -- 172.21.15.90:0/2608154126 >> 172.21.15.90:0/2608154126 conn(0x7f25cc0ff7f0 msgr2=0x7f25cc1086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:40.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.458+0000 7f25d4400640 1 -- 172.21.15.90:0/2608154126 shutdown_connections 2024-04-24T09:18:40.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:40.458+0000 7f25d4400640 1 -- 172.21.15.90:0/2608154126 wait complete. 2024-04-24T09:18:40.670 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:40 smithi114 ceph-mon[32501]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:40.670 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:40 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/2608154126' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:18:40.670 INFO:journalctl@ceph.mds.e.smithi114.stdout:Apr 24 09:18:40 smithi114 podman[62962]: 2024-04-24 09:18:40.31369831 +0000 UTC m=+0.018236454 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:18:40.670 INFO:journalctl@ceph.mds.e.smithi114.stdout:Apr 24 09:18:40 smithi114 podman[62962]: 2024-04-24 09:18:40.418173904 +0000 UTC m=+0.122712041 container create 668c731cf63b2dd70d5c88ef75e206b8a5f8a6a5e6c3d51d2d440ca4e11856f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-e, org.label-schema.schema-version=1.0, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T09:18:40.670 INFO:journalctl@ceph.mds.e.smithi114.stdout:Apr 24 09:18:40 smithi114 podman[62962]: 2024-04-24 09:18:40.671218096 +0000 UTC m=+0.375756246 container init 668c731cf63b2dd70d5c88ef75e206b8a5f8a6a5e6c3d51d2d440ca4e11856f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-e, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, ceph=True) 2024-04-24T09:18:40.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:40 smithi090 ceph-mon[27825]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:40.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:40 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/2608154126' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:18:40.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:40 smithi149 ceph-mon[32592]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:40.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:40 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/2608154126' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:18:40.926 INFO:journalctl@ceph.mds.e.smithi114.stdout:Apr 24 09:18:40 smithi114 podman[62962]: 2024-04-24 09:18:40.673923748 +0000 UTC m=+0.378461898 container start 668c731cf63b2dd70d5c88ef75e206b8a5f8a6a5e6c3d51d2d440ca4e11856f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-e, ceph=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-24T09:18:40.926 INFO:journalctl@ceph.mds.e.smithi114.stdout:Apr 24 09:18:40 smithi114 ceph-mds[62981]: -- 172.21.15.114:0/77721951 <== mon.2 v2:172.21.15.114:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55597faefa40 con 0x55597fac2800 2024-04-24T09:18:40.926 INFO:journalctl@ceph.mds.e.smithi114.stdout:Apr 24 09:18:40 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-e[62977]: starting mds.e at 2024-04-24T09:18:40.926 INFO:journalctl@ceph.mds.e.smithi114.stdout:Apr 24 09:18:40 smithi114 bash[62962]: 668c731cf63b2dd70d5c88ef75e206b8a5f8a6a5e6c3d51d2d440ca4e11856f6 2024-04-24T09:18:40.926 INFO:journalctl@ceph.mds.e.smithi114.stdout:Apr 24 09:18:40 smithi114 systemd[1]: Started Ceph mds.e for 30ee9be2-021a-11ef-bc93-c7b262605968. 2024-04-24T09:18:42.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:41 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:42.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:41 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:42.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:41 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:42.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:41 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:18:42.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:41 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:18:42.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:41 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:18:42.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:41 smithi090 ceph-mon[27825]: Deploying daemon mds.f on smithi149 2024-04-24T09:18:42.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:41 smithi090 ceph-mon[27825]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:42.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:41 smithi090 ceph-mon[27825]: mds.? [v2:172.21.15.114:6834/2565340800,v1:172.21.15.114:6835/2565340800] up:boot 2024-04-24T09:18:42.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:41 smithi090 ceph-mon[27825]: fsmap 5 up:standby 2024-04-24T09:18:42.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:41 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T09:18:42.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:41 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:42.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:41 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:42.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:41 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:42.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:41 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:18:42.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:41 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:18:42.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:41 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:18:42.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:41 smithi149 ceph-mon[32592]: Deploying daemon mds.f on smithi149 2024-04-24T09:18:42.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:41 smithi149 ceph-mon[32592]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:42.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:41 smithi149 ceph-mon[32592]: mds.? [v2:172.21.15.114:6834/2565340800,v1:172.21.15.114:6835/2565340800] up:boot 2024-04-24T09:18:42.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:41 smithi149 ceph-mon[32592]: fsmap 5 up:standby 2024-04-24T09:18:42.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:41 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T09:18:42.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:41 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:42.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:41 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:42.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:41 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:42.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:41 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:18:42.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:41 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:18:42.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:41 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:18:42.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:41 smithi114 ceph-mon[32501]: Deploying daemon mds.f on smithi149 2024-04-24T09:18:42.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:41 smithi114 ceph-mon[32501]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:42.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:41 smithi114 ceph-mon[32501]: mds.? [v2:172.21.15.114:6834/2565340800,v1:172.21.15.114:6835/2565340800] up:boot 2024-04-24T09:18:42.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:41 smithi114 ceph-mon[32501]: fsmap 5 up:standby 2024-04-24T09:18:42.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:41 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T09:18:42.476 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:18:42.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.688+0000 7fa9a4e00640 1 -- 172.21.15.90:0/3661940003 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa9a0109850 msgr2=0x7fa9a0109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:42.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.688+0000 7fa9a4e00640 1 --2- 172.21.15.90:0/3661940003 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa9a0109850 0x7fa9a0109cd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa988009900 tx=0x7fa98802a410 comp rx=0 tx=0).stop 2024-04-24T09:18:42.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.689+0000 7fa9a4e00640 1 -- 172.21.15.90:0/3661940003 shutdown_connections 2024-04-24T09:18:42.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.689+0000 7fa9a4e00640 1 --2- 172.21.15.90:0/3661940003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa9a010a210 0x7fa9a0110aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:42.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.689+0000 7fa9a4e00640 1 --2- 172.21.15.90:0/3661940003 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa9a0109850 0x7fa9a0109cd0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:42.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.689+0000 7fa9a4e00640 1 --2- 172.21.15.90:0/3661940003 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9a0108660 0x7fa9a0108a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:42.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.689+0000 7fa9a4e00640 1 -- 172.21.15.90:0/3661940003 >> 172.21.15.90:0/3661940003 conn(0x7fa9a0077620 msgr2=0x7fa9a0079a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:42.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.689+0000 7fa9a4e00640 1 -- 172.21.15.90:0/3661940003 shutdown_connections 2024-04-24T09:18:42.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.689+0000 7fa9a4e00640 1 -- 172.21.15.90:0/3661940003 wait complete. 2024-04-24T09:18:42.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.690+0000 7fa9a4e00640 1 Processor -- start 2024-04-24T09:18:42.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.690+0000 7fa9a4e00640 1 -- start start 2024-04-24T09:18:42.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.690+0000 7fa9a4e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa9a0108660 0x7fa9a019e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:42.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.690+0000 7fa9a4e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa9a0109850 0x7fa9a019eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:42.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.690+0000 7fa9a4e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9a010a210 0x7fa9a01a5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:42.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.690+0000 7fa9a4e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a0114310 con 0x7fa9a0109850 2024-04-24T09:18:42.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.690+0000 7fa9a4e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a0114190 con 0x7fa9a0108660 2024-04-24T09:18:42.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.690+0000 7fa9a4e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a0114490 con 0x7fa9a010a210 2024-04-24T09:18:42.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.690+0000 7fa997e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa9a0109850 0x7fa9a019eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:42.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.690+0000 7fa997e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa9a0109850 0x7fa9a019eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48438/0 (socket says 172.21.15.90:48438) 2024-04-24T09:18:42.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.690+0000 7fa997e00640 1 -- 172.21.15.90:0/1676869834 learned_addr learned my addr 172.21.15.90:0/1676869834 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:18:42.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.690+0000 7fa99de00640 1 --2- 172.21.15.90:0/1676869834 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa9a0108660 0x7fa9a019e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:42.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.690+0000 7fa99e800640 1 --2- 172.21.15.90:0/1676869834 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9a010a210 0x7fa9a01a5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:42.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.691+0000 7fa997e00640 1 -- 172.21.15.90:0/1676869834 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9a010a210 msgr2=0x7fa9a01a5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:42.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.691+0000 7fa997e00640 1 --2- 172.21.15.90:0/1676869834 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9a010a210 0x7fa9a01a5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:42.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.691+0000 7fa997e00640 1 -- 172.21.15.90:0/1676869834 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa9a0108660 msgr2=0x7fa9a019e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:42.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.691+0000 7fa997e00640 1 --2- 172.21.15.90:0/1676869834 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa9a0108660 0x7fa9a019e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:42.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.691+0000 7fa997e00640 1 -- 172.21.15.90:0/1676869834 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a01a65a0 con 0x7fa9a0109850 2024-04-24T09:18:42.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.691+0000 7fa99de00640 1 --2- 172.21.15.90:0/1676869834 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa9a0108660 0x7fa9a019e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:18:42.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.691+0000 7fa997e00640 1 --2- 172.21.15.90:0/1676869834 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa9a0109850 0x7fa9a019eec0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa988031440 tx=0x7fa988032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:42.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.691+0000 7fa996a00640 1 -- 172.21.15.90:0/1676869834 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98803f030 con 0x7fa9a0109850 2024-04-24T09:18:42.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.691+0000 7fa996a00640 1 -- 172.21.15.90:0/1676869834 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa988003ce0 con 0x7fa9a0109850 2024-04-24T09:18:42.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.691+0000 7fa996a00640 1 -- 172.21.15.90:0/1676869834 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa988035070 con 0x7fa9a0109850 2024-04-24T09:18:42.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.691+0000 7fa9a4e00640 1 -- 172.21.15.90:0/1676869834 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9a01a6830 con 0x7fa9a0109850 2024-04-24T09:18:42.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.692+0000 7fa9a4e00640 1 -- 172.21.15.90:0/1676869834 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9a01a6d20 con 0x7fa9a0109850 2024-04-24T09:18:42.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.693+0000 7fa9a4e00640 1 -- 172.21.15.90:0/1676869834 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9a010f980 con 0x7fa9a0109850 2024-04-24T09:18:42.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.694+0000 7fa996a00640 1 -- 172.21.15.90:0/1676869834 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa988042030 con 0x7fa9a0109850 2024-04-24T09:18:42.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.696+0000 7fa996a00640 1 --2- 172.21.15.90:0/1676869834 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa9840a3bb0 0x7fa9840a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:42.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.696+0000 7fa996a00640 1 -- 172.21.15.90:0/1676869834 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa9880f0470 con 0x7fa9a0109850 2024-04-24T09:18:42.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.696+0000 7fa99de00640 1 --2- 172.21.15.90:0/1676869834 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa9840a3bb0 0x7fa9840a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:42.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.697+0000 7fa99de00640 1 --2- 172.21.15.90:0/1676869834 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa9840a3bb0 0x7fa9840a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa9a01096b0 tx=0x7fa990005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:42.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.700+0000 7fa996a00640 1 -- 172.21.15.90:0/1676869834 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa9880bcc00 con 0x7fa9a0109850 2024-04-24T09:18:42.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.941+0000 7fa9a4e00640 1 -- 172.21.15.90:0/1676869834 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9a0062c00 con 0x7fa9a0109850 2024-04-24T09:18:42.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.942+0000 7fa996a00640 1 -- 172.21.15.90:0/1676869834 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4693 (secure 0 0 0) 0x7fa9880c0f00 con 0x7fa9a0109850 2024-04-24T09:18:42.942 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:18:42.942 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/227251251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":227251251},{"type":"v1","addr":"172.21.15.90:6837","nonce":227251251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/219377141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":219377141},{"type":"v1","addr":"172.21.15.114:6833","nonce":219377141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24446,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2396995634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2396995634},{"type":"v1","addr":"172.21.15.90:6835","nonce":2396995634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/779642417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":779642417},{"type":"v1","addr":"172.21.15.149:6833","nonce":779642417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/2565340800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":2565340800},{"type":"v1","addr":"172.21.15.114:6835","nonce":2565340800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-24T09:18:42.942 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 6 2024-04-24T09:18:42.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.944+0000 7fa9a4e00640 1 -- 172.21.15.90:0/1676869834 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa9840a3bb0 msgr2=0x7fa9840a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:42.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.944+0000 7fa9a4e00640 1 --2- 172.21.15.90:0/1676869834 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa9840a3bb0 0x7fa9840a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa9a01096b0 tx=0x7fa990005f90 comp rx=0 tx=0).stop 2024-04-24T09:18:42.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.945+0000 7fa9a4e00640 1 -- 172.21.15.90:0/1676869834 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa9a0109850 msgr2=0x7fa9a019eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:42.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.945+0000 7fa9a4e00640 1 --2- 172.21.15.90:0/1676869834 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa9a0109850 0x7fa9a019eec0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa988031440 tx=0x7fa988032ce0 comp rx=0 tx=0).stop 2024-04-24T09:18:42.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.945+0000 7fa9a4e00640 1 -- 172.21.15.90:0/1676869834 shutdown_connections 2024-04-24T09:18:42.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.945+0000 7fa9a4e00640 1 --2- 172.21.15.90:0/1676869834 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa9840a3bb0 0x7fa9840a6070 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:42.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.945+0000 7fa9a4e00640 1 --2- 172.21.15.90:0/1676869834 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9a010a210 0x7fa9a01a5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:42.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.945+0000 7fa9a4e00640 1 --2- 172.21.15.90:0/1676869834 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa9a0109850 0x7fa9a019eec0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:42.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.945+0000 7fa9a4e00640 1 --2- 172.21.15.90:0/1676869834 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa9a0108660 0x7fa9a019e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:42.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.945+0000 7fa9a4e00640 1 -- 172.21.15.90:0/1676869834 >> 172.21.15.90:0/1676869834 conn(0x7fa9a0077620 msgr2=0x7fa9a010cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:42.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.945+0000 7fa9a4e00640 1 -- 172.21.15.90:0/1676869834 shutdown_connections 2024-04-24T09:18:42.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:42.945+0000 7fa9a4e00640 1 -- 172.21.15.90:0/1676869834 wait complete. 2024-04-24T09:18:43.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:42 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/1676869834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:18:43.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:42 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/1676869834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:18:43.361 INFO:journalctl@ceph.mds.f.smithi149.stdout:Apr 24 09:18:43 smithi149 systemd[1]: Starting Ceph mds.f for 30ee9be2-021a-11ef-bc93-c7b262605968... 2024-04-24T09:18:43.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:42 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/1676869834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:18:43.611 INFO:journalctl@ceph.mds.f.smithi149.stdout:Apr 24 09:18:43 smithi149 podman[63711]: 2024-04-24 09:18:43.264441101 +0000 UTC m=+0.018426661 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:18:43.612 INFO:journalctl@ceph.mds.f.smithi149.stdout:Apr 24 09:18:43 smithi149 podman[63711]: 2024-04-24 09:18:43.387424121 +0000 UTC m=+0.141409669 container create 01b2fff4c5a1bd5f71a128b84b2a96348e243b4c9bae4a1e1f6a16aefbdf9a39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-f, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240424, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T09:18:44.007 INFO:journalctl@ceph.mds.f.smithi149.stdout:Apr 24 09:18:43 smithi149 podman[63711]: 2024-04-24 09:18:43.650374289 +0000 UTC m=+0.404359836 container init 01b2fff4c5a1bd5f71a128b84b2a96348e243b4c9bae4a1e1f6a16aefbdf9a39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-24T09:18:44.007 INFO:journalctl@ceph.mds.f.smithi149.stdout:Apr 24 09:18:43 smithi149 podman[63711]: 2024-04-24 09:18:43.653345337 +0000 UTC m=+0.407330892 container start 01b2fff4c5a1bd5f71a128b84b2a96348e243b4c9bae4a1e1f6a16aefbdf9a39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-f, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T09:18:44.007 INFO:journalctl@ceph.mds.f.smithi149.stdout:Apr 24 09:18:43 smithi149 ceph-mds[63738]: -- 172.21.15.149:0/311089921 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a90eca1a40 con 0x55a90ec74800 2024-04-24T09:18:44.007 INFO:journalctl@ceph.mds.f.smithi149.stdout:Apr 24 09:18:43 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-f[63729]: starting mds.f at 2024-04-24T09:18:44.007 INFO:journalctl@ceph.mds.f.smithi149.stdout:Apr 24 09:18:43 smithi149 bash[63711]: 01b2fff4c5a1bd5f71a128b84b2a96348e243b4c9bae4a1e1f6a16aefbdf9a39 2024-04-24T09:18:44.007 INFO:journalctl@ceph.mds.f.smithi149.stdout:Apr 24 09:18:43 smithi149 systemd[1]: Started Ceph mds.f for 30ee9be2-021a-11ef-bc93-c7b262605968. 2024-04-24T09:18:44.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:44 smithi090 ceph-mon[27825]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:44.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:44 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:44.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:44 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:44.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:44 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:44.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:44 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:18:44.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:44 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:18:44.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:44 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:18:44.358 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:44 smithi149 ceph-mon[32592]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:44.358 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:44 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:44.358 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:44 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:44.358 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:44 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:44.358 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:44 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:18:44.358 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:44 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:18:44.358 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:44 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:18:44.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:44 smithi114 ceph-mon[32501]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:44.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:44 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:44.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:44 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:44.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:44 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:44.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:44 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:18:44.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:44 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:18:44.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:44 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:18:44.964 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:18:45.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.178+0000 7fec52400640 1 -- 172.21.15.90:0/872865487 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec4c102900 msgr2=0x7fec4c104cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:45.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.178+0000 7fec52400640 1 --2- 172.21.15.90:0/872865487 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec4c102900 0x7fec4c104cf0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fec3c009ca0 tx=0x7fec3c02a8c0 comp rx=0 tx=0).stop 2024-04-24T09:18:45.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.178+0000 7fec52400640 1 -- 172.21.15.90:0/872865487 shutdown_connections 2024-04-24T09:18:45.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.178+0000 7fec52400640 1 --2- 172.21.15.90:0/872865487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fec4c107b60 0x7fec4c109fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:45.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.178+0000 7fec52400640 1 --2- 172.21.15.90:0/872865487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec4c105230 0x7fec4c107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:45.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.178+0000 7fec52400640 1 --2- 172.21.15.90:0/872865487 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec4c102900 0x7fec4c104cf0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:45.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.178+0000 7fec52400640 1 -- 172.21.15.90:0/872865487 >> 172.21.15.90:0/872865487 conn(0x7fec4c0fc2d0 msgr2=0x7fec4c0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:45.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.178+0000 7fec52400640 1 -- 172.21.15.90:0/872865487 shutdown_connections 2024-04-24T09:18:45.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.178+0000 7fec52400640 1 -- 172.21.15.90:0/872865487 wait complete. 2024-04-24T09:18:45.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.179+0000 7fec52400640 1 Processor -- start 2024-04-24T09:18:45.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.179+0000 7fec52400640 1 -- start start 2024-04-24T09:18:45.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.180+0000 7fec52400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec4c102900 0x7fec4c19a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:45.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.180+0000 7fec52400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fec4c105230 0x7fec4c19a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:45.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.180+0000 7fec52400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec4c107b60 0x7fec4c1a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:45.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.180+0000 7fec52400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c10db70 con 0x7fec4c107b60 2024-04-24T09:18:45.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.180+0000 7fec52400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c10d9f0 con 0x7fec4c102900 2024-04-24T09:18:45.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.180+0000 7fec52400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c10dcf0 con 0x7fec4c105230 2024-04-24T09:18:45.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.180+0000 7fec4be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fec4c105230 0x7fec4c19a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:45.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.180+0000 7fec51a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec4c107b60 0x7fec4c1a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:45.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.180+0000 7fec4be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fec4c105230 0x7fec4c19a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:38084/0 (socket says 172.21.15.90:38084) 2024-04-24T09:18:45.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.180+0000 7fec4be00640 1 -- 172.21.15.90:0/3778568296 learned_addr learned my addr 172.21.15.90:0/3778568296 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:18:45.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.180+0000 7fec51000640 1 --2- 172.21.15.90:0/3778568296 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec4c102900 0x7fec4c19a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:45.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.180+0000 7fec51a00640 1 -- 172.21.15.90:0/3778568296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fec4c105230 msgr2=0x7fec4c19a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:45.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.180+0000 7fec51a00640 1 --2- 172.21.15.90:0/3778568296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fec4c105230 0x7fec4c19a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:45.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.180+0000 7fec51a00640 1 -- 172.21.15.90:0/3778568296 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec4c102900 msgr2=0x7fec4c19a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:45.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.180+0000 7fec51a00640 1 --2- 172.21.15.90:0/3778568296 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec4c102900 0x7fec4c19a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:45.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.180+0000 7fec51a00640 1 -- 172.21.15.90:0/3778568296 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec4c1a1fe0 con 0x7fec4c107b60 2024-04-24T09:18:45.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.181+0000 7fec51a00640 1 --2- 172.21.15.90:0/3778568296 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec4c107b60 0x7fec4c1a1970 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fec4400e970 tx=0x7fec4400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:45.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.181+0000 7fec49600640 1 -- 172.21.15.90:0/3778568296 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4400ccc0 con 0x7fec4c107b60 2024-04-24T09:18:45.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.181+0000 7fec52400640 1 -- 172.21.15.90:0/3778568296 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec4c1a22d0 con 0x7fec4c107b60 2024-04-24T09:18:45.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.181+0000 7fec49600640 1 -- 172.21.15.90:0/3778568296 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec44005020 con 0x7fec4c107b60 2024-04-24T09:18:45.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.181+0000 7fec49600640 1 -- 172.21.15.90:0/3778568296 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec44004a00 con 0x7fec4c107b60 2024-04-24T09:18:45.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.181+0000 7fec52400640 1 -- 172.21.15.90:0/3778568296 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec4c1a2870 con 0x7fec4c107b60 2024-04-24T09:18:45.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.182+0000 7fec52400640 1 -- 172.21.15.90:0/3778568296 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec4c1096b0 con 0x7fec4c107b60 2024-04-24T09:18:45.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.184+0000 7fec49600640 1 -- 172.21.15.90:0/3778568296 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fec440051d0 con 0x7fec4c107b60 2024-04-24T09:18:45.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.186+0000 7fec49600640 1 --2- 172.21.15.90:0/3778568296 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fec300a3c30 0x7fec300a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:45.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.186+0000 7fec49600640 1 -- 172.21.15.90:0/3778568296 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fec440cfed0 con 0x7fec4c107b60 2024-04-24T09:18:45.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.186+0000 7fec51000640 1 --2- 172.21.15.90:0/3778568296 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fec300a3c30 0x7fec300a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:45.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.187+0000 7fec51000640 1 --2- 172.21.15.90:0/3778568296 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fec300a3c30 0x7fec300a60f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fec3c005e30 tx=0x7fec3c005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:45.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.190+0000 7fec49600640 1 -- 172.21.15.90:0/3778568296 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fec4409ccb0 con 0x7fec4c107b60 2024-04-24T09:18:45.264 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:45 smithi149 ceph-mon[32592]: Deploying daemon mds.c on smithi149 2024-04-24T09:18:45.264 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:45 smithi149 ceph-mon[32592]: mds.? [v2:172.21.15.149:6834/3157375624,v1:172.21.15.149:6835/3157375624] up:boot 2024-04-24T09:18:45.264 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:45 smithi149 ceph-mon[32592]: fsmap 6 up:standby 2024-04-24T09:18:45.264 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:45 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T09:18:45.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:45 smithi090 ceph-mon[27825]: Deploying daemon mds.c on smithi149 2024-04-24T09:18:45.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:45 smithi090 ceph-mon[27825]: mds.? [v2:172.21.15.149:6834/3157375624,v1:172.21.15.149:6835/3157375624] up:boot 2024-04-24T09:18:45.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:45 smithi090 ceph-mon[27825]: fsmap 6 up:standby 2024-04-24T09:18:45.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:45 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T09:18:45.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:45 smithi114 ceph-mon[32501]: Deploying daemon mds.c on smithi149 2024-04-24T09:18:45.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:45 smithi114 ceph-mon[32501]: mds.? [v2:172.21.15.149:6834/3157375624,v1:172.21.15.149:6835/3157375624] up:boot 2024-04-24T09:18:45.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:45 smithi114 ceph-mon[32501]: fsmap 6 up:standby 2024-04-24T09:18:45.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:45 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T09:18:45.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.434+0000 7fec52400640 1 -- 172.21.15.90:0/3778568296 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec4c062c00 con 0x7fec4c107b60 2024-04-24T09:18:45.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.435+0000 7fec49600640 1 -- 172.21.15.90:0/3778568296 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5521 (secure 0 0 0) 0x7fec4409c5b0 con 0x7fec4c107b60 2024-04-24T09:18:45.436 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:18:45.436 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/227251251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":227251251},{"type":"v1","addr":"172.21.15.90:6837","nonce":227251251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/219377141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":219377141},{"type":"v1","addr":"172.21.15.114:6833","nonce":219377141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24446,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2396995634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2396995634},{"type":"v1","addr":"172.21.15.90:6835","nonce":2396995634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/779642417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":779642417},{"type":"v1","addr":"172.21.15.149:6833","nonce":779642417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/2565340800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":2565340800},{"type":"v1","addr":"172.21.15.114:6835","nonce":2565340800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/3157375624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3157375624},{"type":"v1","addr":"172.21.15.149:6835","nonce":3157375624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-24T09:18:45.436 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 7 2024-04-24T09:18:45.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.438+0000 7fec52400640 1 -- 172.21.15.90:0/3778568296 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fec300a3c30 msgr2=0x7fec300a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:45.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.438+0000 7fec52400640 1 --2- 172.21.15.90:0/3778568296 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fec300a3c30 0x7fec300a60f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fec3c005e30 tx=0x7fec3c005f50 comp rx=0 tx=0).stop 2024-04-24T09:18:45.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.438+0000 7fec52400640 1 -- 172.21.15.90:0/3778568296 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec4c107b60 msgr2=0x7fec4c1a1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:45.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.438+0000 7fec52400640 1 --2- 172.21.15.90:0/3778568296 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec4c107b60 0x7fec4c1a1970 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fec4400e970 tx=0x7fec4400ee30 comp rx=0 tx=0).stop 2024-04-24T09:18:45.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.438+0000 7fec52400640 1 -- 172.21.15.90:0/3778568296 shutdown_connections 2024-04-24T09:18:45.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.438+0000 7fec52400640 1 --2- 172.21.15.90:0/3778568296 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fec300a3c30 0x7fec300a60f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:45.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.438+0000 7fec52400640 1 --2- 172.21.15.90:0/3778568296 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec4c107b60 0x7fec4c1a1970 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:45.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.438+0000 7fec52400640 1 --2- 172.21.15.90:0/3778568296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fec4c105230 0x7fec4c19a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:45.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.438+0000 7fec52400640 1 --2- 172.21.15.90:0/3778568296 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec4c102900 0x7fec4c19a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:45.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.438+0000 7fec52400640 1 -- 172.21.15.90:0/3778568296 >> 172.21.15.90:0/3778568296 conn(0x7fec4c0fc2d0 msgr2=0x7fec4c101f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:45.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.438+0000 7fec52400640 1 -- 172.21.15.90:0/3778568296 shutdown_connections 2024-04-24T09:18:45.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:45.438+0000 7fec52400640 1 -- 172.21.15.90:0/3778568296 wait complete. 2024-04-24T09:18:46.329 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:46 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/3778568296' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:18:46.330 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:46 smithi149 ceph-mon[32592]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:46.330 INFO:journalctl@ceph.mds.c.smithi149.stdout:Apr 24 09:18:46 smithi149 systemd[1]: Starting Ceph mds.c for 30ee9be2-021a-11ef-bc93-c7b262605968... 2024-04-24T09:18:46.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:46 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/3778568296' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:18:46.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:46 smithi090 ceph-mon[27825]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:46.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:46 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/3778568296' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:18:46.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:46 smithi114 ceph-mon[32501]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:46.601 INFO:journalctl@ceph.mds.c.smithi149.stdout:Apr 24 09:18:46 smithi149 podman[64028]: 2024-04-24 09:18:46.233561095 +0000 UTC m=+0.018209794 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:18:46.601 INFO:journalctl@ceph.mds.c.smithi149.stdout:Apr 24 09:18:46 smithi149 podman[64028]: 2024-04-24 09:18:46.43712207 +0000 UTC m=+0.221770756 container create b7df5ffb9489fe665be8064c33ca4a3fe4f3904f20b4df04667ad7a28fc1841c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-c, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-24T09:18:47.111 INFO:journalctl@ceph.mds.c.smithi149.stdout:Apr 24 09:18:46 smithi149 podman[64028]: 2024-04-24 09:18:46.729936233 +0000 UTC m=+0.514584919 container init b7df5ffb9489fe665be8064c33ca4a3fe4f3904f20b4df04667ad7a28fc1841c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-c, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T09:18:47.112 INFO:journalctl@ceph.mds.c.smithi149.stdout:Apr 24 09:18:46 smithi149 podman[64028]: 2024-04-24 09:18:46.732524436 +0000 UTC m=+0.517173128 container start b7df5ffb9489fe665be8064c33ca4a3fe4f3904f20b4df04667ad7a28fc1841c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-c, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=) 2024-04-24T09:18:47.112 INFO:journalctl@ceph.mds.c.smithi149.stdout:Apr 24 09:18:46 smithi149 ceph-mds[64047]: -- 172.21.15.149:0/971813338 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55769a6e1a40 con 0x55769a6b5000 2024-04-24T09:18:47.112 INFO:journalctl@ceph.mds.c.smithi149.stdout:Apr 24 09:18:46 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-c[64043]: starting mds.c at 2024-04-24T09:18:47.112 INFO:journalctl@ceph.mds.c.smithi149.stdout:Apr 24 09:18:46 smithi149 bash[64028]: b7df5ffb9489fe665be8064c33ca4a3fe4f3904f20b4df04667ad7a28fc1841c 2024-04-24T09:18:47.112 INFO:journalctl@ceph.mds.c.smithi149.stdout:Apr 24 09:18:46 smithi149 systemd[1]: Started Ceph mds.c for 30ee9be2-021a-11ef-bc93-c7b262605968. 2024-04-24T09:18:47.458 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:18:47.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.667+0000 7fb525a00640 1 -- 172.21.15.90:0/697018894 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5200749e0 msgr2=0x7fb520074e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:47.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.667+0000 7fb525a00640 1 --2- 172.21.15.90:0/697018894 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5200749e0 0x7fb520074e40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb508009860 tx=0x7fb50802a3d0 comp rx=0 tx=0).stop 2024-04-24T09:18:47.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.668+0000 7fb525a00640 1 -- 172.21.15.90:0/697018894 shutdown_connections 2024-04-24T09:18:47.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.668+0000 7fb525a00640 1 --2- 172.21.15.90:0/697018894 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb520109d20 0x7fb52010c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:47.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.668+0000 7fb525a00640 1 --2- 172.21.15.90:0/697018894 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5200749e0 0x7fb520074e40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:47.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.668+0000 7fb525a00640 1 --2- 172.21.15.90:0/697018894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb520075fc0 0x7fb520074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:47.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.668+0000 7fb525a00640 1 -- 172.21.15.90:0/697018894 >> 172.21.15.90:0/697018894 conn(0x7fb5200fd3e0 msgr2=0x7fb5200ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:47.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.668+0000 7fb525a00640 1 -- 172.21.15.90:0/697018894 shutdown_connections 2024-04-24T09:18:47.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.668+0000 7fb525a00640 1 -- 172.21.15.90:0/697018894 wait complete. 2024-04-24T09:18:47.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.669+0000 7fb525a00640 1 Processor -- start 2024-04-24T09:18:47.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.669+0000 7fb525a00640 1 -- start start 2024-04-24T09:18:47.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.669+0000 7fb525a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5200749e0 0x7fb52019a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:47.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.669+0000 7fb525a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb520075fc0 0x7fb52019a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:47.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.669+0000 7fb525a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb520109d20 0x7fb5201a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:47.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.669+0000 7fb525a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb52010fd10 con 0x7fb5200749e0 2024-04-24T09:18:47.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.669+0000 7fb525a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb52010fb90 con 0x7fb520109d20 2024-04-24T09:18:47.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.669+0000 7fb525a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb52010fe90 con 0x7fb520075fc0 2024-04-24T09:18:47.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.669+0000 7fb51fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5200749e0 0x7fb52019a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:47.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.669+0000 7fb51fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5200749e0 0x7fb52019a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50792/0 (socket says 172.21.15.90:50792) 2024-04-24T09:18:47.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.669+0000 7fb525000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb520109d20 0x7fb5201a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:47.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.669+0000 7fb51fe00640 1 -- 172.21.15.90:0/427068156 learned_addr learned my addr 172.21.15.90:0/427068156 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:18:47.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.670+0000 7fb517e00640 1 --2- 172.21.15.90:0/427068156 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb520075fc0 0x7fb52019a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:47.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.670+0000 7fb51fe00640 1 -- 172.21.15.90:0/427068156 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb520075fc0 msgr2=0x7fb52019a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:47.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.670+0000 7fb51fe00640 1 --2- 172.21.15.90:0/427068156 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb520075fc0 0x7fb52019a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:47.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.670+0000 7fb51fe00640 1 -- 172.21.15.90:0/427068156 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb520109d20 msgr2=0x7fb5201a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:47.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.670+0000 7fb51fe00640 1 --2- 172.21.15.90:0/427068156 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb520109d20 0x7fb5201a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:47.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.670+0000 7fb51fe00640 1 -- 172.21.15.90:0/427068156 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5201a1fd0 con 0x7fb5200749e0 2024-04-24T09:18:47.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.670+0000 7fb525000640 1 --2- 172.21.15.90:0/427068156 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb520109d20 0x7fb5201a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:18:47.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.670+0000 7fb51fe00640 1 --2- 172.21.15.90:0/427068156 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5200749e0 0x7fb52019a3b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb51000e9f0 tx=0x7fb51000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:47.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.670+0000 7fb51d600640 1 -- 172.21.15.90:0/427068156 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51000cd80 con 0x7fb5200749e0 2024-04-24T09:18:47.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.670+0000 7fb51d600640 1 -- 172.21.15.90:0/427068156 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb510005500 con 0x7fb5200749e0 2024-04-24T09:18:47.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.670+0000 7fb525a00640 1 -- 172.21.15.90:0/427068156 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5201a22c0 con 0x7fb5200749e0 2024-04-24T09:18:47.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.671+0000 7fb51d600640 1 -- 172.21.15.90:0/427068156 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb510010690 con 0x7fb5200749e0 2024-04-24T09:18:47.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.671+0000 7fb525a00640 1 -- 172.21.15.90:0/427068156 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5201a27c0 con 0x7fb5200749e0 2024-04-24T09:18:47.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.672+0000 7fb525a00640 1 -- 172.21.15.90:0/427068156 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb520108ae0 con 0x7fb5200749e0 2024-04-24T09:18:47.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.674+0000 7fb51d600640 1 -- 172.21.15.90:0/427068156 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb5100046c0 con 0x7fb5200749e0 2024-04-24T09:18:47.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.675+0000 7fb51d600640 1 --2- 172.21.15.90:0/427068156 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fb5040a3bb0 0x7fb5040a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:47.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.675+0000 7fb51d600640 1 -- 172.21.15.90:0/427068156 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fb5100d0400 con 0x7fb5200749e0 2024-04-24T09:18:47.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.675+0000 7fb517e00640 1 --2- 172.21.15.90:0/427068156 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fb5040a3bb0 0x7fb5040a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:47.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.676+0000 7fb517e00640 1 --2- 172.21.15.90:0/427068156 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fb5040a3bb0 0x7fb5040a6070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb52019b8c0 tx=0x7fb508005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:47.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.680+0000 7fb51d600640 1 -- 172.21.15.90:0/427068156 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb51009d1e0 con 0x7fb5200749e0 2024-04-24T09:18:47.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.926+0000 7fb525a00640 1 -- 172.21.15.90:0/427068156 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb52002d620 con 0x7fb5200749e0 2024-04-24T09:18:47.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.927+0000 7fb51d600640 1 -- 172.21.15.90:0/427068156 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6349 (secure 0 0 0) 0x7fb51009cae0 con 0x7fb5200749e0 2024-04-24T09:18:47.928 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:18:47.928 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/227251251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":227251251},{"type":"v1","addr":"172.21.15.90:6837","nonce":227251251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/219377141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":219377141},{"type":"v1","addr":"172.21.15.114:6833","nonce":219377141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24446,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2396995634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2396995634},{"type":"v1","addr":"172.21.15.90:6835","nonce":2396995634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/779642417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":779642417},{"type":"v1","addr":"172.21.15.149:6833","nonce":779642417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/2565340800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":2565340800},{"type":"v1","addr":"172.21.15.114:6835","nonce":2565340800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/3157375624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3157375624},{"type":"v1","addr":"172.21.15.149:6835","nonce":3157375624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/2250310587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2250310587},{"type":"v1","addr":"172.21.15.149:6837","nonce":2250310587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-24T09:18:47.929 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 8 2024-04-24T09:18:47.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.930+0000 7fb525a00640 1 -- 172.21.15.90:0/427068156 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fb5040a3bb0 msgr2=0x7fb5040a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:47.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.930+0000 7fb525a00640 1 --2- 172.21.15.90:0/427068156 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fb5040a3bb0 0x7fb5040a6070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb52019b8c0 tx=0x7fb508005d50 comp rx=0 tx=0).stop 2024-04-24T09:18:47.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.930+0000 7fb525a00640 1 -- 172.21.15.90:0/427068156 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5200749e0 msgr2=0x7fb52019a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:47.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.930+0000 7fb525a00640 1 --2- 172.21.15.90:0/427068156 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5200749e0 0x7fb52019a3b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb51000e9f0 tx=0x7fb51000edf0 comp rx=0 tx=0).stop 2024-04-24T09:18:47.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.930+0000 7fb525a00640 1 -- 172.21.15.90:0/427068156 shutdown_connections 2024-04-24T09:18:47.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.930+0000 7fb525a00640 1 --2- 172.21.15.90:0/427068156 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fb5040a3bb0 0x7fb5040a6070 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:47.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.930+0000 7fb525a00640 1 --2- 172.21.15.90:0/427068156 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb520109d20 0x7fb5201a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:47.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.930+0000 7fb525a00640 1 --2- 172.21.15.90:0/427068156 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb520075fc0 0x7fb52019a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:47.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.930+0000 7fb525a00640 1 --2- 172.21.15.90:0/427068156 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5200749e0 0x7fb52019a3b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:47.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.930+0000 7fb525a00640 1 -- 172.21.15.90:0/427068156 >> 172.21.15.90:0/427068156 conn(0x7fb5200fd3e0 msgr2=0x7fb520105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:47.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.931+0000 7fb525a00640 1 -- 172.21.15.90:0/427068156 shutdown_connections 2024-04-24T09:18:47.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:47.931+0000 7fb525a00640 1 -- 172.21.15.90:0/427068156 wait complete. 2024-04-24T09:18:48.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:47 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:48.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:47 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:48.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:47 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:48.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:47 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:18:48.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:47 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:18:48.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:47 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:18:48.177 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:47 smithi114 ceph-mon[32501]: Deploying daemon mds.h on smithi114 2024-04-24T09:18:48.177 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:47 smithi114 ceph-mon[32501]: mds.? [v2:172.21.15.149:6836/2250310587,v1:172.21.15.149:6837/2250310587] up:boot 2024-04-24T09:18:48.177 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:47 smithi114 ceph-mon[32501]: fsmap 7 up:standby 2024-04-24T09:18:48.177 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:47 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T09:18:48.177 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:47 smithi114 ceph-mon[32501]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:48.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:47 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:48.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:47 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:48.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:47 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:48.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:47 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:18:48.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:47 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:18:48.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:47 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:18:48.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:47 smithi090 ceph-mon[27825]: Deploying daemon mds.h on smithi114 2024-04-24T09:18:48.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:47 smithi090 ceph-mon[27825]: mds.? [v2:172.21.15.149:6836/2250310587,v1:172.21.15.149:6837/2250310587] up:boot 2024-04-24T09:18:48.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:47 smithi090 ceph-mon[27825]: fsmap 7 up:standby 2024-04-24T09:18:48.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:47 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T09:18:48.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:47 smithi090 ceph-mon[27825]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:48.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:47 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:48.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:47 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:48.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:47 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:48.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:47 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:18:48.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:47 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:18:48.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:47 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:18:48.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:47 smithi149 ceph-mon[32592]: Deploying daemon mds.h on smithi114 2024-04-24T09:18:48.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:47 smithi149 ceph-mon[32592]: mds.? [v2:172.21.15.149:6836/2250310587,v1:172.21.15.149:6837/2250310587] up:boot 2024-04-24T09:18:48.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:47 smithi149 ceph-mon[32592]: fsmap 7 up:standby 2024-04-24T09:18:48.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:47 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T09:18:48.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:47 smithi149 ceph-mon[32592]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:49.096 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:48 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/427068156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:18:49.347 INFO:journalctl@ceph.mds.h.smithi114.stdout:Apr 24 09:18:49 smithi114 podman[63269]: 2024-04-24 09:18:49.217205529 +0000 UTC m=+0.018640730 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:18:49.348 INFO:journalctl@ceph.mds.h.smithi114.stdout:Apr 24 09:18:49 smithi114 podman[63269]: 2024-04-24 09:18:49.348566962 +0000 UTC m=+0.150002166 container create 6c22bab096ce8940148f0dc73ce623c12f4ade40d35fe89ae67842365d427dbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-h, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-24T09:18:49.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:48 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/427068156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:18:49.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:48 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/427068156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:18:49.617 INFO:journalctl@ceph.mds.h.smithi114.stdout:Apr 24 09:18:49 smithi114 podman[63269]: 2024-04-24 09:18:49.592928724 +0000 UTC m=+0.394363930 container init 6c22bab096ce8940148f0dc73ce623c12f4ade40d35fe89ae67842365d427dbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-h, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True) 2024-04-24T09:18:49.617 INFO:journalctl@ceph.mds.h.smithi114.stdout:Apr 24 09:18:49 smithi114 podman[63269]: 2024-04-24 09:18:49.595428238 +0000 UTC m=+0.396863447 container start 6c22bab096ce8940148f0dc73ce623c12f4ade40d35fe89ae67842365d427dbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-h, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424) 2024-04-24T09:18:49.907 INFO:journalctl@ceph.mds.h.smithi114.stdout:Apr 24 09:18:49 smithi114 ceph-mds[63289]: -- 172.21.15.114:0/835755302 <== mon.2 v2:172.21.15.114:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b67e311a40 con 0x55b67e2e5000 2024-04-24T09:18:49.907 INFO:journalctl@ceph.mds.h.smithi114.stdout:Apr 24 09:18:49 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-h[63285]: starting mds.h at 2024-04-24T09:18:49.907 INFO:journalctl@ceph.mds.h.smithi114.stdout:Apr 24 09:18:49 smithi114 bash[63269]: 6c22bab096ce8940148f0dc73ce623c12f4ade40d35fe89ae67842365d427dbe 2024-04-24T09:18:49.907 INFO:journalctl@ceph.mds.h.smithi114.stdout:Apr 24 09:18:49 smithi114 systemd[1]: Started Ceph mds.h for 30ee9be2-021a-11ef-bc93-c7b262605968. 2024-04-24T09:18:49.951 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:18:50.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.162+0000 7f6be8a00640 1 -- 172.21.15.90:0/1594950250 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6be410a030 msgr2=0x7f6be41108c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:50.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.162+0000 7f6be8a00640 1 --2- 172.21.15.90:0/1594950250 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6be410a030 0x7f6be41108c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc009ca0 tx=0x7f6bdc02a8c0 comp rx=0 tx=0).stop 2024-04-24T09:18:50.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.163+0000 7f6be8a00640 1 -- 172.21.15.90:0/1594950250 shutdown_connections 2024-04-24T09:18:50.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.163+0000 7f6be8a00640 1 --2- 172.21.15.90:0/1594950250 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6be410a030 0x7f6be41108c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:50.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.163+0000 7f6be8a00640 1 --2- 172.21.15.90:0/1594950250 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6be4109670 0x7f6be4109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:50.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.163+0000 7f6be8a00640 1 --2- 172.21.15.90:0/1594950250 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6be4108480 0x7f6be41088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:50.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.163+0000 7f6be8a00640 1 -- 172.21.15.90:0/1594950250 >> 172.21.15.90:0/1594950250 conn(0x7f6be4077440 msgr2=0x7f6be4079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:50.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.163+0000 7f6be8a00640 1 -- 172.21.15.90:0/1594950250 shutdown_connections 2024-04-24T09:18:50.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.163+0000 7f6be8a00640 1 -- 172.21.15.90:0/1594950250 wait complete. 2024-04-24T09:18:50.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.164+0000 7f6be8a00640 1 Processor -- start 2024-04-24T09:18:50.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.164+0000 7f6be8a00640 1 -- start start 2024-04-24T09:18:50.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.164+0000 7f6be8a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6be4108480 0x7f6be419e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:50.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.164+0000 7f6be8a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6be4109670 0x7f6be419ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:50.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.164+0000 7f6be8a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6be410a030 0x7f6be41a5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:50.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.164+0000 7f6be8a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6be4114110 con 0x7f6be4108480 2024-04-24T09:18:50.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.164+0000 7f6be8a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6be4113f90 con 0x7f6be4109670 2024-04-24T09:18:50.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.164+0000 7f6be8a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6be4114290 con 0x7f6be410a030 2024-04-24T09:18:50.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.164+0000 7f6be2800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6be4108480 0x7f6be419e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:50.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.164+0000 7f6be3200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6be410a030 0x7f6be41a5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:50.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.164+0000 7f6be3200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6be410a030 0x7f6be41a5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:37022/0 (socket says 172.21.15.90:37022) 2024-04-24T09:18:50.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.164+0000 7f6be3200640 1 -- 172.21.15.90:0/4252544991 learned_addr learned my addr 172.21.15.90:0/4252544991 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:18:50.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.164+0000 7f6be2800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6be4108480 0x7f6be419e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50800/0 (socket says 172.21.15.90:50800) 2024-04-24T09:18:50.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.164+0000 7f6bd9e00640 1 --2- 172.21.15.90:0/4252544991 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6be4109670 0x7f6be419ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:50.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.165+0000 7f6be3200640 1 -- 172.21.15.90:0/4252544991 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6be4109670 msgr2=0x7f6be419ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:50.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.165+0000 7f6be3200640 1 --2- 172.21.15.90:0/4252544991 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6be4109670 0x7f6be419ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:50.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.165+0000 7f6be3200640 1 -- 172.21.15.90:0/4252544991 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6be4108480 msgr2=0x7f6be419e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:50.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.165+0000 7f6be3200640 1 --2- 172.21.15.90:0/4252544991 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6be4108480 0x7f6be419e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:50.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.165+0000 7f6be3200640 1 -- 172.21.15.90:0/4252544991 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6be41a6310 con 0x7f6be410a030 2024-04-24T09:18:50.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.165+0000 7f6be3200640 1 --2- 172.21.15.90:0/4252544991 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6be410a030 0x7f6be41a5ca0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc031ae0 tx=0x7f6bdc031ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:50.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.165+0000 7f6bdbe00640 1 -- 172.21.15.90:0/4252544991 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc003b70 con 0x7f6be410a030 2024-04-24T09:18:50.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.165+0000 7f6bdbe00640 1 -- 172.21.15.90:0/4252544991 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bdc003d10 con 0x7f6be410a030 2024-04-24T09:18:50.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.165+0000 7f6bdbe00640 1 -- 172.21.15.90:0/4252544991 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc02fc40 con 0x7f6be410a030 2024-04-24T09:18:50.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.165+0000 7f6be8a00640 1 -- 172.21.15.90:0/4252544991 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6be41a65a0 con 0x7f6be410a030 2024-04-24T09:18:50.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.166+0000 7f6be8a00640 1 -- 172.21.15.90:0/4252544991 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6be41a6af0 con 0x7f6be410a030 2024-04-24T09:18:50.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.167+0000 7f6be8a00640 1 -- 172.21.15.90:0/4252544991 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6be410f7a0 con 0x7f6be410a030 2024-04-24T09:18:50.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.169+0000 7f6bdbe00640 1 -- 172.21.15.90:0/4252544991 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6bdc004050 con 0x7f6be410a030 2024-04-24T09:18:50.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.170+0000 7f6bdbe00640 1 --2- 172.21.15.90:0/4252544991 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f6bc80a3bb0 0x7f6bc80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:50.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.170+0000 7f6bdbe00640 1 -- 172.21.15.90:0/4252544991 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f6bdc0f1090 con 0x7f6be410a030 2024-04-24T09:18:50.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.170+0000 7f6be2800640 1 --2- 172.21.15.90:0/4252544991 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f6bc80a3bb0 0x7f6bc80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:50.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.171+0000 7f6be2800640 1 --2- 172.21.15.90:0/4252544991 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f6bc80a3bb0 0x7f6bc80a6070 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6be41094d0 tx=0x7f6bd4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:50.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.174+0000 7f6bdbe00640 1 -- 172.21.15.90:0/4252544991 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6bdc0bde70 con 0x7f6be410a030 2024-04-24T09:18:50.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:49 smithi114 ceph-mon[32501]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:50.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:49 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:50.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:49 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:50.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:49 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:50.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:49 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:18:50.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:49 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:18:50.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:49 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:18:50.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:49 smithi090 ceph-mon[27825]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:50.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:49 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:50.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:49 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:50.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:49 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:50.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:49 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:18:50.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:49 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:18:50.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:49 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:18:50.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:49 smithi149 ceph-mon[32592]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:50.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:49 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:50.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:49 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:50.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:49 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:50.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:49 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:18:50.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:49 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:18:50.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:49 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:18:50.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.416+0000 7f6be8a00640 1 -- 172.21.15.90:0/4252544991 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6be4062c00 con 0x7f6be410a030 2024-04-24T09:18:50.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.417+0000 7f6bdbe00640 1 -- 172.21.15.90:0/4252544991 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7174 (secure 0 0 0) 0x7f6bdc0bd770 con 0x7f6be410a030 2024-04-24T09:18:50.417 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:18:50.418 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/227251251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":227251251},{"type":"v1","addr":"172.21.15.90:6837","nonce":227251251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/219377141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":219377141},{"type":"v1","addr":"172.21.15.114:6833","nonce":219377141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24446,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2396995634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2396995634},{"type":"v1","addr":"172.21.15.90:6835","nonce":2396995634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/779642417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":779642417},{"type":"v1","addr":"172.21.15.149:6833","nonce":779642417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/2565340800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":2565340800},{"type":"v1","addr":"172.21.15.114:6835","nonce":2565340800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/875324280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":875324280},{"type":"v1","addr":"172.21.15.114:6837","nonce":875324280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/3157375624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3157375624},{"type":"v1","addr":"172.21.15.149:6835","nonce":3157375624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/2250310587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2250310587},{"type":"v1","addr":"172.21.15.149:6837","nonce":2250310587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-24T09:18:50.418 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 9 2024-04-24T09:18:50.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.420+0000 7f6be8a00640 1 -- 172.21.15.90:0/4252544991 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f6bc80a3bb0 msgr2=0x7f6bc80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:50.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.420+0000 7f6be8a00640 1 --2- 172.21.15.90:0/4252544991 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f6bc80a3bb0 0x7f6bc80a6070 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6be41094d0 tx=0x7f6bd4005f90 comp rx=0 tx=0).stop 2024-04-24T09:18:50.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.421+0000 7f6be8a00640 1 -- 172.21.15.90:0/4252544991 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6be410a030 msgr2=0x7f6be41a5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:50.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.421+0000 7f6be8a00640 1 --2- 172.21.15.90:0/4252544991 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6be410a030 0x7f6be41a5ca0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc031ae0 tx=0x7f6bdc031ee0 comp rx=0 tx=0).stop 2024-04-24T09:18:50.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.421+0000 7f6be8a00640 1 -- 172.21.15.90:0/4252544991 shutdown_connections 2024-04-24T09:18:50.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.421+0000 7f6be8a00640 1 --2- 172.21.15.90:0/4252544991 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f6bc80a3bb0 0x7f6bc80a6070 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:50.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.421+0000 7f6be8a00640 1 --2- 172.21.15.90:0/4252544991 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6be410a030 0x7f6be41a5ca0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:50.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.421+0000 7f6be8a00640 1 --2- 172.21.15.90:0/4252544991 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6be4109670 0x7f6be419ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:50.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.421+0000 7f6be8a00640 1 --2- 172.21.15.90:0/4252544991 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6be4108480 0x7f6be419e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:50.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.421+0000 7f6be8a00640 1 -- 172.21.15.90:0/4252544991 >> 172.21.15.90:0/4252544991 conn(0x7f6be4077440 msgr2=0x7f6be410c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:50.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.421+0000 7f6be8a00640 1 -- 172.21.15.90:0/4252544991 shutdown_connections 2024-04-24T09:18:50.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:50.421+0000 7f6be8a00640 1 -- 172.21.15.90:0/4252544991 wait complete. 2024-04-24T09:18:51.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:50 smithi149 ceph-mon[32592]: Deploying daemon mds.i on smithi149 2024-04-24T09:18:51.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:50 smithi149 ceph-mon[32592]: mds.? [v2:172.21.15.114:6836/875324280,v1:172.21.15.114:6837/875324280] up:boot 2024-04-24T09:18:51.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:50 smithi149 ceph-mon[32592]: fsmap 8 up:standby 2024-04-24T09:18:51.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:50 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T09:18:51.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:50 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/4252544991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:18:51.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:50 smithi114 ceph-mon[32501]: Deploying daemon mds.i on smithi149 2024-04-24T09:18:51.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:50 smithi114 ceph-mon[32501]: mds.? [v2:172.21.15.114:6836/875324280,v1:172.21.15.114:6837/875324280] up:boot 2024-04-24T09:18:51.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:50 smithi114 ceph-mon[32501]: fsmap 8 up:standby 2024-04-24T09:18:51.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:50 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T09:18:51.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:50 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/4252544991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:18:51.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:50 smithi090 ceph-mon[27825]: Deploying daemon mds.i on smithi149 2024-04-24T09:18:51.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:50 smithi090 ceph-mon[27825]: mds.? [v2:172.21.15.114:6836/875324280,v1:172.21.15.114:6837/875324280] up:boot 2024-04-24T09:18:51.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:50 smithi090 ceph-mon[27825]: fsmap 8 up:standby 2024-04-24T09:18:51.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:50 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T09:18:51.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:50 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/4252544991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:18:52.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:51 smithi149 ceph-mon[32592]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:52.112 INFO:journalctl@ceph.mds.i.smithi149.stdout:Apr 24 09:18:51 smithi149 systemd[1]: Starting Ceph mds.i for 30ee9be2-021a-11ef-bc93-c7b262605968... 2024-04-24T09:18:52.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:51 smithi114 ceph-mon[32501]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:52.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:51 smithi090 ceph-mon[27825]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:52.424 INFO:journalctl@ceph.mds.i.smithi149.stdout:Apr 24 09:18:52 smithi149 podman[64332]: 2024-04-24 09:18:52.065216362 +0000 UTC m=+0.017603667 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:18:52.424 INFO:journalctl@ceph.mds.i.smithi149.stdout:Apr 24 09:18:52 smithi149 podman[64332]: 2024-04-24 09:18:52.182039152 +0000 UTC m=+0.134426439 container create 35e767d0635deb1489a42c375b43025d9f0b3b5e93ccb56f237120dcdf16885a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-i, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T09:18:52.425 INFO:journalctl@ceph.mds.i.smithi149.stdout:Apr 24 09:18:52 smithi149 podman[64332]: 2024-04-24 09:18:52.424821261 +0000 UTC m=+0.377208548 container init 35e767d0635deb1489a42c375b43025d9f0b3b5e93ccb56f237120dcdf16885a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True, GIT_CLEAN=True) 2024-04-24T09:18:52.440 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:18:52.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.652+0000 7fafdb200640 1 -- 172.21.15.90:0/3206387643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafdc074a00 msgr2=0x7fafdc074e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:52.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.652+0000 7fafdb200640 1 --2- 172.21.15.90:0/3206387643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafdc074a00 0x7fafdc074e60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fafc8009900 tx=0x7fafc802a480 comp rx=0 tx=0).stop 2024-04-24T09:18:52.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.652+0000 7fafdb200640 1 -- 172.21.15.90:0/3206387643 shutdown_connections 2024-04-24T09:18:52.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.652+0000 7fafdb200640 1 --2- 172.21.15.90:0/3206387643 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fafdc109d30 0x7fafdc10c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:52.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.652+0000 7fafdb200640 1 --2- 172.21.15.90:0/3206387643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafdc074a00 0x7fafdc074e60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:52.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.652+0000 7fafdb200640 1 --2- 172.21.15.90:0/3206387643 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fafdc075fe0 0x7fafdc074430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:52.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.652+0000 7fafdb200640 1 -- 172.21.15.90:0/3206387643 >> 172.21.15.90:0/3206387643 conn(0x7fafdc0fd410 msgr2=0x7fafdc0ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:52.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.652+0000 7fafdb200640 1 -- 172.21.15.90:0/3206387643 shutdown_connections 2024-04-24T09:18:52.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.653+0000 7fafdb200640 1 -- 172.21.15.90:0/3206387643 wait complete. 2024-04-24T09:18:52.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.653+0000 7fafdb200640 1 Processor -- start 2024-04-24T09:18:52.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.654+0000 7fafdb200640 1 -- start start 2024-04-24T09:18:52.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.654+0000 7fafdb200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafdc074a00 0x7fafdc19a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:52.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.654+0000 7fafdb200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fafdc075fe0 0x7fafdc19a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:52.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.654+0000 7fafdb200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fafdc109d30 0x7fafdc1a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:52.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.654+0000 7fafdb200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fafdc10fd80 con 0x7fafdc075fe0 2024-04-24T09:18:52.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.654+0000 7fafdb200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fafdc10fc00 con 0x7fafdc109d30 2024-04-24T09:18:52.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.654+0000 7fafdb200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fafdc10ff00 con 0x7fafdc074a00 2024-04-24T09:18:52.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.654+0000 7fafd9e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafdc074a00 0x7fafdc19a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:52.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.654+0000 7fafd9400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fafdc075fe0 0x7fafdc19a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:52.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.654+0000 7fafd9e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafdc074a00 0x7fafdc19a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:37044/0 (socket says 172.21.15.90:37044) 2024-04-24T09:18:52.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.654+0000 7fafd9e00640 1 -- 172.21.15.90:0/1058094551 learned_addr learned my addr 172.21.15.90:0/1058094551 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:18:52.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.654+0000 7fafda800640 1 --2- 172.21.15.90:0/1058094551 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fafdc109d30 0x7fafdc1a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:52.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.655+0000 7fafd9400640 1 -- 172.21.15.90:0/1058094551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafdc074a00 msgr2=0x7fafdc19a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:52.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.655+0000 7fafd9400640 1 --2- 172.21.15.90:0/1058094551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafdc074a00 0x7fafdc19a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:52.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.655+0000 7fafd9400640 1 -- 172.21.15.90:0/1058094551 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fafdc109d30 msgr2=0x7fafdc1a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:52.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.655+0000 7fafd9400640 1 --2- 172.21.15.90:0/1058094551 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fafdc109d30 0x7fafdc1a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:52.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.655+0000 7fafd9400640 1 -- 172.21.15.90:0/1058094551 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafdc1a1fd0 con 0x7fafdc075fe0 2024-04-24T09:18:52.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.655+0000 7fafd9e00640 1 --2- 172.21.15.90:0/1058094551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafdc074a00 0x7fafdc19a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:18:52.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.655+0000 7fafd9400640 1 --2- 172.21.15.90:0/1058094551 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fafdc075fe0 0x7fafdc19a8f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fafc802a9a0 tx=0x7fafc8032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:52.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.657+0000 7fafd2a00640 1 -- 172.21.15.90:0/1058094551 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc803f030 con 0x7fafdc075fe0 2024-04-24T09:18:52.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.658+0000 7fafdb200640 1 -- 172.21.15.90:0/1058094551 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafdc1a2260 con 0x7fafdc075fe0 2024-04-24T09:18:52.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.658+0000 7fafd2a00640 1 -- 172.21.15.90:0/1058094551 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafc8003ce0 con 0x7fafdc075fe0 2024-04-24T09:18:52.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.658+0000 7fafd2a00640 1 -- 172.21.15.90:0/1058094551 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc8035070 con 0x7fafdc075fe0 2024-04-24T09:18:52.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.658+0000 7fafdb200640 1 -- 172.21.15.90:0/1058094551 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafdc1a2770 con 0x7fafdc075fe0 2024-04-24T09:18:52.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.659+0000 7fafdb200640 1 -- 172.21.15.90:0/1058094551 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafdc108af0 con 0x7fafdc075fe0 2024-04-24T09:18:52.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.661+0000 7fafd2a00640 1 -- 172.21.15.90:0/1058094551 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fafc8042030 con 0x7fafdc075fe0 2024-04-24T09:18:52.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.663+0000 7fafd2a00640 1 --2- 172.21.15.90:0/1058094551 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fafc00a3c30 0x7fafc00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:52.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.663+0000 7fafd2a00640 1 -- 172.21.15.90:0/1058094551 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fafc80f0080 con 0x7fafdc075fe0 2024-04-24T09:18:52.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.664+0000 7fafd9e00640 1 --2- 172.21.15.90:0/1058094551 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fafc00a3c30 0x7fafc00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:52.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.664+0000 7fafd9e00640 1 --2- 172.21.15.90:0/1058094551 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fafc00a3c30 0x7fafc00a60f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fafcc011550 tx=0x7fafcc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:52.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.668+0000 7fafd2a00640 1 -- 172.21.15.90:0/1058094551 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fafc80bc810 con 0x7fafdc075fe0 2024-04-24T09:18:52.862 INFO:journalctl@ceph.mds.i.smithi149.stdout:Apr 24 09:18:52 smithi149 podman[64332]: 2024-04-24 09:18:52.428109829 +0000 UTC m=+0.380497119 container start 35e767d0635deb1489a42c375b43025d9f0b3b5e93ccb56f237120dcdf16885a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-i, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-24T09:18:52.862 INFO:journalctl@ceph.mds.i.smithi149.stdout:Apr 24 09:18:52 smithi149 ceph-mds[64351]: -- 172.21.15.149:0/625315684 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5625fb383a40 con 0x5625fb356c00 2024-04-24T09:18:52.862 INFO:journalctl@ceph.mds.i.smithi149.stdout:Apr 24 09:18:52 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-i[64347]: starting mds.i at 2024-04-24T09:18:52.862 INFO:journalctl@ceph.mds.i.smithi149.stdout:Apr 24 09:18:52 smithi149 bash[64332]: 35e767d0635deb1489a42c375b43025d9f0b3b5e93ccb56f237120dcdf16885a 2024-04-24T09:18:52.862 INFO:journalctl@ceph.mds.i.smithi149.stdout:Apr 24 09:18:52 smithi149 systemd[1]: Started Ceph mds.i for 30ee9be2-021a-11ef-bc93-c7b262605968. 2024-04-24T09:18:52.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.912+0000 7fafdb200640 1 -- 172.21.15.90:0/1058094551 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fafdc02d620 con 0x7fafdc075fe0 2024-04-24T09:18:52.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.913+0000 7fafd2a00640 1 -- 172.21.15.90:0/1058094551 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7174 (secure 0 0 0) 0x7fafc80c0b10 con 0x7fafdc075fe0 2024-04-24T09:18:52.913 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:18:52.913 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/227251251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":227251251},{"type":"v1","addr":"172.21.15.90:6837","nonce":227251251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/219377141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":219377141},{"type":"v1","addr":"172.21.15.114:6833","nonce":219377141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24446,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2396995634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2396995634},{"type":"v1","addr":"172.21.15.90:6835","nonce":2396995634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/779642417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":779642417},{"type":"v1","addr":"172.21.15.149:6833","nonce":779642417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/2565340800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":2565340800},{"type":"v1","addr":"172.21.15.114:6835","nonce":2565340800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/875324280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":875324280},{"type":"v1","addr":"172.21.15.114:6837","nonce":875324280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/3157375624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3157375624},{"type":"v1","addr":"172.21.15.149:6835","nonce":3157375624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/2250310587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2250310587},{"type":"v1","addr":"172.21.15.149:6837","nonce":2250310587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-24T09:18:52.914 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 9 2024-04-24T09:18:52.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.916+0000 7fafdb200640 1 -- 172.21.15.90:0/1058094551 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fafc00a3c30 msgr2=0x7fafc00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:52.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.916+0000 7fafdb200640 1 --2- 172.21.15.90:0/1058094551 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fafc00a3c30 0x7fafc00a60f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fafcc011550 tx=0x7fafcc005f90 comp rx=0 tx=0).stop 2024-04-24T09:18:52.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.916+0000 7fafdb200640 1 -- 172.21.15.90:0/1058094551 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fafdc075fe0 msgr2=0x7fafdc19a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:52.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.916+0000 7fafdb200640 1 --2- 172.21.15.90:0/1058094551 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fafdc075fe0 0x7fafdc19a8f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fafc802a9a0 tx=0x7fafc8032ce0 comp rx=0 tx=0).stop 2024-04-24T09:18:52.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.916+0000 7fafdb200640 1 -- 172.21.15.90:0/1058094551 shutdown_connections 2024-04-24T09:18:52.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.916+0000 7fafdb200640 1 --2- 172.21.15.90:0/1058094551 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fafc00a3c30 0x7fafc00a60f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:52.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.916+0000 7fafdb200640 1 --2- 172.21.15.90:0/1058094551 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fafdc109d30 0x7fafdc1a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:52.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.916+0000 7fafdb200640 1 --2- 172.21.15.90:0/1058094551 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fafdc075fe0 0x7fafdc19a8f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:52.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.916+0000 7fafdb200640 1 --2- 172.21.15.90:0/1058094551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafdc074a00 0x7fafdc19a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:52.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.916+0000 7fafdb200640 1 -- 172.21.15.90:0/1058094551 >> 172.21.15.90:0/1058094551 conn(0x7fafdc0fd410 msgr2=0x7fafdc105cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:52.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.916+0000 7fafdb200640 1 -- 172.21.15.90:0/1058094551 shutdown_connections 2024-04-24T09:18:52.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:52.916+0000 7fafdb200640 1 -- 172.21.15.90:0/1058094551 wait complete. 2024-04-24T09:18:53.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:53 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:53.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:53 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:53.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:53 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:53.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:53 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:18:53.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:53 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:18:53.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:53 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:18:53.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:53 smithi090 ceph-mon[27825]: Deploying daemon mds.g on smithi090 2024-04-24T09:18:53.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:53 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/1058094551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:18:53.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:53 smithi090 ceph-mon[27825]: mds.? [v2:172.21.15.149:6838/652734263,v1:172.21.15.149:6839/652734263] up:boot 2024-04-24T09:18:53.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:53 smithi090 ceph-mon[27825]: fsmap 9 up:standby 2024-04-24T09:18:53.854 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:53 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T09:18:53.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:53 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:53.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:53 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:53.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:53 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:53.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:53 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:18:53.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:53 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:18:53.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:53 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:18:53.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:53 smithi149 ceph-mon[32592]: Deploying daemon mds.g on smithi090 2024-04-24T09:18:53.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:53 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/1058094551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:18:53.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:53 smithi149 ceph-mon[32592]: mds.? [v2:172.21.15.149:6838/652734263,v1:172.21.15.149:6839/652734263] up:boot 2024-04-24T09:18:53.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:53 smithi149 ceph-mon[32592]: fsmap 9 up:standby 2024-04-24T09:18:53.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:53 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T09:18:53.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:53 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:53.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:53 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:53.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:53 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:53.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:53 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:18:53.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:53 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:18:53.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:53 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:18:53.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:53 smithi114 ceph-mon[32501]: Deploying daemon mds.g on smithi090 2024-04-24T09:18:53.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:53 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/1058094551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:18:53.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:53 smithi114 ceph-mon[32501]: mds.? [v2:172.21.15.149:6838/652734263,v1:172.21.15.149:6839/652734263] up:boot 2024-04-24T09:18:53.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:53 smithi114 ceph-mon[32501]: fsmap 9 up:standby 2024-04-24T09:18:53.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:53 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T09:18:54.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:54 smithi090 ceph-mon[27825]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:54.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:54 smithi149 ceph-mon[32592]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:54.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:54 smithi114 ceph-mon[32501]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:54.936 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:18:55.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.155+0000 7feb68200640 1 -- 172.21.15.90:0/2285768969 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb60101210 msgr2=0x7feb60101670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:55.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.155+0000 7feb68200640 1 --2- 172.21.15.90:0/2285768969 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb60101210 0x7feb60101670 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7feb4c009860 tx=0x7feb4c02a340 comp rx=0 tx=0).stop 2024-04-24T09:18:55.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.155+0000 7feb68200640 1 -- 172.21.15.90:0/2285768969 shutdown_connections 2024-04-24T09:18:55.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.155+0000 7feb68200640 1 --2- 172.21.15.90:0/2285768969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb60101bb0 0x7feb60102030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:55.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.155+0000 7feb68200640 1 --2- 172.21.15.90:0/2285768969 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb60101210 0x7feb60101670 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:55.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.155+0000 7feb68200640 1 --2- 172.21.15.90:0/2285768969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb601008b0 0x7feb60100cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:55.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.155+0000 7feb68200640 1 -- 172.21.15.90:0/2285768969 >> 172.21.15.90:0/2285768969 conn(0x7feb600fc4f0 msgr2=0x7feb600fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:55.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.155+0000 7feb68200640 1 -- 172.21.15.90:0/2285768969 shutdown_connections 2024-04-24T09:18:55.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.155+0000 7feb68200640 1 -- 172.21.15.90:0/2285768969 wait complete. 2024-04-24T09:18:55.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.156+0000 7feb68200640 1 Processor -- start 2024-04-24T09:18:55.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.157+0000 7feb68200640 1 -- start start 2024-04-24T09:18:55.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.157+0000 7feb68200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb601008b0 0x7feb6019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:55.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.157+0000 7feb68200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb60101210 0x7feb6019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:55.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.157+0000 7feb68200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb60101bb0 0x7feb601a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:55.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.157+0000 7feb68200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7feb6010fee0 con 0x7feb601008b0 2024-04-24T09:18:55.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.157+0000 7feb68200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7feb6010fd60 con 0x7feb60101bb0 2024-04-24T09:18:55.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.157+0000 7feb68200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7feb60110060 con 0x7feb60101210 2024-04-24T09:18:55.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.157+0000 7feb65800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb601008b0 0x7feb6019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:55.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.157+0000 7feb65800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb601008b0 0x7feb6019a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50830/0 (socket says 172.21.15.90:50830) 2024-04-24T09:18:55.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.157+0000 7feb65800640 1 -- 172.21.15.90:0/1660956793 learned_addr learned my addr 172.21.15.90:0/1660956793 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:18:55.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.157+0000 7feb64e00640 1 --2- 172.21.15.90:0/1660956793 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb60101210 0x7feb6019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:55.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.157+0000 7feb66200640 1 --2- 172.21.15.90:0/1660956793 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb60101bb0 0x7feb601a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:55.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.158+0000 7feb65800640 1 -- 172.21.15.90:0/1660956793 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb60101210 msgr2=0x7feb6019aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:55.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.158+0000 7feb65800640 1 --2- 172.21.15.90:0/1660956793 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb60101210 0x7feb6019aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:55.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.158+0000 7feb65800640 1 -- 172.21.15.90:0/1660956793 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb60101bb0 msgr2=0x7feb601a1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:55.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.158+0000 7feb65800640 1 --2- 172.21.15.90:0/1660956793 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb60101bb0 0x7feb601a1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:55.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.158+0000 7feb65800640 1 -- 172.21.15.90:0/1660956793 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb601a2140 con 0x7feb601008b0 2024-04-24T09:18:55.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.158+0000 7feb66200640 1 --2- 172.21.15.90:0/1660956793 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb60101bb0 0x7feb601a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:18:55.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.158+0000 7feb65800640 1 --2- 172.21.15.90:0/1660956793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb601008b0 0x7feb6019a520 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7feb5400e9c0 tx=0x7feb5400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:55.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.158+0000 7feb5e000640 1 -- 172.21.15.90:0/1660956793 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5400ccc0 con 0x7feb601008b0 2024-04-24T09:18:55.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.158+0000 7feb5e000640 1 -- 172.21.15.90:0/1660956793 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb54005500 con 0x7feb601008b0 2024-04-24T09:18:55.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.158+0000 7feb68200640 1 -- 172.21.15.90:0/1660956793 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb601a2430 con 0x7feb601008b0 2024-04-24T09:18:55.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.158+0000 7feb5e000640 1 -- 172.21.15.90:0/1660956793 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb54010430 con 0x7feb601008b0 2024-04-24T09:18:55.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.158+0000 7feb68200640 1 -- 172.21.15.90:0/1660956793 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb601a2950 con 0x7feb601008b0 2024-04-24T09:18:55.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.160+0000 7feb68200640 1 -- 172.21.15.90:0/1660956793 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb6010b7b0 con 0x7feb601008b0 2024-04-24T09:18:55.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.162+0000 7feb5e000640 1 -- 172.21.15.90:0/1660956793 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7feb540046c0 con 0x7feb601008b0 2024-04-24T09:18:55.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.163+0000 7feb5e000640 1 --2- 172.21.15.90:0/1660956793 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7feb480a3bb0 0x7feb480a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:55.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.164+0000 7feb5e000640 1 -- 172.21.15.90:0/1660956793 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7feb540d0350 con 0x7feb601008b0 2024-04-24T09:18:55.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.164+0000 7feb64e00640 1 --2- 172.21.15.90:0/1660956793 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7feb480a3bb0 0x7feb480a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:55.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.165+0000 7feb64e00640 1 --2- 172.21.15.90:0/1660956793 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7feb480a3bb0 0x7feb480a6070 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7feb4c009830 tx=0x7feb4c030550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:55.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.168+0000 7feb5e000640 1 -- 172.21.15.90:0/1660956793 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feb5409d130 con 0x7feb601008b0 2024-04-24T09:18:55.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.516+0000 7feb68200640 1 -- 172.21.15.90:0/1660956793 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb6002d620 con 0x7feb601008b0 2024-04-24T09:18:55.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.517+0000 7feb5e000640 1 -- 172.21.15.90:0/1660956793 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8001 (secure 0 0 0) 0x7feb5409ca30 con 0x7feb601008b0 2024-04-24T09:18:55.517 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:18:55.517 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/227251251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":227251251},{"type":"v1","addr":"172.21.15.90:6837","nonce":227251251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/219377141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":219377141},{"type":"v1","addr":"172.21.15.114:6833","nonce":219377141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24446,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2396995634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2396995634},{"type":"v1","addr":"172.21.15.90:6835","nonce":2396995634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/779642417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":779642417},{"type":"v1","addr":"172.21.15.149:6833","nonce":779642417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/2565340800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":2565340800},{"type":"v1","addr":"172.21.15.114:6835","nonce":2565340800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/875324280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":875324280},{"type":"v1","addr":"172.21.15.114:6837","nonce":875324280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/3157375624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3157375624},{"type":"v1","addr":"172.21.15.149:6835","nonce":3157375624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/2250310587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2250310587},{"type":"v1","addr":"172.21.15.149:6837","nonce":2250310587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6839/652734263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":652734263},{"type":"v1","addr":"172.21.15.149:6839","nonce":652734263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-24T09:18:55.517 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 10 2024-04-24T09:18:55.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.519+0000 7feb68200640 1 -- 172.21.15.90:0/1660956793 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7feb480a3bb0 msgr2=0x7feb480a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:55.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.519+0000 7feb68200640 1 --2- 172.21.15.90:0/1660956793 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7feb480a3bb0 0x7feb480a6070 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7feb4c009830 tx=0x7feb4c030550 comp rx=0 tx=0).stop 2024-04-24T09:18:55.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.519+0000 7feb68200640 1 -- 172.21.15.90:0/1660956793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb601008b0 msgr2=0x7feb6019a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:55.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.519+0000 7feb68200640 1 --2- 172.21.15.90:0/1660956793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb601008b0 0x7feb6019a520 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7feb5400e9c0 tx=0x7feb5400ee80 comp rx=0 tx=0).stop 2024-04-24T09:18:55.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.519+0000 7feb68200640 1 -- 172.21.15.90:0/1660956793 shutdown_connections 2024-04-24T09:18:55.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.519+0000 7feb68200640 1 --2- 172.21.15.90:0/1660956793 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7feb480a3bb0 0x7feb480a6070 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:55.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.519+0000 7feb68200640 1 --2- 172.21.15.90:0/1660956793 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb60101bb0 0x7feb601a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:55.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.519+0000 7feb68200640 1 --2- 172.21.15.90:0/1660956793 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb60101210 0x7feb6019aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:55.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.519+0000 7feb68200640 1 --2- 172.21.15.90:0/1660956793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb601008b0 0x7feb6019a520 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:55.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.519+0000 7feb68200640 1 -- 172.21.15.90:0/1660956793 >> 172.21.15.90:0/1660956793 conn(0x7feb600fc4f0 msgr2=0x7feb600fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:55.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.519+0000 7feb68200640 1 -- 172.21.15.90:0/1660956793 shutdown_connections 2024-04-24T09:18:55.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:55.519+0000 7feb68200640 1 -- 172.21.15.90:0/1660956793 wait complete. 2024-04-24T09:18:55.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:55 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/1660956793' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:18:55.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:55 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/1660956793' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:18:55.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:55 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/1660956793' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:18:56.318 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 24 09:18:55 smithi090 systemd[1]: Starting Ceph mds.g for 30ee9be2-021a-11ef-bc93-c7b262605968... 2024-04-24T09:18:56.603 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 24 09:18:56 smithi090 podman[69029]: 2024-04-24 09:18:56.222598846 +0000 UTC m=+0.033825352 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:18:56.603 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 24 09:18:56 smithi090 podman[69029]: 2024-04-24 09:18:56.358618385 +0000 UTC m=+0.169844928 container create 8d2f12b5f7dea0a340d2580fd34517df9702c7de3e9d70e59f0fe60d29abc6cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-g, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0) 2024-04-24T09:18:56.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:56 smithi149 ceph-mon[32592]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:56.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:56 smithi114 ceph-mon[32501]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:57.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:56 smithi090 ceph-mon[27825]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:57.102 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 24 09:18:56 smithi090 podman[69029]: 2024-04-24 09:18:56.64671284 +0000 UTC m=+0.457939390 container init 8d2f12b5f7dea0a340d2580fd34517df9702c7de3e9d70e59f0fe60d29abc6cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-g, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-24T09:18:57.102 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 24 09:18:56 smithi090 podman[69029]: 2024-04-24 09:18:56.656078354 +0000 UTC m=+0.467304879 container start 8d2f12b5f7dea0a340d2580fd34517df9702c7de3e9d70e59f0fe60d29abc6cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-g, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T09:18:57.102 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 24 09:18:56 smithi090 ceph-mds[69050]: -- 172.21.15.90:0/3274529106 <== mon.2 v2:172.21.15.114:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564c04e1fa40 con 0x564c04df2800 2024-04-24T09:18:57.102 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 24 09:18:56 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-g[69046]: starting mds.g at 2024-04-24T09:18:57.102 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 24 09:18:56 smithi090 bash[69029]: 8d2f12b5f7dea0a340d2580fd34517df9702c7de3e9d70e59f0fe60d29abc6cf 2024-04-24T09:18:57.102 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 24 09:18:56 smithi090 systemd[1]: Started Ceph mds.g for 30ee9be2-021a-11ef-bc93-c7b262605968. 2024-04-24T09:18:57.540 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:18:57.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.728+0000 7f29e2c00640 1 -- 172.21.15.90:0/3111653901 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29dc100d80 msgr2=0x7f29dc1011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:57.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.728+0000 7f29e2c00640 1 --2- 172.21.15.90:0/3111653901 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29dc100d80 0x7f29dc1011a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f29d0009860 tx=0x7f29d002a340 comp rx=0 tx=0).stop 2024-04-24T09:18:57.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.728+0000 7f29e2c00640 1 -- 172.21.15.90:0/3111653901 shutdown_connections 2024-04-24T09:18:57.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.728+0000 7f29e2c00640 1 --2- 172.21.15.90:0/3111653901 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29dc102930 0x7f29dc1091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:57.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.728+0000 7f29e2c00640 1 --2- 172.21.15.90:0/3111653901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29dc101f70 0x7f29dc1023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:57.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.728+0000 7f29e2c00640 1 --2- 172.21.15.90:0/3111653901 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29dc100d80 0x7f29dc1011a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:57.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.728+0000 7f29e2c00640 1 -- 172.21.15.90:0/3111653901 >> 172.21.15.90:0/3111653901 conn(0x7f29dc0fc2d0 msgr2=0x7f29dc0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:57.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.728+0000 7f29e2c00640 1 -- 172.21.15.90:0/3111653901 shutdown_connections 2024-04-24T09:18:57.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.728+0000 7f29e2c00640 1 -- 172.21.15.90:0/3111653901 wait complete. 2024-04-24T09:18:57.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.729+0000 7f29e2c00640 1 Processor -- start 2024-04-24T09:18:57.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.729+0000 7f29e2c00640 1 -- start start 2024-04-24T09:18:57.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.729+0000 7f29e2c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29dc100d80 0x7f29dc19a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:57.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.729+0000 7f29e2c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29dc101f70 0x7f29dc19a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:57.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.729+0000 7f29e2c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29dc102930 0x7f29dc1a1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:57.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.729+0000 7f29e2c00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f29dc10c9d0 con 0x7f29dc101f70 2024-04-24T09:18:57.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.729+0000 7f29e2c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f29dc10c850 con 0x7f29dc100d80 2024-04-24T09:18:57.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.729+0000 7f29e2c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f29dc10cb50 con 0x7f29dc102930 2024-04-24T09:18:57.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.730+0000 7f29e0e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29dc101f70 0x7f29dc19a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:57.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.730+0000 7f29e0e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29dc101f70 0x7f29dc19a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56220/0 (socket says 172.21.15.90:56220) 2024-04-24T09:18:57.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.730+0000 7f29e0e00640 1 -- 172.21.15.90:0/2191008401 learned_addr learned my addr 172.21.15.90:0/2191008401 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:18:57.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.730+0000 7f29e1800640 1 --2- 172.21.15.90:0/2191008401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29dc100d80 0x7f29dc19a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:57.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.730+0000 7f29e2200640 1 --2- 172.21.15.90:0/2191008401 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29dc102930 0x7f29dc1a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:57.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.730+0000 7f29e0e00640 1 -- 172.21.15.90:0/2191008401 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29dc102930 msgr2=0x7f29dc1a1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:57.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.730+0000 7f29e0e00640 1 --2- 172.21.15.90:0/2191008401 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29dc102930 0x7f29dc1a1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:57.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.730+0000 7f29e0e00640 1 -- 172.21.15.90:0/2191008401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29dc100d80 msgr2=0x7f29dc19a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:57.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.730+0000 7f29e0e00640 1 --2- 172.21.15.90:0/2191008401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29dc100d80 0x7f29dc19a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:57.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.730+0000 7f29e0e00640 1 -- 172.21.15.90:0/2191008401 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29dc1a1fc0 con 0x7f29dc101f70 2024-04-24T09:18:57.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.730+0000 7f29e2200640 1 --2- 172.21.15.90:0/2191008401 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29dc102930 0x7f29dc1a1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:18:57.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.730+0000 7f29e0e00640 1 --2- 172.21.15.90:0/2191008401 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29dc101f70 0x7f29dc19a8e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f29c800e9f0 tx=0x7f29c800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:57.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.731+0000 7f29da000640 1 -- 172.21.15.90:0/2191008401 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c800cd80 con 0x7f29dc101f70 2024-04-24T09:18:57.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.731+0000 7f29da000640 1 -- 172.21.15.90:0/2191008401 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29c8010910 con 0x7f29dc101f70 2024-04-24T09:18:57.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.731+0000 7f29e2c00640 1 -- 172.21.15.90:0/2191008401 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29dc1a22b0 con 0x7f29dc101f70 2024-04-24T09:18:57.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.731+0000 7f29da000640 1 -- 172.21.15.90:0/2191008401 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c8018990 con 0x7f29dc101f70 2024-04-24T09:18:57.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.731+0000 7f29e2c00640 1 -- 172.21.15.90:0/2191008401 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29dc1a2800 con 0x7f29dc101f70 2024-04-24T09:18:57.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.732+0000 7f29e2c00640 1 -- 172.21.15.90:0/2191008401 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29dc1a2a60 con 0x7f29dc101f70 2024-04-24T09:18:57.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.735+0000 7f29da000640 1 -- 172.21.15.90:0/2191008401 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f29c8010ac0 con 0x7f29dc101f70 2024-04-24T09:18:57.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.736+0000 7f29da000640 1 --2- 172.21.15.90:0/2191008401 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f29c40a3bb0 0x7f29c40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:57.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.736+0000 7f29da000640 1 -- 172.21.15.90:0/2191008401 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f29c80d07e0 con 0x7f29dc101f70 2024-04-24T09:18:57.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.736+0000 7f29e1800640 1 --2- 172.21.15.90:0/2191008401 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f29c40a3bb0 0x7f29c40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:57.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.737+0000 7f29e1800640 1 --2- 172.21.15.90:0/2191008401 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f29c40a3bb0 0x7f29c40a6070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f29d0030410 tx=0x7f29d0030550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:57.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.738+0000 7f29da000640 1 -- 172.21.15.90:0/2191008401 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f29c809d5c0 con 0x7f29dc101f70 2024-04-24T09:18:57.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.973+0000 7f29e2c00640 1 -- 172.21.15.90:0/2191008401 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29dc1a2c30 con 0x7f29dc101f70 2024-04-24T09:18:57.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.974+0000 7f29da000640 1 -- 172.21.15.90:0/2191008401 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8827 (secure 0 0 0) 0x7f29c809cec0 con 0x7f29dc101f70 2024-04-24T09:18:57.974 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:18:57.974 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/227251251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":227251251},{"type":"v1","addr":"172.21.15.90:6837","nonce":227251251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/1904617971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":1904617971},{"type":"v1","addr":"172.21.15.90:6839","nonce":1904617971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/219377141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":219377141},{"type":"v1","addr":"172.21.15.114:6833","nonce":219377141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24446,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2396995634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2396995634},{"type":"v1","addr":"172.21.15.90:6835","nonce":2396995634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/779642417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":779642417},{"type":"v1","addr":"172.21.15.149:6833","nonce":779642417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/2565340800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":2565340800},{"type":"v1","addr":"172.21.15.114:6835","nonce":2565340800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/875324280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":875324280},{"type":"v1","addr":"172.21.15.114:6837","nonce":875324280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/3157375624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3157375624},{"type":"v1","addr":"172.21.15.149:6835","nonce":3157375624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/2250310587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2250310587},{"type":"v1","addr":"172.21.15.149:6837","nonce":2250310587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6839/652734263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":652734263},{"type":"v1","addr":"172.21.15.149:6839","nonce":652734263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-24T09:18:57.975 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 11 2024-04-24T09:18:57.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.976+0000 7f29e2c00640 1 -- 172.21.15.90:0/2191008401 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f29c40a3bb0 msgr2=0x7f29c40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:57.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.976+0000 7f29e2c00640 1 --2- 172.21.15.90:0/2191008401 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f29c40a3bb0 0x7f29c40a6070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f29d0030410 tx=0x7f29d0030550 comp rx=0 tx=0).stop 2024-04-24T09:18:57.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.976+0000 7f29e2c00640 1 -- 172.21.15.90:0/2191008401 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29dc101f70 msgr2=0x7f29dc19a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:57.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.976+0000 7f29e2c00640 1 --2- 172.21.15.90:0/2191008401 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29dc101f70 0x7f29dc19a8e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f29c800e9f0 tx=0x7f29c800eeb0 comp rx=0 tx=0).stop 2024-04-24T09:18:57.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.977+0000 7f29e2c00640 1 -- 172.21.15.90:0/2191008401 shutdown_connections 2024-04-24T09:18:57.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.977+0000 7f29e2c00640 1 --2- 172.21.15.90:0/2191008401 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f29c40a3bb0 0x7f29c40a6070 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:57.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.977+0000 7f29e2c00640 1 --2- 172.21.15.90:0/2191008401 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29dc102930 0x7f29dc1a1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:57.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.977+0000 7f29e2c00640 1 --2- 172.21.15.90:0/2191008401 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29dc101f70 0x7f29dc19a8e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:57.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.977+0000 7f29e2c00640 1 --2- 172.21.15.90:0/2191008401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29dc100d80 0x7f29dc19a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:57.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.977+0000 7f29e2c00640 1 -- 172.21.15.90:0/2191008401 >> 172.21.15.90:0/2191008401 conn(0x7f29dc0fc2d0 msgr2=0x7f29dc0fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:57.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.977+0000 7f29e2c00640 1 -- 172.21.15.90:0/2191008401 shutdown_connections 2024-04-24T09:18:57.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:18:57.977+0000 7f29e2c00640 1 -- 172.21.15.90:0/2191008401 wait complete. 2024-04-24T09:18:58.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:57 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:58.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:57 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:58.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:57 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:58.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:57 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:18:58.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:57 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:18:58.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:57 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:18:58.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:57 smithi090 ceph-mon[27825]: Deploying daemon mds.a on smithi090 2024-04-24T09:18:58.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:57 smithi090 ceph-mon[27825]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:58.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:57 smithi090 ceph-mon[27825]: mds.? [v2:172.21.15.90:6838/1904617971,v1:172.21.15.90:6839/1904617971] up:boot 2024-04-24T09:18:58.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:57 smithi090 ceph-mon[27825]: fsmap 10 up:standby 2024-04-24T09:18:58.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:57 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T09:18:58.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:57 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:58.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:57 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:58.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:57 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:58.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:57 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:18:58.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:57 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:18:58.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:57 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:18:58.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:57 smithi114 ceph-mon[32501]: Deploying daemon mds.a on smithi090 2024-04-24T09:18:58.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:57 smithi114 ceph-mon[32501]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:58.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:57 smithi114 ceph-mon[32501]: mds.? [v2:172.21.15.90:6838/1904617971,v1:172.21.15.90:6839/1904617971] up:boot 2024-04-24T09:18:58.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:57 smithi114 ceph-mon[32501]: fsmap 10 up:standby 2024-04-24T09:18:58.177 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:57 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T09:18:58.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:57 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:58.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:57 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:58.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:57 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:18:58.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:57 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:18:58.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:57 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:18:58.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:57 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:18:58.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:57 smithi149 ceph-mon[32592]: Deploying daemon mds.a on smithi090 2024-04-24T09:18:58.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:57 smithi149 ceph-mon[32592]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:18:58.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:57 smithi149 ceph-mon[32592]: mds.? [v2:172.21.15.90:6838/1904617971,v1:172.21.15.90:6839/1904617971] up:boot 2024-04-24T09:18:58.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:57 smithi149 ceph-mon[32592]: fsmap 10 up:standby 2024-04-24T09:18:58.362 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:57 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T09:18:59.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:58 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/2191008401' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:18:59.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:58 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/2191008401' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:18:59.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:58 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/2191008401' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:18:59.995 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:19:00.091 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:18:59 smithi090 ceph-mon[27825]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:19:00.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:18:59 smithi114 ceph-mon[32501]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:19:00.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.221+0000 7f09fb000640 1 -- 172.21.15.90:0/1240641509 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09f4101f70 msgr2=0x7f09f41023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:00.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.221+0000 7f09fb000640 1 --2- 172.21.15.90:0/1240641509 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09f4101f70 0x7f09f41023f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f09e0009900 tx=0x7f09e002a3f0 comp rx=0 tx=0).stop 2024-04-24T09:19:00.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.221+0000 7f09fb000640 1 -- 172.21.15.90:0/1240641509 shutdown_connections 2024-04-24T09:19:00.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.221+0000 7f09fb000640 1 --2- 172.21.15.90:0/1240641509 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09f4102930 0x7f09f41091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:00.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.221+0000 7f09fb000640 1 --2- 172.21.15.90:0/1240641509 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09f4101f70 0x7f09f41023f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:00.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.221+0000 7f09fb000640 1 --2- 172.21.15.90:0/1240641509 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09f4100d80 0x7f09f41011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:00.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.221+0000 7f09fb000640 1 -- 172.21.15.90:0/1240641509 >> 172.21.15.90:0/1240641509 conn(0x7f09f40fc310 msgr2=0x7f09f40fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:00.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.221+0000 7f09fb000640 1 -- 172.21.15.90:0/1240641509 shutdown_connections 2024-04-24T09:19:00.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.221+0000 7f09fb000640 1 -- 172.21.15.90:0/1240641509 wait complete. 2024-04-24T09:19:00.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.223+0000 7f09fb000640 1 Processor -- start 2024-04-24T09:19:00.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.223+0000 7f09fb000640 1 -- start start 2024-04-24T09:19:00.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.223+0000 7f09fb000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09f4100d80 0x7f09f419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:00.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.223+0000 7f09fb000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09f4101f70 0x7f09f419a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:00.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.223+0000 7f09fb000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09f4102930 0x7f09f41a19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:00.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.223+0000 7f09fb000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f09f410ca30 con 0x7f09f4101f70 2024-04-24T09:19:00.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.223+0000 7f09fb000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f09f410c8b0 con 0x7f09f4100d80 2024-04-24T09:19:00.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.223+0000 7f09fb000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f09f410cbb0 con 0x7f09f4102930 2024-04-24T09:19:00.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.223+0000 7f09f9c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09f4100d80 0x7f09f419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:00.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.223+0000 7f09f9c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09f4100d80 0x7f09f419a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.90:33980/0 (socket says 172.21.15.90:33980) 2024-04-24T09:19:00.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.223+0000 7f09f9c00640 1 -- 172.21.15.90:0/1246451040 learned_addr learned my addr 172.21.15.90:0/1246451040 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:19:00.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.223+0000 7f09f9200640 1 --2- 172.21.15.90:0/1246451040 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09f4101f70 0x7f09f419a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:00.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.224+0000 7f09fa600640 1 --2- 172.21.15.90:0/1246451040 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09f4102930 0x7f09f41a19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:00.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.224+0000 7f09f9c00640 1 -- 172.21.15.90:0/1246451040 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09f4102930 msgr2=0x7f09f41a19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:00.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.224+0000 7f09f9c00640 1 --2- 172.21.15.90:0/1246451040 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09f4102930 0x7f09f41a19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:00.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.224+0000 7f09f9c00640 1 -- 172.21.15.90:0/1246451040 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09f4101f70 msgr2=0x7f09f419a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:00.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.224+0000 7f09f9c00640 1 --2- 172.21.15.90:0/1246451040 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09f4101f70 0x7f09f419a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:00.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.224+0000 7f09f9c00640 1 -- 172.21.15.90:0/1246451040 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09f41a2030 con 0x7f09f4100d80 2024-04-24T09:19:00.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.224+0000 7f09f9c00640 1 --2- 172.21.15.90:0/1246451040 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09f4100d80 0x7f09f419a460 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f09e800e9c0 tx=0x7f09e800edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:00.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.224+0000 7f09f2000640 1 -- 172.21.15.90:0/1246451040 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e800cd80 con 0x7f09f4100d80 2024-04-24T09:19:00.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.225+0000 7f09f2000640 1 -- 172.21.15.90:0/1246451040 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09e8010910 con 0x7f09f4100d80 2024-04-24T09:19:00.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.225+0000 7f09f2000640 1 -- 172.21.15.90:0/1246451040 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e8018950 con 0x7f09f4100d80 2024-04-24T09:19:00.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.225+0000 7f09fb000640 1 -- 172.21.15.90:0/1246451040 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09f41a2320 con 0x7f09f4100d80 2024-04-24T09:19:00.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.225+0000 7f09fb000640 1 -- 172.21.15.90:0/1246451040 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09f41a27f0 con 0x7f09f4100d80 2024-04-24T09:19:00.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.226+0000 7f09fb000640 1 -- 172.21.15.90:0/1246451040 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09f41a2a50 con 0x7f09f4100d80 2024-04-24T09:19:00.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.228+0000 7f09f2000640 1 -- 172.21.15.90:0/1246451040 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f09e801f020 con 0x7f09f4100d80 2024-04-24T09:19:00.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.229+0000 7f09f2000640 1 --2- 172.21.15.90:0/1246451040 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f09dc0a3c30 0x7f09dc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:00.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.229+0000 7f09f2000640 1 -- 172.21.15.90:0/1246451040 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f09e80cfff0 con 0x7f09f4100d80 2024-04-24T09:19:00.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.230+0000 7f09f9200640 1 --2- 172.21.15.90:0/1246451040 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f09dc0a3c30 0x7f09dc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:00.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.230+0000 7f09f9200640 1 --2- 172.21.15.90:0/1246451040 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f09dc0a3c30 0x7f09dc0a60f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f09e0031440 tx=0x7f09e00315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:00.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.233+0000 7f09f2000640 1 -- 172.21.15.90:0/1246451040 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f09e809cdf0 con 0x7f09f4100d80 2024-04-24T09:19:00.352 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 24 09:19:00 smithi090 systemd[1]: Starting Ceph mds.a for 30ee9be2-021a-11ef-bc93-c7b262605968... 2024-04-24T09:19:00.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:18:59 smithi149 ceph-mon[32592]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:19:00.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.488+0000 7f09fb000640 1 -- 172.21.15.90:0/1246451040 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09f41a2de0 con 0x7f09f4100d80 2024-04-24T09:19:00.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.489+0000 7f09f2000640 1 -- 172.21.15.90:0/1246451040 <== mon.2 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8827 (secure 0 0 0) 0x7f09e809c6f0 con 0x7f09f4100d80 2024-04-24T09:19:00.489 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:19:00.489 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/227251251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":227251251},{"type":"v1","addr":"172.21.15.90:6837","nonce":227251251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/1904617971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":1904617971},{"type":"v1","addr":"172.21.15.90:6839","nonce":1904617971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/219377141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":219377141},{"type":"v1","addr":"172.21.15.114:6833","nonce":219377141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24446,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2396995634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2396995634},{"type":"v1","addr":"172.21.15.90:6835","nonce":2396995634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/779642417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":779642417},{"type":"v1","addr":"172.21.15.149:6833","nonce":779642417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/2565340800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":2565340800},{"type":"v1","addr":"172.21.15.114:6835","nonce":2565340800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/875324280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":875324280},{"type":"v1","addr":"172.21.15.114:6837","nonce":875324280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/3157375624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3157375624},{"type":"v1","addr":"172.21.15.149:6835","nonce":3157375624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/2250310587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2250310587},{"type":"v1","addr":"172.21.15.149:6837","nonce":2250310587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6839/652734263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":652734263},{"type":"v1","addr":"172.21.15.149:6839","nonce":652734263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-24T09:19:00.490 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 11 2024-04-24T09:19:00.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.492+0000 7f09fb000640 1 -- 172.21.15.90:0/1246451040 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f09dc0a3c30 msgr2=0x7f09dc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:00.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.492+0000 7f09fb000640 1 --2- 172.21.15.90:0/1246451040 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f09dc0a3c30 0x7f09dc0a60f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f09e0031440 tx=0x7f09e00315b0 comp rx=0 tx=0).stop 2024-04-24T09:19:00.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.492+0000 7f09fb000640 1 -- 172.21.15.90:0/1246451040 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09f4100d80 msgr2=0x7f09f419a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:00.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.492+0000 7f09fb000640 1 --2- 172.21.15.90:0/1246451040 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09f4100d80 0x7f09f419a460 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f09e800e9c0 tx=0x7f09e800edc0 comp rx=0 tx=0).stop 2024-04-24T09:19:00.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.492+0000 7f09fb000640 1 -- 172.21.15.90:0/1246451040 shutdown_connections 2024-04-24T09:19:00.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.492+0000 7f09fb000640 1 --2- 172.21.15.90:0/1246451040 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f09dc0a3c30 0x7f09dc0a60f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:00.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.492+0000 7f09fb000640 1 --2- 172.21.15.90:0/1246451040 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09f4102930 0x7f09f41a19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:00.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.492+0000 7f09fb000640 1 --2- 172.21.15.90:0/1246451040 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09f4101f70 0x7f09f419a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:00.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.492+0000 7f09fb000640 1 --2- 172.21.15.90:0/1246451040 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09f4100d80 0x7f09f419a460 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:00.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.492+0000 7f09fb000640 1 -- 172.21.15.90:0/1246451040 >> 172.21.15.90:0/1246451040 conn(0x7f09f40fc310 msgr2=0x7f09f40fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:00.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.492+0000 7f09fb000640 1 -- 172.21.15.90:0/1246451040 shutdown_connections 2024-04-24T09:19:00.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:00.493+0000 7f09fb000640 1 -- 172.21.15.90:0/1246451040 wait complete. 2024-04-24T09:19:00.742 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 24 09:19:00 smithi090 podman[69414]: 2024-04-24 09:19:00.386920799 +0000 UTC m=+0.033227848 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:19:00.742 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 24 09:19:00 smithi090 podman[69414]: 2024-04-24 09:19:00.534537932 +0000 UTC m=+0.180844984 container create ec3fa743ec6c4ea68c526bbde9171b9fc1a8969c791fe7faf0104528c78c778d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-a, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True) 2024-04-24T09:19:01.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:00 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/1246451040' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:19:01.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:00 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/1246451040' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:19:01.353 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 24 09:19:00 smithi090 podman[69414]: 2024-04-24 09:19:00.860634256 +0000 UTC m=+0.506941349 container init ec3fa743ec6c4ea68c526bbde9171b9fc1a8969c791fe7faf0104528c78c778d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-a, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS) 2024-04-24T09:19:01.353 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 24 09:19:00 smithi090 podman[69414]: 2024-04-24 09:19:00.871679229 +0000 UTC m=+0.517986295 container start ec3fa743ec6c4ea68c526bbde9171b9fc1a8969c791fe7faf0104528c78c778d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-a, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T09:19:01.353 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 24 09:19:00 smithi090 ceph-mds[69436]: -- 172.21.15.90:0/988152964 <== mon.2 v2:172.21.15.114:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5574b277fa40 con 0x5574b2752800 2024-04-24T09:19:01.353 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 24 09:19:00 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-a[69432]: starting mds.a at 2024-04-24T09:19:01.353 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 24 09:19:00 smithi090 bash[69414]: ec3fa743ec6c4ea68c526bbde9171b9fc1a8969c791fe7faf0104528c78c778d 2024-04-24T09:19:01.353 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 24 09:19:00 smithi090 systemd[1]: Started Ceph mds.a for 30ee9be2-021a-11ef-bc93-c7b262605968. 2024-04-24T09:19:01.361 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:00 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/1246451040' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:19:02.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:02 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:02.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:02 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:02.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:02 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:02.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:02 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:19:02.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:02 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:19:02.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:02 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:19:02.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:02 smithi114 ceph-mon[32501]: Deploying daemon mds.b on smithi114 2024-04-24T09:19:02.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:02 smithi114 ceph-mon[32501]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:19:02.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:02 smithi114 ceph-mon[32501]: mds.? [v2:172.21.15.90:6840/397593398,v1:172.21.15.90:6841/397593398] up:boot 2024-04-24T09:19:02.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:02 smithi114 ceph-mon[32501]: fsmap 11 up:standby 2024-04-24T09:19:02.427 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:02 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T09:19:02.514 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:19:02.592 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:02 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:02.592 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:02 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:02.592 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:02 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:02.592 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:02 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:19:02.592 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:02 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:19:02.592 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:02 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:19:02.593 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:02 smithi090 ceph-mon[27825]: Deploying daemon mds.b on smithi114 2024-04-24T09:19:02.593 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:02 smithi090 ceph-mon[27825]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:19:02.593 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:02 smithi090 ceph-mon[27825]: mds.? [v2:172.21.15.90:6840/397593398,v1:172.21.15.90:6841/397593398] up:boot 2024-04-24T09:19:02.593 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:02 smithi090 ceph-mon[27825]: fsmap 11 up:standby 2024-04-24T09:19:02.593 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:02 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T09:19:02.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:02 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:02.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:02 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:02.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:02 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:02.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:02 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T09:19:02.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:02 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T09:19:02.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:02 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:19:02.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:02 smithi149 ceph-mon[32592]: Deploying daemon mds.b on smithi114 2024-04-24T09:19:02.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:02 smithi149 ceph-mon[32592]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:19:02.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:02 smithi149 ceph-mon[32592]: mds.? [v2:172.21.15.90:6840/397593398,v1:172.21.15.90:6841/397593398] up:boot 2024-04-24T09:19:02.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:02 smithi149 ceph-mon[32592]: fsmap 11 up:standby 2024-04-24T09:19:02.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:02 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T09:19:02.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.729+0000 7efd6e800640 1 -- 172.21.15.90:0/1709669568 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd68109670 msgr2=0x7efd68109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:02.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.729+0000 7efd6e800640 1 --2- 172.21.15.90:0/1709669568 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd68109670 0x7efd68109af0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efd54009900 tx=0x7efd5402a410 comp rx=0 tx=0).stop 2024-04-24T09:19:02.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.729+0000 7efd6e800640 1 -- 172.21.15.90:0/1709669568 shutdown_connections 2024-04-24T09:19:02.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.729+0000 7efd6e800640 1 --2- 172.21.15.90:0/1709669568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd6810a030 0x7efd681108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:02.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.729+0000 7efd6e800640 1 --2- 172.21.15.90:0/1709669568 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd68109670 0x7efd68109af0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:02.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.729+0000 7efd6e800640 1 --2- 172.21.15.90:0/1709669568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efd68108480 0x7efd681088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:02.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.729+0000 7efd6e800640 1 -- 172.21.15.90:0/1709669568 >> 172.21.15.90:0/1709669568 conn(0x7efd68077440 msgr2=0x7efd68079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:02.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.729+0000 7efd6e800640 1 -- 172.21.15.90:0/1709669568 shutdown_connections 2024-04-24T09:19:02.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.729+0000 7efd6e800640 1 -- 172.21.15.90:0/1709669568 wait complete. 2024-04-24T09:19:02.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.730+0000 7efd6e800640 1 Processor -- start 2024-04-24T09:19:02.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.730+0000 7efd6e800640 1 -- start start 2024-04-24T09:19:02.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.730+0000 7efd6e800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efd68108480 0x7efd6819e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:02.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.730+0000 7efd6e800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd68109670 0x7efd6819ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:02.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.730+0000 7efd6e800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd6810a030 0x7efd681a5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:02.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.730+0000 7efd6e800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efd68114160 con 0x7efd6810a030 2024-04-24T09:19:02.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.730+0000 7efd6e800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7efd68113fe0 con 0x7efd68108480 2024-04-24T09:19:02.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.730+0000 7efd6e800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7efd681142e0 con 0x7efd68109670 2024-04-24T09:19:02.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.731+0000 7efd6de00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd6810a030 0x7efd681a5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:02.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.731+0000 7efd6ca00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd68109670 0x7efd6819ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:02.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.731+0000 7efd6ca00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd68109670 0x7efd6819ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:41604/0 (socket says 172.21.15.90:41604) 2024-04-24T09:19:02.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.731+0000 7efd6ca00640 1 -- 172.21.15.90:0/519801301 learned_addr learned my addr 172.21.15.90:0/519801301 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:19:02.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.731+0000 7efd6de00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd6810a030 0x7efd681a5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56260/0 (socket says 172.21.15.90:56260) 2024-04-24T09:19:02.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.731+0000 7efd6d400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efd68108480 0x7efd6819e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:02.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.731+0000 7efd6ca00640 1 -- 172.21.15.90:0/519801301 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efd68108480 msgr2=0x7efd6819e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:02.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.731+0000 7efd6ca00640 1 --2- 172.21.15.90:0/519801301 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efd68108480 0x7efd6819e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:02.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.731+0000 7efd6ca00640 1 -- 172.21.15.90:0/519801301 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd6810a030 msgr2=0x7efd681a5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:02.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.731+0000 7efd6ca00640 1 --2- 172.21.15.90:0/519801301 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd6810a030 0x7efd681a5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:02.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.731+0000 7efd6ca00640 1 -- 172.21.15.90:0/519801301 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd681a6370 con 0x7efd68109670 2024-04-24T09:19:02.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.731+0000 7efd6de00640 1 --2- 172.21.15.90:0/519801301 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd6810a030 0x7efd681a5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:19:02.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.731+0000 7efd6ca00640 1 --2- 172.21.15.90:0/519801301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd68109670 0x7efd6819ed30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7efd54031440 tx=0x7efd54032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:02.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.732+0000 7efd66000640 1 -- 172.21.15.90:0/519801301 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5403f030 con 0x7efd68109670 2024-04-24T09:19:02.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.732+0000 7efd66000640 1 -- 172.21.15.90:0/519801301 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd54003ce0 con 0x7efd68109670 2024-04-24T09:19:02.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.732+0000 7efd6e800640 1 -- 172.21.15.90:0/519801301 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd681a6600 con 0x7efd68109670 2024-04-24T09:19:02.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.732+0000 7efd66000640 1 -- 172.21.15.90:0/519801301 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd54035070 con 0x7efd68109670 2024-04-24T09:19:02.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.732+0000 7efd6e800640 1 -- 172.21.15.90:0/519801301 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd681a6af0 con 0x7efd68109670 2024-04-24T09:19:02.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.733+0000 7efd6e800640 1 -- 172.21.15.90:0/519801301 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd6810f7a0 con 0x7efd68109670 2024-04-24T09:19:02.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.736+0000 7efd66000640 1 -- 172.21.15.90:0/519801301 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7efd54042030 con 0x7efd68109670 2024-04-24T09:19:02.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.737+0000 7efd66000640 1 --2- 172.21.15.90:0/519801301 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7efd4c0a3bb0 0x7efd4c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:02.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.737+0000 7efd66000640 1 -- 172.21.15.90:0/519801301 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7efd540f0a60 con 0x7efd68109670 2024-04-24T09:19:02.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.737+0000 7efd6d400640 1 --2- 172.21.15.90:0/519801301 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7efd4c0a3bb0 0x7efd4c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:02.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.738+0000 7efd6d400640 1 --2- 172.21.15.90:0/519801301 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7efd4c0a3bb0 0x7efd4c0a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7efd681094d0 tx=0x7efd58009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:02.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.739+0000 7efd66000640 1 -- 172.21.15.90:0/519801301 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efd540bd1f0 con 0x7efd68109670 2024-04-24T09:19:02.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.984+0000 7efd6e800640 1 -- 172.21.15.90:0/519801301 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd6802d620 con 0x7efd68109670 2024-04-24T09:19:02.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.986+0000 7efd66000640 1 -- 172.21.15.90:0/519801301 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9650 (secure 0 0 0) 0x7efd540c14f0 con 0x7efd68109670 2024-04-24T09:19:02.986 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:19:02.986 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/227251251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":227251251},{"type":"v1","addr":"172.21.15.90:6837","nonce":227251251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/1904617971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":1904617971},{"type":"v1","addr":"172.21.15.90:6839","nonce":1904617971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/219377141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":219377141},{"type":"v1","addr":"172.21.15.114:6833","nonce":219377141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24446,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2396995634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2396995634},{"type":"v1","addr":"172.21.15.90:6835","nonce":2396995634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/779642417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":779642417},{"type":"v1","addr":"172.21.15.149:6833","nonce":779642417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/2565340800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":2565340800},{"type":"v1","addr":"172.21.15.114:6835","nonce":2565340800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/875324280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":875324280},{"type":"v1","addr":"172.21.15.114:6837","nonce":875324280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/3157375624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3157375624},{"type":"v1","addr":"172.21.15.149:6835","nonce":3157375624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/2250310587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2250310587},{"type":"v1","addr":"172.21.15.149:6837","nonce":2250310587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24545,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6841/397593398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":397593398},{"type":"v1","addr":"172.21.15.90:6841","nonce":397593398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6839/652734263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":652734263},{"type":"v1","addr":"172.21.15.149:6839","nonce":652734263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-24T09:19:02.986 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 12 2024-04-24T09:19:02.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.988+0000 7efd6e800640 1 -- 172.21.15.90:0/519801301 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7efd4c0a3bb0 msgr2=0x7efd4c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:02.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.988+0000 7efd6e800640 1 --2- 172.21.15.90:0/519801301 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7efd4c0a3bb0 0x7efd4c0a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7efd681094d0 tx=0x7efd58009500 comp rx=0 tx=0).stop 2024-04-24T09:19:02.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.989+0000 7efd6e800640 1 -- 172.21.15.90:0/519801301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd68109670 msgr2=0x7efd6819ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:02.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.989+0000 7efd6e800640 1 --2- 172.21.15.90:0/519801301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd68109670 0x7efd6819ed30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7efd54031440 tx=0x7efd54032ce0 comp rx=0 tx=0).stop 2024-04-24T09:19:02.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.989+0000 7efd6e800640 1 -- 172.21.15.90:0/519801301 shutdown_connections 2024-04-24T09:19:02.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.989+0000 7efd6e800640 1 --2- 172.21.15.90:0/519801301 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7efd4c0a3bb0 0x7efd4c0a6070 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:02.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.989+0000 7efd6e800640 1 --2- 172.21.15.90:0/519801301 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd6810a030 0x7efd681a5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:02.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.989+0000 7efd6e800640 1 --2- 172.21.15.90:0/519801301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd68109670 0x7efd6819ed30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:02.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.989+0000 7efd6e800640 1 --2- 172.21.15.90:0/519801301 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efd68108480 0x7efd6819e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:02.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.989+0000 7efd6e800640 1 -- 172.21.15.90:0/519801301 >> 172.21.15.90:0/519801301 conn(0x7efd68077440 msgr2=0x7efd6810c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:02.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.989+0000 7efd6e800640 1 -- 172.21.15.90:0/519801301 shutdown_connections 2024-04-24T09:19:02.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:02.989+0000 7efd6e800640 1 -- 172.21.15.90:0/519801301 wait complete. 2024-04-24T09:19:03.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:03 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/519801301' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:19:03.426 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:03 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/519801301' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:19:03.426 INFO:journalctl@ceph.mds.b.smithi114.stdout:Apr 24 09:19:03 smithi114 systemd[1]: Starting Ceph mds.b for 30ee9be2-021a-11ef-bc93-c7b262605968... 2024-04-24T09:19:03.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:03 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/519801301' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:19:03.771 INFO:journalctl@ceph.mds.b.smithi114.stdout:Apr 24 09:19:03 smithi114 podman[63593]: 2024-04-24 09:19:03.409380449 +0000 UTC m=+0.017734014 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:19:03.772 INFO:journalctl@ceph.mds.b.smithi114.stdout:Apr 24 09:19:03 smithi114 podman[63593]: 2024-04-24 09:19:03.537018095 +0000 UTC m=+0.145371661 container create a4d4b1357aac81c5119ea551977cf821f220f08a9204ef98aa2e475262d2e74d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T09:19:03.772 INFO:journalctl@ceph.mds.b.smithi114.stdout:Apr 24 09:19:03 smithi114 podman[63593]: 2024-04-24 09:19:03.772907755 +0000 UTC m=+0.381261320 container init a4d4b1357aac81c5119ea551977cf821f220f08a9204ef98aa2e475262d2e74d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-b, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-24T09:19:04.176 INFO:journalctl@ceph.mds.b.smithi114.stdout:Apr 24 09:19:03 smithi114 podman[63593]: 2024-04-24 09:19:03.775800223 +0000 UTC m=+0.384153790 container start a4d4b1357aac81c5119ea551977cf821f220f08a9204ef98aa2e475262d2e74d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-b, io.buildah.version=1.33.5, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-24T09:19:04.176 INFO:journalctl@ceph.mds.b.smithi114.stdout:Apr 24 09:19:03 smithi114 ceph-mds[63612]: -- 172.21.15.114:0/388541633 <== mon.2 v2:172.21.15.114:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ee0e207a40 con 0x55ee0e1db000 2024-04-24T09:19:04.176 INFO:journalctl@ceph.mds.b.smithi114.stdout:Apr 24 09:19:03 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-b[63608]: starting mds.b at 2024-04-24T09:19:04.176 INFO:journalctl@ceph.mds.b.smithi114.stdout:Apr 24 09:19:03 smithi114 bash[63593]: a4d4b1357aac81c5119ea551977cf821f220f08a9204ef98aa2e475262d2e74d 2024-04-24T09:19:04.176 INFO:journalctl@ceph.mds.b.smithi114.stdout:Apr 24 09:19:03 smithi114 systemd[1]: Started Ceph mds.b for 30ee9be2-021a-11ef-bc93-c7b262605968. 2024-04-24T09:19:04.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:04 smithi090 ceph-mon[27825]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:19:04.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:04 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:04.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:04 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:04.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:04 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:04.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:04 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:04.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:04 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:04.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:04 smithi149 ceph-mon[32592]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:19:04.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:04 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:04.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:04 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:04.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:04 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:04.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:04 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:04.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:04 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:04.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:04 smithi114 ceph-mon[32501]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:19:04.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:04 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:04.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:04 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:04.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:04 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:04.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:04 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:04.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:04 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:05.010 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:19:05.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.206+0000 7f0fe6a00640 1 -- 172.21.15.90:0/2201371975 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0fe0105e10 msgr2=0x7f0fe010c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:05.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.206+0000 7f0fe6a00640 1 --2- 172.21.15.90:0/2201371975 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0fe0105e10 0x7f0fe010c6a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0fd8009ca0 tx=0x7f0fd802a8c0 comp rx=0 tx=0).stop 2024-04-24T09:19:05.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.206+0000 7f0fe6a00640 1 -- 172.21.15.90:0/2201371975 shutdown_connections 2024-04-24T09:19:05.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.206+0000 7f0fe6a00640 1 --2- 172.21.15.90:0/2201371975 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0fe0105e10 0x7f0fe010c6a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:05.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.206+0000 7f0fe6a00640 1 --2- 172.21.15.90:0/2201371975 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fe0105450 0x7f0fe01058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:05.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.206+0000 7f0fe6a00640 1 --2- 172.21.15.90:0/2201371975 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fe0104260 0x7f0fe0104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:05.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.206+0000 7f0fe6a00640 1 -- 172.21.15.90:0/2201371975 >> 172.21.15.90:0/2201371975 conn(0x7f0fe00ff7f0 msgr2=0x7f0fe0101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:05.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.206+0000 7f0fe6a00640 1 -- 172.21.15.90:0/2201371975 shutdown_connections 2024-04-24T09:19:05.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.206+0000 7f0fe6a00640 1 -- 172.21.15.90:0/2201371975 wait complete. 2024-04-24T09:19:05.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.207+0000 7f0fe6a00640 1 Processor -- start 2024-04-24T09:19:05.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.207+0000 7f0fe6a00640 1 -- start start 2024-04-24T09:19:05.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.207+0000 7f0fe6a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fe0104260 0x7f0fe019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:05.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.207+0000 7f0fe6a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0fe0105450 0x7f0fe019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:05.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.207+0000 7f0fe6a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fe0105e10 0x7f0fe01a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:05.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.207+0000 7f0fe6a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe010ff10 con 0x7f0fe0105450 2024-04-24T09:19:05.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.207+0000 7f0fe6a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe010fd90 con 0x7f0fe0105e10 2024-04-24T09:19:05.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.207+0000 7f0fe6a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe0110090 con 0x7f0fe0104260 2024-04-24T09:19:05.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.208+0000 7f0fdf400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0fe0105450 0x7f0fe019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:05.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.208+0000 7f0fdfe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fe0104260 0x7f0fe019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:05.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.208+0000 7f0fdfe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fe0104260 0x7f0fe019a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:41618/0 (socket says 172.21.15.90:41618) 2024-04-24T09:19:05.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.208+0000 7f0fdfe00640 1 -- 172.21.15.90:0/4047874416 learned_addr learned my addr 172.21.15.90:0/4047874416 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:19:05.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.208+0000 7f0fe4a00640 1 --2- 172.21.15.90:0/4047874416 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fe0105e10 0x7f0fe01a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:05.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.208+0000 7f0fdfe00640 1 -- 172.21.15.90:0/4047874416 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fe0105e10 msgr2=0x7f0fe01a1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:05.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.208+0000 7f0fdfe00640 1 --2- 172.21.15.90:0/4047874416 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fe0105e10 0x7f0fe01a1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:05.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.208+0000 7f0fdfe00640 1 -- 172.21.15.90:0/4047874416 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0fe0105450 msgr2=0x7f0fe019aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:05.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.208+0000 7f0fdfe00640 1 --2- 172.21.15.90:0/4047874416 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0fe0105450 0x7f0fe019aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:05.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.208+0000 7f0fdfe00640 1 -- 172.21.15.90:0/4047874416 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fe01a2140 con 0x7f0fe0104260 2024-04-24T09:19:05.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.208+0000 7f0fe4a00640 1 --2- 172.21.15.90:0/4047874416 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fe0105e10 0x7f0fe01a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:19:05.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.208+0000 7f0fdfe00640 1 --2- 172.21.15.90:0/4047874416 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fe0104260 0x7f0fe019a520 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0fd000e9c0 tx=0x7f0fd000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:05.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.209+0000 7f0fdcc00640 1 -- 172.21.15.90:0/4047874416 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd000cde0 con 0x7f0fe0104260 2024-04-24T09:19:05.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.209+0000 7f0fdcc00640 1 -- 172.21.15.90:0/4047874416 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fd0010910 con 0x7f0fe0104260 2024-04-24T09:19:05.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.209+0000 7f0fe6a00640 1 -- 172.21.15.90:0/4047874416 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fe01a2430 con 0x7f0fe0104260 2024-04-24T09:19:05.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.209+0000 7f0fdcc00640 1 -- 172.21.15.90:0/4047874416 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd0018a00 con 0x7f0fe0104260 2024-04-24T09:19:05.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.209+0000 7f0fe6a00640 1 -- 172.21.15.90:0/4047874416 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fe01a2980 con 0x7f0fe0104260 2024-04-24T09:19:05.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.210+0000 7f0fe6a00640 1 -- 172.21.15.90:0/4047874416 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fe010b520 con 0x7f0fe0104260 2024-04-24T09:19:05.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.213+0000 7f0fdcc00640 1 -- 172.21.15.90:0/4047874416 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0fd001f020 con 0x7f0fe0104260 2024-04-24T09:19:05.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.215+0000 7f0fdcc00640 1 --2- 172.21.15.90:0/4047874416 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f0fc40a3bb0 0x7f0fc40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:05.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.215+0000 7f0fdcc00640 1 -- 172.21.15.90:0/4047874416 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f0fd00d0280 con 0x7f0fe0104260 2024-04-24T09:19:05.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.215+0000 7f0fdf400640 1 --2- 172.21.15.90:0/4047874416 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f0fc40a3bb0 0x7f0fc40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:05.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.216+0000 7f0fdf400640 1 --2- 172.21.15.90:0/4047874416 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f0fc40a3bb0 0x7f0fc40a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0fe019ba30 tx=0x7f0fcc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:05.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.219+0000 7f0fdcc00640 1 -- 172.21.15.90:0/4047874416 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0fd009d060 con 0x7f0fe0104260 2024-04-24T09:19:05.337 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:05 smithi114 ceph-mon[32501]: mds.? [v2:172.21.15.114:6838/1025403710,v1:172.21.15.114:6839/1025403710] up:boot 2024-04-24T09:19:05.337 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:05 smithi114 ceph-mon[32501]: fsmap 12 up:standby 2024-04-24T09:19:05.337 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:05 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T09:19:05.337 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:05 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:19:05.352 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:05 smithi149 ceph-mon[32592]: mds.? [v2:172.21.15.114:6838/1025403710,v1:172.21.15.114:6839/1025403710] up:boot 2024-04-24T09:19:05.352 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:05 smithi149 ceph-mon[32592]: fsmap 12 up:standby 2024-04-24T09:19:05.352 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:05 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T09:19:05.352 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:05 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:19:05.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:05 smithi090 ceph-mon[27825]: mds.? [v2:172.21.15.114:6838/1025403710,v1:172.21.15.114:6839/1025403710] up:boot 2024-04-24T09:19:05.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:05 smithi090 ceph-mon[27825]: fsmap 12 up:standby 2024-04-24T09:19:05.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:05 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T09:19:05.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:05 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:19:05.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.456+0000 7f0fe6a00640 1 -- 172.21.15.90:0/4047874416 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0fe002d620 con 0x7f0fe0104260 2024-04-24T09:19:05.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.457+0000 7f0fdcc00640 1 -- 172.21.15.90:0/4047874416 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10479 (secure 0 0 0) 0x7f0fd009c960 con 0x7f0fe0104260 2024-04-24T09:19:05.458 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:19:05.458 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/227251251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":227251251},{"type":"v1","addr":"172.21.15.90:6837","nonce":227251251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/1904617971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":1904617971},{"type":"v1","addr":"172.21.15.90:6839","nonce":1904617971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6833/219377141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":219377141},{"type":"v1","addr":"172.21.15.114:6833","nonce":219377141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24446,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2396995634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2396995634},{"type":"v1","addr":"172.21.15.90:6835","nonce":2396995634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/779642417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":779642417},{"type":"v1","addr":"172.21.15.149:6833","nonce":779642417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/2565340800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":2565340800},{"type":"v1","addr":"172.21.15.114:6835","nonce":2565340800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24500,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/875324280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":875324280},{"type":"v1","addr":"172.21.15.114:6837","nonce":875324280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/3157375624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3157375624},{"type":"v1","addr":"172.21.15.149:6835","nonce":3157375624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/2250310587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2250310587},{"type":"v1","addr":"172.21.15.149:6837","nonce":2250310587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24545,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6841/397593398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":397593398},{"type":"v1","addr":"172.21.15.90:6841","nonce":397593398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6839/652734263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":652734263},{"type":"v1","addr":"172.21.15.149:6839","nonce":652734263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6839/1025403710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":1025403710},{"type":"v1","addr":"172.21.15.114:6839","nonce":1025403710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-24T09:19:05.458 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 13 2024-04-24T09:19:05.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.460+0000 7f0fe6a00640 1 -- 172.21.15.90:0/4047874416 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f0fc40a3bb0 msgr2=0x7f0fc40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:05.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.460+0000 7f0fe6a00640 1 --2- 172.21.15.90:0/4047874416 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f0fc40a3bb0 0x7f0fc40a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0fe019ba30 tx=0x7f0fcc005f90 comp rx=0 tx=0).stop 2024-04-24T09:19:05.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.460+0000 7f0fe6a00640 1 -- 172.21.15.90:0/4047874416 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fe0104260 msgr2=0x7f0fe019a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:05.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.460+0000 7f0fe6a00640 1 --2- 172.21.15.90:0/4047874416 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fe0104260 0x7f0fe019a520 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0fd000e9c0 tx=0x7f0fd000ee80 comp rx=0 tx=0).stop 2024-04-24T09:19:05.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.460+0000 7f0fe6a00640 1 -- 172.21.15.90:0/4047874416 shutdown_connections 2024-04-24T09:19:05.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.460+0000 7f0fe6a00640 1 --2- 172.21.15.90:0/4047874416 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f0fc40a3bb0 0x7f0fc40a6070 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:05.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.460+0000 7f0fe6a00640 1 --2- 172.21.15.90:0/4047874416 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0fe0105e10 0x7f0fe01a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:05.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.460+0000 7f0fe6a00640 1 --2- 172.21.15.90:0/4047874416 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0fe0105450 0x7f0fe019aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:05.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.460+0000 7f0fe6a00640 1 --2- 172.21.15.90:0/4047874416 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fe0104260 0x7f0fe019a520 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:05.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.460+0000 7f0fe6a00640 1 -- 172.21.15.90:0/4047874416 >> 172.21.15.90:0/4047874416 conn(0x7f0fe00ff7f0 msgr2=0x7f0fe01086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:05.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.461+0000 7f0fe6a00640 1 -- 172.21.15.90:0/4047874416 shutdown_connections 2024-04-24T09:19:05.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.461+0000 7f0fe6a00640 1 -- 172.21.15.90:0/4047874416 wait complete. 2024-04-24T09:19:05.479 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T09:19:05.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.658+0000 7f35db200640 1 -- 172.21.15.90:0/772868939 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35d41040a0 msgr2=0x7f35d41044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:05.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.658+0000 7f35db200640 1 --2- 172.21.15.90:0/772868939 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35d41040a0 0x7f35d41044c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f35c8009900 tx=0x7f35c802a410 comp rx=0 tx=0).stop 2024-04-24T09:19:05.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.658+0000 7f35db200640 1 -- 172.21.15.90:0/772868939 shutdown_connections 2024-04-24T09:19:05.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.658+0000 7f35db200640 1 --2- 172.21.15.90:0/772868939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f35d4105c50 0x7f35d410c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:05.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.658+0000 7f35db200640 1 --2- 172.21.15.90:0/772868939 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f35d4105290 0x7f35d4105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:05.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.658+0000 7f35db200640 1 --2- 172.21.15.90:0/772868939 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35d41040a0 0x7f35d41044c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:05.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.658+0000 7f35db200640 1 -- 172.21.15.90:0/772868939 >> 172.21.15.90:0/772868939 conn(0x7f35d40ff610 msgr2=0x7f35d4101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:05.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.658+0000 7f35db200640 1 -- 172.21.15.90:0/772868939 shutdown_connections 2024-04-24T09:19:05.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.658+0000 7f35db200640 1 -- 172.21.15.90:0/772868939 wait complete. 2024-04-24T09:19:05.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.659+0000 7f35db200640 1 Processor -- start 2024-04-24T09:19:05.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.659+0000 7f35db200640 1 -- start start 2024-04-24T09:19:05.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.659+0000 7f35db200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f35d41040a0 0x7f35d419a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:05.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.660+0000 7f35db200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f35d4105290 0x7f35d419a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:05.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.660+0000 7f35db200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35d4105c50 0x7f35d41a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:05.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.660+0000 7f35db200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f35d410fd80 con 0x7f35d4105c50 2024-04-24T09:19:05.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.660+0000 7f35db200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f35d410fc00 con 0x7f35d4105290 2024-04-24T09:19:05.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.660+0000 7f35db200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f35d410ff00 con 0x7f35d41040a0 2024-04-24T09:19:05.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.660+0000 7f35d9e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f35d41040a0 0x7f35d419a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:05.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.660+0000 7f35d9e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f35d41040a0 0x7f35d419a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:41628/0 (socket says 172.21.15.90:41628) 2024-04-24T09:19:05.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.660+0000 7f35d9e00640 1 -- 172.21.15.90:0/1814106781 learned_addr learned my addr 172.21.15.90:0/1814106781 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:19:05.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.660+0000 7f35d9400640 1 --2- 172.21.15.90:0/1814106781 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f35d4105290 0x7f35d419a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:05.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.660+0000 7f35da800640 1 --2- 172.21.15.90:0/1814106781 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35d4105c50 0x7f35d41a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:05.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.660+0000 7f35d9e00640 1 -- 172.21.15.90:0/1814106781 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f35d4105290 msgr2=0x7f35d419a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:05.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.660+0000 7f35d9e00640 1 --2- 172.21.15.90:0/1814106781 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f35d4105290 0x7f35d419a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:05.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.660+0000 7f35d9e00640 1 -- 172.21.15.90:0/1814106781 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35d4105c50 msgr2=0x7f35d41a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:05.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.660+0000 7f35d9e00640 1 --2- 172.21.15.90:0/1814106781 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35d4105c50 0x7f35d41a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:05.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.660+0000 7f35d9e00640 1 -- 172.21.15.90:0/1814106781 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d41a1fd0 con 0x7f35d41040a0 2024-04-24T09:19:05.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.661+0000 7f35d9e00640 1 --2- 172.21.15.90:0/1814106781 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f35d41040a0 0x7f35d419a3b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f35c8031440 tx=0x7f35c8032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:05.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.661+0000 7f35cea00640 1 -- 172.21.15.90:0/1814106781 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c803f030 con 0x7f35d41040a0 2024-04-24T09:19:05.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.661+0000 7f35db200640 1 -- 172.21.15.90:0/1814106781 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35d41a2260 con 0x7f35d41040a0 2024-04-24T09:19:05.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.661+0000 7f35cea00640 1 -- 172.21.15.90:0/1814106781 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35c8003ce0 con 0x7f35d41040a0 2024-04-24T09:19:05.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.661+0000 7f35cea00640 1 -- 172.21.15.90:0/1814106781 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c8035070 con 0x7f35d41040a0 2024-04-24T09:19:05.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.661+0000 7f35db200640 1 -- 172.21.15.90:0/1814106781 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35d41a2750 con 0x7f35d41040a0 2024-04-24T09:19:05.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.662+0000 7f35db200640 1 -- 172.21.15.90:0/1814106781 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35d410b410 con 0x7f35d41040a0 2024-04-24T09:19:05.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.665+0000 7f35cea00640 1 -- 172.21.15.90:0/1814106781 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f35c8042030 con 0x7f35d41040a0 2024-04-24T09:19:05.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.666+0000 7f35cea00640 1 --2- 172.21.15.90:0/1814106781 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f35bc0a3bb0 0x7f35bc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:05.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.667+0000 7f35cea00640 1 -- 172.21.15.90:0/1814106781 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f35c80f0450 con 0x7f35d41040a0 2024-04-24T09:19:05.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.667+0000 7f35d9400640 1 --2- 172.21.15.90:0/1814106781 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f35bc0a3bb0 0x7f35bc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:05.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.668+0000 7f35d9400640 1 --2- 172.21.15.90:0/1814106781 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f35bc0a3bb0 0x7f35bc0a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f35c40095f0 tx=0x7f35c4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:05.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.671+0000 7f35cea00640 1 -- 172.21.15.90:0/1814106781 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f35c80bcbe0 con 0x7f35d41040a0 2024-04-24T09:19:05.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.876+0000 7f35db200640 1 -- 172.21.15.90:0/1814106781 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f35d4062c00 con 0x7f35d41040a0 2024-04-24T09:19:05.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.878+0000 7f35cea00640 1 -- 172.21.15.90:0/1814106781 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18468 (secure 0 0 0) 0x7f35c80c0ee0 con 0x7f35d41040a0 2024-04-24T09:19:05.878 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:19:05.879 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":82,"fsid":"30ee9be2-021a-11ef-bc93-c7b262605968","created":"2024-04-24T09:09:49.496019+0000","modified":"2024-04-24T09:18:11.472103+0000","last_up_change":"2024-04-24T09:18:10.455632+0000","last_in_change":"2024-04-24T09:17:50.600366+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T09:13:45.548774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13a59b9f-b46e-4ec4-90af-bb7a3d5c753b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6803","nonce":2864288066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6805","nonce":2864288066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6809","nonce":2864288066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6807","nonce":2864288066}]},"public_addr":"172.21.15.90:6803/2864288066","cluster_addr":"172.21.15.90:6805/2864288066","heartbeat_back_addr":"172.21.15.90:6809/2864288066","heartbeat_front_addr":"172.21.15.90:6807/2864288066","state":["exists","up"]},{"osd":1,"uuid":"1567ce8f-24e6-42cf-bf64-f6f0c534a9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6801","nonce":344222414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6803","nonce":344222414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6807","nonce":344222414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6805","nonce":344222414}]},"public_addr":"172.21.15.114:6801/344222414","cluster_addr":"172.21.15.114:6803/344222414","heartbeat_back_addr":"172.21.15.114:6807/344222414","heartbeat_front_addr":"172.21.15.114:6805/344222414","state":["exists","up"]},{"osd":2,"uuid":"f861b304-9c7d-4629-8522-37f33a747945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6801","nonce":3257067419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6803","nonce":3257067419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6807","nonce":3257067419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6805","nonce":3257067419}]},"public_addr":"172.21.15.149:6801/3257067419","cluster_addr":"172.21.15.149:6803/3257067419","heartbeat_back_addr":"172.21.15.149:6807/3257067419","heartbeat_front_addr":"172.21.15.149:6805/3257067419","state":["exists","up"]},{"osd":3,"uuid":"9377ab6e-5996-45fb-93a9-475a2e5f17a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6811","nonce":3068524970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6813","nonce":3068524970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6817","nonce":3068524970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6815","nonce":3068524970}]},"public_addr":"172.21.15.90:6811/3068524970","cluster_addr":"172.21.15.90:6813/3068524970","heartbeat_back_addr":"172.21.15.90:6817/3068524970","heartbeat_front_addr":"172.21.15.90:6815/3068524970","state":["exists","up"]},{"osd":4,"uuid":"e2ddd19e-e641-4d61-b342-02b5f12b1c4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6809","nonce":2782015522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6811","nonce":2782015522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6815","nonce":2782015522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6813","nonce":2782015522}]},"public_addr":"172.21.15.114:6809/2782015522","cluster_addr":"172.21.15.114:6811/2782015522","heartbeat_back_addr":"172.21.15.114:6815/2782015522","heartbeat_front_addr":"172.21.15.114:6813/2782015522","state":["exists","up"]},{"osd":5,"uuid":"b6ff9d10-5210-4b46-a192-6dcbf9d1f156","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6809","nonce":4266460479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6811","nonce":4266460479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6815","nonce":4266460479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6813","nonce":4266460479}]},"public_addr":"172.21.15.149:6809/4266460479","cluster_addr":"172.21.15.149:6811/4266460479","heartbeat_back_addr":"172.21.15.149:6815/4266460479","heartbeat_front_addr":"172.21.15.149:6813/4266460479","state":["exists","up"]},{"osd":6,"uuid":"4287b5a1-f953-421c-bf62-5e390bee834d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6819","nonce":1121461276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6821","nonce":1121461276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6825","nonce":1121461276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6823","nonce":1121461276}]},"public_addr":"172.21.15.90:6819/1121461276","cluster_addr":"172.21.15.90:6821/1121461276","heartbeat_back_addr":"172.21.15.90:6825/1121461276","heartbeat_front_addr":"172.21.15.90:6823/1121461276","state":["exists","up"]},{"osd":7,"uuid":"a54e507d-d6ba-4bf2-9a7f-a7b95c3859bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6817","nonce":3222546280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6819","nonce":3222546280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6823","nonce":3222546280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6821","nonce":3222546280}]},"public_addr":"172.21.15.114:6817/3222546280","cluster_addr":"172.21.15.114:6819/3222546280","heartbeat_back_addr":"172.21.15.114:6823/3222546280","heartbeat_front_addr":"172.21.15.114:6821/3222546280","state":["exists","up"]},{"osd":8,"uuid":"9950b010-5865-4109-ae37-582b1dbd2f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6817","nonce":157615149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6819","nonce":157615149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6823","nonce":157615149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6821","nonce":157615149}]},"public_addr":"172.21.15.149:6817/157615149","cluster_addr":"172.21.15.149:6819/157615149","heartbeat_back_addr":"172.21.15.149:6823/157615149","heartbeat_front_addr":"172.21.15.149:6821/157615149","state":["exists","up"]},{"osd":9,"uuid":"31e2635c-e234-421f-9757-592fd673c936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6827","nonce":1307318076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6829","nonce":1307318076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6833","nonce":1307318076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6831","nonce":1307318076}]},"public_addr":"172.21.15.90:6827/1307318076","cluster_addr":"172.21.15.90:6829/1307318076","heartbeat_back_addr":"172.21.15.90:6833/1307318076","heartbeat_front_addr":"172.21.15.90:6831/1307318076","state":["exists","up"]},{"osd":10,"uuid":"46a3b2a6-e6e5-4578-b4bb-f3e998b9bc95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6825","nonce":1719802140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6827","nonce":1719802140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6831","nonce":1719802140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6829","nonce":1719802140}]},"public_addr":"172.21.15.114:6825/1719802140","cluster_addr":"172.21.15.114:6827/1719802140","heartbeat_back_addr":"172.21.15.114:6831/1719802140","heartbeat_front_addr":"172.21.15.114:6829/1719802140","state":["exists","up"]},{"osd":11,"uuid":"ccd4456b-e947-46cf-a779-148ffa289b58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6825","nonce":1737275166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6827","nonce":1737275166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6831","nonce":1737275166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6829","nonce":1737275166}]},"public_addr":"172.21.15.149:6825/1737275166","cluster_addr":"172.21.15.149:6827/1737275166","heartbeat_back_addr":"172.21.15.149:6831/1737275166","heartbeat_front_addr":"172.21.15.149:6829/1737275166","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:12:53.015078+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:13:18.047053+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:13:41.935891+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:14:15.095388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:14:42.512355+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:15:08.113775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:15:43.828550+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:16:10.901909+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:16:36.422962+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:17:13.924446+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:17:42.814401+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:18:08.672793+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/3090852580":"2024-04-25T09:10:57.377655+0000","172.21.15.90:0/687197394":"2024-04-25T09:10:57.377655+0000","172.21.15.90:6801/1809038230":"2024-04-25T09:10:57.377655+0000","172.21.15.90:6800/1809038230":"2024-04-25T09:10:57.377655+0000","172.21.15.90:0/2964149071":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/3617388113":"2024-04-25T09:10:22.599674+0000","172.21.15.90:6801/3089140842":"2024-04-25T09:10:22.599674+0000","172.21.15.90:6800/3089140842":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/2840416149":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/305432710":"2024-04-25T09:10:57.377655+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T09:19:05.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.881+0000 7f35db200640 1 -- 172.21.15.90:0/1814106781 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f35bc0a3bb0 msgr2=0x7f35bc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:05.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.881+0000 7f35db200640 1 --2- 172.21.15.90:0/1814106781 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f35bc0a3bb0 0x7f35bc0a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f35c40095f0 tx=0x7f35c4005f90 comp rx=0 tx=0).stop 2024-04-24T09:19:05.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.881+0000 7f35db200640 1 -- 172.21.15.90:0/1814106781 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f35d41040a0 msgr2=0x7f35d419a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:05.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.881+0000 7f35db200640 1 --2- 172.21.15.90:0/1814106781 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f35d41040a0 0x7f35d419a3b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f35c8031440 tx=0x7f35c8032ce0 comp rx=0 tx=0).stop 2024-04-24T09:19:05.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.881+0000 7f35db200640 1 -- 172.21.15.90:0/1814106781 shutdown_connections 2024-04-24T09:19:05.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.881+0000 7f35db200640 1 --2- 172.21.15.90:0/1814106781 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f35bc0a3bb0 0x7f35bc0a6070 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:05.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.881+0000 7f35db200640 1 --2- 172.21.15.90:0/1814106781 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35d4105c50 0x7f35d41a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:05.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.881+0000 7f35db200640 1 --2- 172.21.15.90:0/1814106781 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f35d4105290 0x7f35d419a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:05.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.881+0000 7f35db200640 1 --2- 172.21.15.90:0/1814106781 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f35d41040a0 0x7f35d419a3b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:05.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.881+0000 7f35db200640 1 -- 172.21.15.90:0/1814106781 >> 172.21.15.90:0/1814106781 conn(0x7f35d40ff610 msgr2=0x7f35d41085e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:05.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.881+0000 7f35db200640 1 -- 172.21.15.90:0/1814106781 shutdown_connections 2024-04-24T09:19:05.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:05.881+0000 7f35db200640 1 -- 172.21.15.90:0/1814106781 wait complete. 2024-04-24T09:19:05.899 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T09:13:45.548774+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T09:19:05.900 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T09:19:06.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.104+0000 7f366d200640 1 -- 172.21.15.90:0/2284205682 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3668105e30 msgr2=0x7f366810c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:06.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.104+0000 7f366d200640 1 --2- 172.21.15.90:0/2284205682 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3668105e30 0x7f366810c6c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3660009be0 tx=0x7f366002a6d0 comp rx=0 tx=0).stop 2024-04-24T09:19:06.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.104+0000 7f366d200640 1 -- 172.21.15.90:0/2284205682 shutdown_connections 2024-04-24T09:19:06.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.104+0000 7f366d200640 1 --2- 172.21.15.90:0/2284205682 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3668105e30 0x7f366810c6c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:06.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.104+0000 7f366d200640 1 --2- 172.21.15.90:0/2284205682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3668105470 0x7f36681058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:06.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.104+0000 7f366d200640 1 --2- 172.21.15.90:0/2284205682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3668104280 0x7f36681046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:06.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.104+0000 7f366d200640 1 -- 172.21.15.90:0/2284205682 >> 172.21.15.90:0/2284205682 conn(0x7f36680ff7f0 msgr2=0x7f3668101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:06.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.104+0000 7f366d200640 1 -- 172.21.15.90:0/2284205682 shutdown_connections 2024-04-24T09:19:06.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.104+0000 7f366d200640 1 -- 172.21.15.90:0/2284205682 wait complete. 2024-04-24T09:19:06.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.105+0000 7f366d200640 1 Processor -- start 2024-04-24T09:19:06.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.106+0000 7f366d200640 1 -- start start 2024-04-24T09:19:06.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.106+0000 7f366d200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3668104280 0x7f366819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:06.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.106+0000 7f366d200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3668105470 0x7f366819aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:06.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.106+0000 7f366d200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3668105e30 0x7f36681a1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:06.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.106+0000 7f366d200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f366810ff10 con 0x7f3668105e30 2024-04-24T09:19:06.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.106+0000 7f366d200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f366810fd90 con 0x7f3668105470 2024-04-24T09:19:06.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.106+0000 7f366d200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3668110090 con 0x7f3668104280 2024-04-24T09:19:06.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.106+0000 7f3665e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3668104280 0x7f366819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:06.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.106+0000 7f3665e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3668104280 0x7f366819a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:39202/0 (socket says 172.21.15.90:39202) 2024-04-24T09:19:06.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.106+0000 7f3665e00640 1 -- 172.21.15.90:0/2234498924 learned_addr learned my addr 172.21.15.90:0/2234498924 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:19:06.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.106+0000 7f3665400640 1 --2- 172.21.15.90:0/2234498924 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3668105470 0x7f366819aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:06.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.107+0000 7f3666800640 1 --2- 172.21.15.90:0/2234498924 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3668105e30 0x7f36681a1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:06.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.107+0000 7f3665e00640 1 -- 172.21.15.90:0/2234498924 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3668105470 msgr2=0x7f366819aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:06.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.107+0000 7f3665e00640 1 --2- 172.21.15.90:0/2234498924 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3668105470 0x7f366819aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:06.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.107+0000 7f3665e00640 1 -- 172.21.15.90:0/2234498924 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3668105e30 msgr2=0x7f36681a1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:06.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.107+0000 7f3665e00640 1 --2- 172.21.15.90:0/2234498924 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3668105e30 0x7f36681a1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:06.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.107+0000 7f3665e00640 1 -- 172.21.15.90:0/2234498924 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36681a2100 con 0x7f3668104280 2024-04-24T09:19:06.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.107+0000 7f3665e00640 1 --2- 172.21.15.90:0/2234498924 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3668104280 0x7f366819a4e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f365800b2c0 tx=0x7f365800b6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:06.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.107+0000 7f365ea00640 1 -- 172.21.15.90:0/2234498924 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3658004020 con 0x7f3668104280 2024-04-24T09:19:06.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.107+0000 7f365ea00640 1 -- 172.21.15.90:0/2234498924 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36580041c0 con 0x7f3668104280 2024-04-24T09:19:06.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.107+0000 7f365ea00640 1 -- 172.21.15.90:0/2234498924 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365801b7e0 con 0x7f3668104280 2024-04-24T09:19:06.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.108+0000 7f366d200640 1 -- 172.21.15.90:0/2234498924 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36680764a0 con 0x7f3668104280 2024-04-24T09:19:06.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.108+0000 7f366d200640 1 -- 172.21.15.90:0/2234498924 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36680769a0 con 0x7f3668104280 2024-04-24T09:19:06.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.109+0000 7f366d200640 1 -- 172.21.15.90:0/2234498924 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f366810b5a0 con 0x7f3668104280 2024-04-24T09:19:06.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.112+0000 7f365ea00640 1 -- 172.21.15.90:0/2234498924 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3658024020 con 0x7f3668104280 2024-04-24T09:19:06.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.113+0000 7f365ea00640 1 --2- 172.21.15.90:0/2234498924 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f364c0a3bb0 0x7f364c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:06.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.113+0000 7f365ea00640 1 -- 172.21.15.90:0/2234498924 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f36580d7d70 con 0x7f3668104280 2024-04-24T09:19:06.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.113+0000 7f3665400640 1 --2- 172.21.15.90:0/2234498924 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f364c0a3bb0 0x7f364c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:06.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.114+0000 7f3665400640 1 --2- 172.21.15.90:0/2234498924 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f364c0a3bb0 0x7f364c0a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f366819b9f0 tx=0x7f3654009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:06.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.117+0000 7f365ea00640 1 -- 172.21.15.90:0/2234498924 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3658058ff0 con 0x7f3668104280 2024-04-24T09:19:06.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.319+0000 7f366d200640 1 -- 172.21.15.90:0/2234498924 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f366802d8b0 con 0x7f3668104280 2024-04-24T09:19:06.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.320+0000 7f365ea00640 1 -- 172.21.15.90:0/2234498924 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f36580232c0 con 0x7f3668104280 2024-04-24T09:19:06.320 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 1 2024-04-24T09:19:06.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.322+0000 7f366d200640 1 -- 172.21.15.90:0/2234498924 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f364c0a3bb0 msgr2=0x7f364c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:06.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.322+0000 7f366d200640 1 --2- 172.21.15.90:0/2234498924 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f364c0a3bb0 0x7f364c0a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f366819b9f0 tx=0x7f3654009260 comp rx=0 tx=0).stop 2024-04-24T09:19:06.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.322+0000 7f366d200640 1 -- 172.21.15.90:0/2234498924 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3668104280 msgr2=0x7f366819a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:06.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.322+0000 7f366d200640 1 --2- 172.21.15.90:0/2234498924 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3668104280 0x7f366819a4e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f365800b2c0 tx=0x7f365800b6c0 comp rx=0 tx=0).stop 2024-04-24T09:19:06.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.322+0000 7f366d200640 1 -- 172.21.15.90:0/2234498924 shutdown_connections 2024-04-24T09:19:06.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.322+0000 7f366d200640 1 --2- 172.21.15.90:0/2234498924 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f364c0a3bb0 0x7f364c0a6070 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:06.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.322+0000 7f366d200640 1 --2- 172.21.15.90:0/2234498924 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3668105e30 0x7f36681a1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:06.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.322+0000 7f366d200640 1 --2- 172.21.15.90:0/2234498924 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3668105470 0x7f366819aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:06.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.322+0000 7f366d200640 1 --2- 172.21.15.90:0/2234498924 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3668104280 0x7f366819a4e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:06.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.322+0000 7f366d200640 1 -- 172.21.15.90:0/2234498924 >> 172.21.15.90:0/2234498924 conn(0x7f36680ff7f0 msgr2=0x7f3668108770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:06.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.322+0000 7f366d200640 1 -- 172.21.15.90:0/2234498924 shutdown_connections 2024-04-24T09:19:06.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.322+0000 7f366d200640 1 -- 172.21.15.90:0/2234498924 wait complete. 2024-04-24T09:19:06.342 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-24T09:19:06.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.523+0000 7fdb8dc00640 1 -- 172.21.15.90:0/3811782613 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb88105450 msgr2=0x7fdb881058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:06.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.523+0000 7fdb8dc00640 1 --2- 172.21.15.90:0/3811782613 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb88105450 0x7fdb881058d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdb70009900 tx=0x7fdb7002a3f0 comp rx=0 tx=0).stop 2024-04-24T09:19:06.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.523+0000 7fdb8dc00640 1 -- 172.21.15.90:0/3811782613 shutdown_connections 2024-04-24T09:19:06.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.523+0000 7fdb8dc00640 1 --2- 172.21.15.90:0/3811782613 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb88105e10 0x7fdb8810c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:06.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.523+0000 7fdb8dc00640 1 --2- 172.21.15.90:0/3811782613 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb88105450 0x7fdb881058d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:06.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.523+0000 7fdb8dc00640 1 --2- 172.21.15.90:0/3811782613 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb88104260 0x7fdb88104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:06.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.523+0000 7fdb8dc00640 1 -- 172.21.15.90:0/3811782613 >> 172.21.15.90:0/3811782613 conn(0x7fdb880ff7f0 msgr2=0x7fdb88101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:06.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.523+0000 7fdb8dc00640 1 -- 172.21.15.90:0/3811782613 shutdown_connections 2024-04-24T09:19:06.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.523+0000 7fdb8dc00640 1 -- 172.21.15.90:0/3811782613 wait complete. 2024-04-24T09:19:06.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.525+0000 7fdb8dc00640 1 Processor -- start 2024-04-24T09:19:06.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.525+0000 7fdb8dc00640 1 -- start start 2024-04-24T09:19:06.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.525+0000 7fdb8dc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb88104260 0x7fdb8819a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:06.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.525+0000 7fdb8dc00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb88105450 0x7fdb8819aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:06.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.525+0000 7fdb8dc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb88105e10 0x7fdb881a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:06.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.525+0000 7fdb8dc00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8810fef0 con 0x7fdb88105e10 2024-04-24T09:19:06.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.525+0000 7fdb8dc00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8810fd70 con 0x7fdb88105450 2024-04-24T09:19:06.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.525+0000 7fdb8dc00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fdb88110070 con 0x7fdb88104260 2024-04-24T09:19:06.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.525+0000 7fdb86a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb88104260 0x7fdb8819a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:06.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.525+0000 7fdb86a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb88104260 0x7fdb8819a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:39218/0 (socket says 172.21.15.90:39218) 2024-04-24T09:19:06.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.526+0000 7fdb86a00640 1 -- 172.21.15.90:0/4127635389 learned_addr learned my addr 172.21.15.90:0/4127635389 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:19:06.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.526+0000 7fdb87400640 1 --2- 172.21.15.90:0/4127635389 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb88105e10 0x7fdb881a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:06.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.526+0000 7fdb7fe00640 1 --2- 172.21.15.90:0/4127635389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb88105450 0x7fdb8819aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:06.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.526+0000 7fdb86a00640 1 -- 172.21.15.90:0/4127635389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb88105450 msgr2=0x7fdb8819aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:06.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.526+0000 7fdb86a00640 1 --2- 172.21.15.90:0/4127635389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb88105450 0x7fdb8819aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:06.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.526+0000 7fdb86a00640 1 -- 172.21.15.90:0/4127635389 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb88105e10 msgr2=0x7fdb881a1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:06.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.526+0000 7fdb86a00640 1 --2- 172.21.15.90:0/4127635389 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb88105e10 0x7fdb881a1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:06.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.526+0000 7fdb86a00640 1 -- 172.21.15.90:0/4127635389 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb881a2180 con 0x7fdb88104260 2024-04-24T09:19:06.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.526+0000 7fdb86a00640 1 --2- 172.21.15.90:0/4127635389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb88104260 0x7fdb8819a560 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdb7800e9c0 tx=0x7fdb7800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:06.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.527+0000 7fdb7f400640 1 -- 172.21.15.90:0/4127635389 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7800cde0 con 0x7fdb88104260 2024-04-24T09:19:06.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.527+0000 7fdb7f400640 1 -- 172.21.15.90:0/4127635389 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb78010910 con 0x7fdb88104260 2024-04-24T09:19:06.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.527+0000 7fdb7f400640 1 -- 172.21.15.90:0/4127635389 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb780189a0 con 0x7fdb88104260 2024-04-24T09:19:06.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.527+0000 7fdb8dc00640 1 -- 172.21.15.90:0/4127635389 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb881a2470 con 0x7fdb88104260 2024-04-24T09:19:06.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.527+0000 7fdb8dc00640 1 -- 172.21.15.90:0/4127635389 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb881a2940 con 0x7fdb88104260 2024-04-24T09:19:06.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.528+0000 7fdb8dc00640 1 -- 172.21.15.90:0/4127635389 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb8810b580 con 0x7fdb88104260 2024-04-24T09:19:06.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.530+0000 7fdb7f400640 1 -- 172.21.15.90:0/4127635389 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdb7801f020 con 0x7fdb88104260 2024-04-24T09:19:06.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.531+0000 7fdb7f400640 1 --2- 172.21.15.90:0/4127635389 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fdb6c0a3c30 0x7fdb6c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:06.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.531+0000 7fdb7f400640 1 -- 172.21.15.90:0/4127635389 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fdb780d0140 con 0x7fdb88104260 2024-04-24T09:19:06.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.531+0000 7fdb7fe00640 1 --2- 172.21.15.90:0/4127635389 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fdb6c0a3c30 0x7fdb6c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:06.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.532+0000 7fdb7fe00640 1 --2- 172.21.15.90:0/4127635389 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fdb6c0a3c30 0x7fdb6c0a60f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fdb700314e0 tx=0x7fdb70000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:06.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.535+0000 7fdb7f400640 1 -- 172.21.15.90:0/4127635389 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdb7809cf20 con 0x7fdb88104260 2024-04-24T09:19:06.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:06 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/4047874416' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:19:06.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:06 smithi090 ceph-mon[27825]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:19:06.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:06 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/1814106781' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:19:06.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:06 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:06.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:06 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:06.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:06 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:06.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:06 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:06.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:06 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/4047874416' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:19:06.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:06 smithi149 ceph-mon[32592]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:19:06.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:06 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/1814106781' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:19:06.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:06 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:06.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:06 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:06.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:06 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:06.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:06 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:06.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:06 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/4047874416' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:19:06.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:06 smithi114 ceph-mon[32501]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:19:06.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:06 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/1814106781' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:19:06.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:06 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:06.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:06 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:06.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:06 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:06.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:06 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:06.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.756+0000 7fdb8dc00640 1 -- 172.21.15.90:0/4127635389 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fdb88062c00 con 0x7fdb88104260 2024-04-24T09:19:06.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.757+0000 7fdb7f400640 1 -- 172.21.15.90:0/4127635389 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7fdb7809c820 con 0x7fdb88104260 2024-04-24T09:19:06.757 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:19:06.757 INFO:teuthology.orchestra.run.smithi090.stdout:[ 2024-04-24T09:19:06.757 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:19:06.757 INFO:teuthology.orchestra.run.smithi090.stdout: "poolnum": 1, 2024-04-24T09:19:06.757 INFO:teuthology.orchestra.run.smithi090.stdout: "poolname": ".mgr" 2024-04-24T09:19:06.757 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:19:06.757 INFO:teuthology.orchestra.run.smithi090.stdout:] 2024-04-24T09:19:06.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.760+0000 7fdb8dc00640 1 -- 172.21.15.90:0/4127635389 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fdb6c0a3c30 msgr2=0x7fdb6c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:06.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.760+0000 7fdb8dc00640 1 --2- 172.21.15.90:0/4127635389 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fdb6c0a3c30 0x7fdb6c0a60f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fdb700314e0 tx=0x7fdb70000bc0 comp rx=0 tx=0).stop 2024-04-24T09:19:06.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.760+0000 7fdb8dc00640 1 -- 172.21.15.90:0/4127635389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb88104260 msgr2=0x7fdb8819a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:06.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.760+0000 7fdb8dc00640 1 --2- 172.21.15.90:0/4127635389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb88104260 0x7fdb8819a560 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdb7800e9c0 tx=0x7fdb7800ee80 comp rx=0 tx=0).stop 2024-04-24T09:19:06.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.760+0000 7fdb8dc00640 1 -- 172.21.15.90:0/4127635389 shutdown_connections 2024-04-24T09:19:06.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.760+0000 7fdb8dc00640 1 --2- 172.21.15.90:0/4127635389 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fdb6c0a3c30 0x7fdb6c0a60f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:06.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.760+0000 7fdb8dc00640 1 --2- 172.21.15.90:0/4127635389 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb88105e10 0x7fdb881a1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:06.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.760+0000 7fdb8dc00640 1 --2- 172.21.15.90:0/4127635389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb88105450 0x7fdb8819aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:06.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.760+0000 7fdb8dc00640 1 --2- 172.21.15.90:0/4127635389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb88104260 0x7fdb8819a560 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:06.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.760+0000 7fdb8dc00640 1 -- 172.21.15.90:0/4127635389 >> 172.21.15.90:0/4127635389 conn(0x7fdb880ff7f0 msgr2=0x7fdb88108750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:06.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.760+0000 7fdb8dc00640 1 -- 172.21.15.90:0/4127635389 shutdown_connections 2024-04-24T09:19:06.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.760+0000 7fdb8dc00640 1 -- 172.21.15.90:0/4127635389 wait complete. 2024-04-24T09:19:06.777 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-24T09:19:06.777 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-24T09:19:06.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.962+0000 7ff0eec00640 1 -- 172.21.15.90:0/2544215961 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0e8105e10 msgr2=0x7ff0e810c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:06.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.962+0000 7ff0eec00640 1 --2- 172.21.15.90:0/2544215961 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0e8105e10 0x7ff0e810c6a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0009ca0 tx=0x7ff0e002a8c0 comp rx=0 tx=0).stop 2024-04-24T09:19:06.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.962+0000 7ff0eec00640 1 -- 172.21.15.90:0/2544215961 shutdown_connections 2024-04-24T09:19:06.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.962+0000 7ff0eec00640 1 --2- 172.21.15.90:0/2544215961 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0e8105e10 0x7ff0e810c6a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:06.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.962+0000 7ff0eec00640 1 --2- 172.21.15.90:0/2544215961 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0e8105450 0x7ff0e81058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:06.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.962+0000 7ff0eec00640 1 --2- 172.21.15.90:0/2544215961 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0e8104260 0x7ff0e8104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:06.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.962+0000 7ff0eec00640 1 -- 172.21.15.90:0/2544215961 >> 172.21.15.90:0/2544215961 conn(0x7ff0e80ff7b0 msgr2=0x7ff0e8101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:06.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.963+0000 7ff0eec00640 1 -- 172.21.15.90:0/2544215961 shutdown_connections 2024-04-24T09:19:06.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.963+0000 7ff0eec00640 1 -- 172.21.15.90:0/2544215961 wait complete. 2024-04-24T09:19:06.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.963+0000 7ff0eec00640 1 Processor -- start 2024-04-24T09:19:06.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.964+0000 7ff0eec00640 1 -- start start 2024-04-24T09:19:06.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.964+0000 7ff0eec00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0e8104260 0x7ff0e819a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:06.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.964+0000 7ff0eec00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0e8105450 0x7ff0e819aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:06.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.964+0000 7ff0eec00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0e8105e10 0x7ff0e81a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:06.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.964+0000 7ff0eec00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e810ff40 con 0x7ff0e8105450 2024-04-24T09:19:06.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.964+0000 7ff0eec00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e810fdc0 con 0x7ff0e8105e10 2024-04-24T09:19:06.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.964+0000 7ff0eec00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e81100c0 con 0x7ff0e8104260 2024-04-24T09:19:06.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.964+0000 7ff0e7400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0e8105450 0x7ff0e819aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:06.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.964+0000 7ff0ecc00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0e8105e10 0x7ff0e81a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:06.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.964+0000 7ff0e7400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0e8105450 0x7ff0e819aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55678/0 (socket says 172.21.15.90:55678) 2024-04-24T09:19:06.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.964+0000 7ff0e7400640 1 -- 172.21.15.90:0/694959122 learned_addr learned my addr 172.21.15.90:0/694959122 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:19:06.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.964+0000 7ff0e7e00640 1 --2- 172.21.15.90:0/694959122 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0e8104260 0x7ff0e819a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:06.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.965+0000 7ff0e7400640 1 -- 172.21.15.90:0/694959122 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0e8104260 msgr2=0x7ff0e819a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:06.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.965+0000 7ff0e7400640 1 --2- 172.21.15.90:0/694959122 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0e8104260 0x7ff0e819a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:06.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.965+0000 7ff0e7400640 1 -- 172.21.15.90:0/694959122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0e8105e10 msgr2=0x7ff0e81a1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:06.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.965+0000 7ff0e7400640 1 --2- 172.21.15.90:0/694959122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0e8105e10 0x7ff0e81a1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:06.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.965+0000 7ff0e7400640 1 -- 172.21.15.90:0/694959122 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0e81a2130 con 0x7ff0e8105450 2024-04-24T09:19:06.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.965+0000 7ff0ecc00640 1 --2- 172.21.15.90:0/694959122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0e8105e10 0x7ff0e81a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:19:06.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.965+0000 7ff0e7400640 1 --2- 172.21.15.90:0/694959122 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0e8105450 0x7ff0e819aa50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff0d400e9f0 tx=0x7ff0d400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:06.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.965+0000 7ff0e4c00640 1 -- 172.21.15.90:0/694959122 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d400cd80 con 0x7ff0e8105450 2024-04-24T09:19:06.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.965+0000 7ff0e4c00640 1 -- 172.21.15.90:0/694959122 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0d4005500 con 0x7ff0e8105450 2024-04-24T09:19:06.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.965+0000 7ff0eec00640 1 -- 172.21.15.90:0/694959122 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0e81a2420 con 0x7ff0e8105450 2024-04-24T09:19:06.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.965+0000 7ff0e4c00640 1 -- 172.21.15.90:0/694959122 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d4010690 con 0x7ff0e8105450 2024-04-24T09:19:06.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.965+0000 7ff0eec00640 1 -- 172.21.15.90:0/694959122 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0e81a2920 con 0x7ff0e8105450 2024-04-24T09:19:06.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.967+0000 7ff0eec00640 1 -- 172.21.15.90:0/694959122 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0e810b520 con 0x7ff0e8105450 2024-04-24T09:19:06.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.969+0000 7ff0e4c00640 1 -- 172.21.15.90:0/694959122 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff0d40046c0 con 0x7ff0e8105450 2024-04-24T09:19:06.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.971+0000 7ff0e4c00640 1 --2- 172.21.15.90:0/694959122 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7ff0cc0a3bb0 0x7ff0cc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:06.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.971+0000 7ff0e4c00640 1 -- 172.21.15.90:0/694959122 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ff0d40d0600 con 0x7ff0e8105450 2024-04-24T09:19:06.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.971+0000 7ff0e7e00640 1 --2- 172.21.15.90:0/694959122 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7ff0cc0a3bb0 0x7ff0cc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:06.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.972+0000 7ff0e7e00640 1 --2- 172.21.15.90:0/694959122 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7ff0cc0a3bb0 0x7ff0cc0a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff0e81052b0 tx=0x7ff0d8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:06.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:06.973+0000 7ff0e4c00640 1 -- 172.21.15.90:0/694959122 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff0d4053ff0 con 0x7ff0e8105450 2024-04-24T09:19:07.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:07.172+0000 7ff0eec00640 1 -- 172.21.15.90:0/694959122 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ff0e802d620 con 0x7ff0e8105450 2024-04-24T09:19:07.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:07 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/2234498924' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:19:07.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:07 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/4127635389' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-24T09:19:07.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:07 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:07.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:07 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:07.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:07 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:19:07.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:07 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:19:07.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:07 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:07.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:07 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/694959122' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-24T09:19:07.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:07 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:19:07.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:07 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/2234498924' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:19:07.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:07 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/4127635389' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-24T09:19:07.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:07 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:07.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:07 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:07.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:07 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:19:07.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:07 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:19:07.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:07 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:07.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:07 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/694959122' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-24T09:19:07.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:07 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:19:07.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:07 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/2234498924' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:19:07.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:07 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/4127635389' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-24T09:19:07.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:07 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:07.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:07 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:07.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:07 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:19:07.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:07 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:19:07.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:07 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:07.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:07 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/694959122' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-24T09:19:07.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:07 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:19:07.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:07.930+0000 7ff0e4c00640 1 -- 172.21.15.90:0/694959122 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7ff0d409d3e0 con 0x7ff0e8105450 2024-04-24T09:19:07.930 INFO:teuthology.orchestra.run.smithi090.stderr:pool 'cephfs_metadata' created 2024-04-24T09:19:07.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:07.933+0000 7ff0eec00640 1 -- 172.21.15.90:0/694959122 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7ff0cc0a3bb0 msgr2=0x7ff0cc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:07.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:07.933+0000 7ff0eec00640 1 --2- 172.21.15.90:0/694959122 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7ff0cc0a3bb0 0x7ff0cc0a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff0e81052b0 tx=0x7ff0d8005f90 comp rx=0 tx=0).stop 2024-04-24T09:19:07.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:07.934+0000 7ff0eec00640 1 -- 172.21.15.90:0/694959122 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0e8105450 msgr2=0x7ff0e819aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:07.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:07.934+0000 7ff0eec00640 1 --2- 172.21.15.90:0/694959122 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0e8105450 0x7ff0e819aa50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff0d400e9f0 tx=0x7ff0d400edf0 comp rx=0 tx=0).stop 2024-04-24T09:19:07.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:07.934+0000 7ff0eec00640 1 -- 172.21.15.90:0/694959122 shutdown_connections 2024-04-24T09:19:07.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:07.934+0000 7ff0eec00640 1 --2- 172.21.15.90:0/694959122 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7ff0cc0a3bb0 0x7ff0cc0a6070 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:07.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:07.934+0000 7ff0eec00640 1 --2- 172.21.15.90:0/694959122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0e8105e10 0x7ff0e81a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:07.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:07.934+0000 7ff0eec00640 1 --2- 172.21.15.90:0/694959122 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0e8105450 0x7ff0e819aa50 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:07.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:07.934+0000 7ff0eec00640 1 --2- 172.21.15.90:0/694959122 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0e8104260 0x7ff0e819a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:07.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:07.934+0000 7ff0eec00640 1 -- 172.21.15.90:0/694959122 >> 172.21.15.90:0/694959122 conn(0x7ff0e80ff7b0 msgr2=0x7ff0e81086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:07.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:07.935+0000 7ff0eec00640 1 -- 172.21.15.90:0/694959122 shutdown_connections 2024-04-24T09:19:07.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:07.935+0000 7ff0eec00640 1 -- 172.21.15.90:0/694959122 wait complete. 2024-04-24T09:19:07.957 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-24T09:19:08.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.141+0000 7fcdc8a00640 1 -- 172.21.15.90:0/1055166758 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcdc41052c0 msgr2=0x7fcdc41076b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:08.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.141+0000 7fcdc8a00640 1 --2- 172.21.15.90:0/1055166758 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcdc41052c0 0x7fcdc41076b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0009860 tx=0x7fcdb002a340 comp rx=0 tx=0).stop 2024-04-24T09:19:08.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.141+0000 7fcdc8a00640 1 -- 172.21.15.90:0/1055166758 shutdown_connections 2024-04-24T09:19:08.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.141+0000 7fcdc8a00640 1 --2- 172.21.15.90:0/1055166758 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcdc4107bf0 0x7fcdc410a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:08.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.141+0000 7fcdc8a00640 1 --2- 172.21.15.90:0/1055166758 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcdc41052c0 0x7fcdc41076b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:08.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.141+0000 7fcdc8a00640 1 --2- 172.21.15.90:0/1055166758 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcdc4102990 0x7fcdc4104d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:08.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.141+0000 7fcdc8a00640 1 -- 172.21.15.90:0/1055166758 >> 172.21.15.90:0/1055166758 conn(0x7fcdc40fc3a0 msgr2=0x7fcdc40fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:08.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.141+0000 7fcdc8a00640 1 -- 172.21.15.90:0/1055166758 shutdown_connections 2024-04-24T09:19:08.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.141+0000 7fcdc8a00640 1 -- 172.21.15.90:0/1055166758 wait complete. 2024-04-24T09:19:08.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.142+0000 7fcdc8a00640 1 Processor -- start 2024-04-24T09:19:08.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.142+0000 7fcdc8a00640 1 -- start start 2024-04-24T09:19:08.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.142+0000 7fcdc8a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcdc4102990 0x7fcdc419a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:08.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.142+0000 7fcdc8a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcdc41052c0 0x7fcdc419a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:08.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.142+0000 7fcdc8a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcdc4107bf0 0x7fcdc41a1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:08.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.142+0000 7fcdc8a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc410db80 con 0x7fcdc4102990 2024-04-24T09:19:08.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.142+0000 7fcdc8a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc410da00 con 0x7fcdc4107bf0 2024-04-24T09:19:08.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.142+0000 7fcdc8a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc410dd00 con 0x7fcdc41052c0 2024-04-24T09:19:08.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.143+0000 7fcdc2800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcdc4102990 0x7fcdc419a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:08.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.143+0000 7fcdc1e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcdc41052c0 0x7fcdc419a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:08.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.143+0000 7fcdc1e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcdc41052c0 0x7fcdc419a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:39260/0 (socket says 172.21.15.90:39260) 2024-04-24T09:19:08.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.143+0000 7fcdc3200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcdc4107bf0 0x7fcdc41a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:08.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.143+0000 7fcdc2800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcdc4102990 0x7fcdc419a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55682/0 (socket says 172.21.15.90:55682) 2024-04-24T09:19:08.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.143+0000 7fcdc1e00640 1 -- 172.21.15.90:0/3418848592 learned_addr learned my addr 172.21.15.90:0/3418848592 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:19:08.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.143+0000 7fcdc1e00640 1 -- 172.21.15.90:0/3418848592 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcdc4107bf0 msgr2=0x7fcdc41a1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:08.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.143+0000 7fcdc1e00640 1 --2- 172.21.15.90:0/3418848592 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcdc4107bf0 0x7fcdc41a1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:08.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.143+0000 7fcdc1e00640 1 -- 172.21.15.90:0/3418848592 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcdc4102990 msgr2=0x7fcdc419a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:08.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.143+0000 7fcdc1e00640 1 --2- 172.21.15.90:0/3418848592 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcdc4102990 0x7fcdc419a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:08.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.143+0000 7fcdc1e00640 1 -- 172.21.15.90:0/3418848592 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdc41a1ff0 con 0x7fcdc41052c0 2024-04-24T09:19:08.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.143+0000 7fcdc1e00640 1 --2- 172.21.15.90:0/3418848592 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcdc41052c0 0x7fcdc419a910 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0035040 tx=0x7fcdb0033d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:08.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.144+0000 7fcdbb400640 1 -- 172.21.15.90:0/3418848592 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0037040 con 0x7fcdc41052c0 2024-04-24T09:19:08.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.144+0000 7fcdbb400640 1 -- 172.21.15.90:0/3418848592 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdb0030850 con 0x7fcdc41052c0 2024-04-24T09:19:08.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.144+0000 7fcdbb400640 1 -- 172.21.15.90:0/3418848592 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0004920 con 0x7fcdc41052c0 2024-04-24T09:19:08.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.144+0000 7fcdc8a00640 1 -- 172.21.15.90:0/3418848592 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdc41a2280 con 0x7fcdc41052c0 2024-04-24T09:19:08.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.144+0000 7fcdc8a00640 1 -- 172.21.15.90:0/3418848592 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdc41a2720 con 0x7fcdc41052c0 2024-04-24T09:19:08.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.145+0000 7fcdc8a00640 1 -- 172.21.15.90:0/3418848592 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdc4109740 con 0x7fcdc41052c0 2024-04-24T09:19:08.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.148+0000 7fcdbb400640 1 -- 172.21.15.90:0/3418848592 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fcdb0004050 con 0x7fcdc41052c0 2024-04-24T09:19:08.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.149+0000 7fcdbb400640 1 --2- 172.21.15.90:0/3418848592 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fcda80a3c30 0x7fcda80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:08.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.150+0000 7fcdbb400640 1 -- 172.21.15.90:0/3418848592 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7497+0+0 (secure 0 0 0) 0x7fcdb00eff60 con 0x7fcdc41052c0 2024-04-24T09:19:08.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.150+0000 7fcdc2800640 1 --2- 172.21.15.90:0/3418848592 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fcda80a3c30 0x7fcda80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:08.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.150+0000 7fcdc2800640 1 --2- 172.21.15.90:0/3418848592 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fcda80a3c30 0x7fcda80a60f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcdb40045f0 tx=0x7fcdb400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:08.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.154+0000 7fcdbb400640 1 -- 172.21.15.90:0/3418848592 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcdb00bcc70 con 0x7fcdc41052c0 2024-04-24T09:19:08.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.358+0000 7fcdc8a00640 1 -- 172.21.15.90:0/3418848592 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fcdc4062c00 con 0x7fcdc41052c0 2024-04-24T09:19:08.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:08 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:08.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:08 smithi090 ceph-mon[27825]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:19:08.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:08 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/694959122' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-24T09:19:08.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:08 smithi090 ceph-mon[27825]: osdmap e83: 12 total, 12 up, 12 in 2024-04-24T09:19:08.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:08 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/3418848592' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-24T09:19:08.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:08 smithi090 ceph-mon[27825]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-24T09:19:08.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:08 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:08.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:08 smithi149 ceph-mon[32592]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:19:08.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:08 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/694959122' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-24T09:19:08.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:08 smithi149 ceph-mon[32592]: osdmap e83: 12 total, 12 up, 12 in 2024-04-24T09:19:08.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:08 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/3418848592' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-24T09:19:08.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:08 smithi149 ceph-mon[32592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-24T09:19:08.925 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:08 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:08.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:08 smithi114 ceph-mon[32501]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:19:08.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:08 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/694959122' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-24T09:19:08.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:08 smithi114 ceph-mon[32501]: osdmap e83: 12 total, 12 up, 12 in 2024-04-24T09:19:08.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:08 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/3418848592' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-24T09:19:08.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:08 smithi114 ceph-mon[32501]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-24T09:19:08.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.941+0000 7fcdbb400640 1 -- 172.21.15.90:0/3418848592 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v84) ==== 170+0+0 (secure 0 0 0) 0x7fcdb00bc570 con 0x7fcdc41052c0 2024-04-24T09:19:08.941 INFO:teuthology.orchestra.run.smithi090.stderr:pool 'cephfs_data' created 2024-04-24T09:19:08.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.947+0000 7fcdc8a00640 1 -- 172.21.15.90:0/3418848592 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fcda80a3c30 msgr2=0x7fcda80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:08.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.947+0000 7fcdc8a00640 1 --2- 172.21.15.90:0/3418848592 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fcda80a3c30 0x7fcda80a60f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcdb40045f0 tx=0x7fcdb400b390 comp rx=0 tx=0).stop 2024-04-24T09:19:08.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.947+0000 7fcdc8a00640 1 -- 172.21.15.90:0/3418848592 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcdc41052c0 msgr2=0x7fcdc419a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:08.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.947+0000 7fcdc8a00640 1 --2- 172.21.15.90:0/3418848592 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcdc41052c0 0x7fcdc419a910 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0035040 tx=0x7fcdb0033d90 comp rx=0 tx=0).stop 2024-04-24T09:19:08.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.947+0000 7fcdc8a00640 1 -- 172.21.15.90:0/3418848592 shutdown_connections 2024-04-24T09:19:08.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.947+0000 7fcdc8a00640 1 --2- 172.21.15.90:0/3418848592 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fcda80a3c30 0x7fcda80a60f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:08.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.947+0000 7fcdc8a00640 1 --2- 172.21.15.90:0/3418848592 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcdc4107bf0 0x7fcdc41a1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:08.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.947+0000 7fcdc8a00640 1 --2- 172.21.15.90:0/3418848592 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcdc41052c0 0x7fcdc419a910 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0035040 tx=0x7fcdb0033d90 comp rx=0 tx=0).stop 2024-04-24T09:19:08.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.947+0000 7fcdc8a00640 1 --2- 172.21.15.90:0/3418848592 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcdc4102990 0x7fcdc419a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:08.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.947+0000 7fcdc8a00640 1 -- 172.21.15.90:0/3418848592 >> 172.21.15.90:0/3418848592 conn(0x7fcdc40fc3a0 msgr2=0x7fcdc4102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:08.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.948+0000 7fcdc8a00640 1 -- 172.21.15.90:0/3418848592 shutdown_connections 2024-04-24T09:19:08.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:08.948+0000 7fcdc8a00640 1 -- 172.21.15.90:0/3418848592 wait complete. 2024-04-24T09:19:08.980 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-24T09:19:09.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.174+0000 7febb4a00640 1 -- 172.21.15.90:0/1985734482 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febb0107d20 msgr2=0x7febb010a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:09.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.174+0000 7febb4a00640 1 --2- 172.21.15.90:0/1985734482 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febb0107d20 0x7febb010a170 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7feba8009ca0 tx=0x7feba802a8c0 comp rx=0 tx=0).stop 2024-04-24T09:19:09.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.174+0000 7febb4a00640 1 -- 172.21.15.90:0/1985734482 shutdown_connections 2024-04-24T09:19:09.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.174+0000 7febb4a00640 1 --2- 172.21.15.90:0/1985734482 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febb0107d20 0x7febb010a170 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:09.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.174+0000 7febb4a00640 1 --2- 172.21.15.90:0/1985734482 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7febb01053f0 0x7febb01077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:09.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.174+0000 7febb4a00640 1 --2- 172.21.15.90:0/1985734482 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7febb0102ac0 0x7febb0104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:09.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.174+0000 7febb4a00640 1 -- 172.21.15.90:0/1985734482 >> 172.21.15.90:0/1985734482 conn(0x7febb00fc4b0 msgr2=0x7febb00fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:09.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.174+0000 7febb4a00640 1 -- 172.21.15.90:0/1985734482 shutdown_connections 2024-04-24T09:19:09.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.174+0000 7febb4a00640 1 -- 172.21.15.90:0/1985734482 wait complete. 2024-04-24T09:19:09.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.175+0000 7febb4a00640 1 Processor -- start 2024-04-24T09:19:09.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.175+0000 7febb4a00640 1 -- start start 2024-04-24T09:19:09.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.175+0000 7febb4a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7febb0102ac0 0x7febb019a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:09.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.175+0000 7febb4a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7febb01053f0 0x7febb019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:09.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.175+0000 7febb4a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febb0107d20 0x7febb01a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:09.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.175+0000 7febb4a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7febb010dcd0 con 0x7febb0102ac0 2024-04-24T09:19:09.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.175+0000 7febb4a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7febb010db50 con 0x7febb01053f0 2024-04-24T09:19:09.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.175+0000 7febb4a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7febb010de50 con 0x7febb0107d20 2024-04-24T09:19:09.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.176+0000 7febade00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7febb0102ac0 0x7febb019a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:09.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.176+0000 7feba7e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7febb01053f0 0x7febb019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:09.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.176+0000 7febade00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7febb0102ac0 0x7febb019a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55700/0 (socket says 172.21.15.90:55700) 2024-04-24T09:19:09.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.176+0000 7febade00640 1 -- 172.21.15.90:0/665273441 learned_addr learned my addr 172.21.15.90:0/665273441 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:19:09.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.176+0000 7feba7e00640 1 -- 172.21.15.90:0/665273441 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febb0107d20 msgr2=0x7febb01a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:09.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.176+0000 7feba7e00640 1 --2- 172.21.15.90:0/665273441 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febb0107d20 0x7febb01a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:09.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.176+0000 7feba7e00640 1 -- 172.21.15.90:0/665273441 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7febb0102ac0 msgr2=0x7febb019a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:09.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.176+0000 7feba7e00640 1 --2- 172.21.15.90:0/665273441 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7febb0102ac0 0x7febb019a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:09.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.176+0000 7feba7e00640 1 -- 172.21.15.90:0/665273441 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febb01a2160 con 0x7febb01053f0 2024-04-24T09:19:09.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.177+0000 7feba7e00640 1 --2- 172.21.15.90:0/665273441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7febb01053f0 0x7febb019aa80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7feb9800e9c0 tx=0x7feb9800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:09.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.177+0000 7feba6a00640 1 -- 172.21.15.90:0/665273441 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9800cd80 con 0x7febb01053f0 2024-04-24T09:19:09.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.177+0000 7febb4a00640 1 -- 172.21.15.90:0/665273441 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febb01a2450 con 0x7febb01053f0 2024-04-24T09:19:09.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.177+0000 7febb4a00640 1 -- 172.21.15.90:0/665273441 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febb01a29f0 con 0x7febb01053f0 2024-04-24T09:19:09.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.177+0000 7feba6a00640 1 -- 172.21.15.90:0/665273441 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb980035f0 con 0x7febb01053f0 2024-04-24T09:19:09.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.177+0000 7feba6a00640 1 -- 172.21.15.90:0/665273441 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb98018690 con 0x7febb01053f0 2024-04-24T09:19:09.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.178+0000 7febb4a00640 1 -- 172.21.15.90:0/665273441 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febb0109810 con 0x7febb01053f0 2024-04-24T09:19:09.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.181+0000 7feba6a00640 1 -- 172.21.15.90:0/665273441 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7feb98004020 con 0x7febb01053f0 2024-04-24T09:19:09.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.183+0000 7feba6a00640 1 --2- 172.21.15.90:0/665273441 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7feb940a3bb0 0x7feb940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:09.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.183+0000 7feba6a00640 1 -- 172.21.15.90:0/665273441 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7879+0+0 (secure 0 0 0) 0x7feb980d0620 con 0x7febb01053f0 2024-04-24T09:19:09.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.183+0000 7febade00640 1 --2- 172.21.15.90:0/665273441 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7feb940a3bb0 0x7feb940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:09.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.184+0000 7febade00640 1 --2- 172.21.15.90:0/665273441 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7feb940a3bb0 0x7feb940a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7feba00095f0 tx=0x7feba0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:09.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.187+0000 7feba6a00640 1 -- 172.21.15.90:0/665273441 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feb9809d110 con 0x7febb01053f0 2024-04-24T09:19:09.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.428+0000 7febb4a00640 1 -- 172.21.15.90:0/665273441 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7febb0062c70 con 0x7febb01053f0 2024-04-24T09:19:09.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.945+0000 7feba6a00640 1 -- 172.21.15.90:0/665273441 <== mon.2 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7feb9809ca10 con 0x7febb01053f0 2024-04-24T09:19:09.945 INFO:teuthology.orchestra.run.smithi090.stderr:new fs with metadata pool 2 and data pool 3 2024-04-24T09:19:09.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.947+0000 7febb4a00640 1 -- 172.21.15.90:0/665273441 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7feb940a3bb0 msgr2=0x7feb940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:09.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.947+0000 7febb4a00640 1 --2- 172.21.15.90:0/665273441 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7feb940a3bb0 0x7feb940a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7feba00095f0 tx=0x7feba0005f90 comp rx=0 tx=0).stop 2024-04-24T09:19:09.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.947+0000 7febb4a00640 1 -- 172.21.15.90:0/665273441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7febb01053f0 msgr2=0x7febb019aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:09.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.947+0000 7febb4a00640 1 --2- 172.21.15.90:0/665273441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7febb01053f0 0x7febb019aa80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7feb9800e9c0 tx=0x7feb9800ee80 comp rx=0 tx=0).stop 2024-04-24T09:19:09.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.948+0000 7febb4a00640 1 -- 172.21.15.90:0/665273441 shutdown_connections 2024-04-24T09:19:09.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.948+0000 7febb4a00640 1 --2- 172.21.15.90:0/665273441 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7feb940a3bb0 0x7feb940a6070 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:09.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.948+0000 7febb4a00640 1 --2- 172.21.15.90:0/665273441 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febb0107d20 0x7febb01a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:09.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.948+0000 7febb4a00640 1 --2- 172.21.15.90:0/665273441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7febb01053f0 0x7febb019aa80 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:09.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.948+0000 7febb4a00640 1 --2- 172.21.15.90:0/665273441 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7febb0102ac0 0x7febb019a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:09.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.948+0000 7febb4a00640 1 -- 172.21.15.90:0/665273441 >> 172.21.15.90:0/665273441 conn(0x7febb00fc4b0 msgr2=0x7febb01021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:09.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.948+0000 7febb4a00640 1 -- 172.21.15.90:0/665273441 shutdown_connections 2024-04-24T09:19:09.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:09.948+0000 7febb4a00640 1 -- 172.21.15.90:0/665273441 wait complete. 2024-04-24T09:19:09.971 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T09:19:10.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:09 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:10.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:09 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:10.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:09 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:10.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:09 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:10.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:09 smithi090 ceph-mon[27825]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-24T09:19:10.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:09 smithi090 ceph-mon[27825]: osdmap e84: 12 total, 12 up, 12 in 2024-04-24T09:19:10.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:09 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/665273441' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-24T09:19:10.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:09 smithi090 ceph-mon[27825]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-24T09:19:10.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:09 smithi090 ceph-mon[27825]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-24T09:19:10.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:09 smithi090 ceph-mon[27825]: pgmap v320: 97 pgs: 11 creating+peering, 76 unknown, 10 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:19:10.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:09 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:10.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:09 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:10.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:09 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:10.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:09 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:10.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:09 smithi149 ceph-mon[32592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-24T09:19:10.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:09 smithi149 ceph-mon[32592]: osdmap e84: 12 total, 12 up, 12 in 2024-04-24T09:19:10.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:09 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/665273441' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-24T09:19:10.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:09 smithi149 ceph-mon[32592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-24T09:19:10.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:09 smithi149 ceph-mon[32592]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-24T09:19:10.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:09 smithi149 ceph-mon[32592]: pgmap v320: 97 pgs: 11 creating+peering, 76 unknown, 10 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:19:10.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.147+0000 7f7e6f400640 1 -- 172.21.15.90:0/3208644925 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e68077c00 msgr2=0x7f7e68078060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:10.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.147+0000 7f7e6f400640 1 --2- 172.21.15.90:0/3208644925 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e68077c00 0x7f7e68078060 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7e54009900 tx=0x7f7e5402a410 comp rx=0 tx=0).stop 2024-04-24T09:19:10.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.148+0000 7f7e6f400640 1 -- 172.21.15.90:0/3208644925 shutdown_connections 2024-04-24T09:19:10.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.148+0000 7f7e6f400640 1 --2- 172.21.15.90:0/3208644925 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e680785a0 0x7f7e68078e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:10.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.148+0000 7f7e6f400640 1 --2- 172.21.15.90:0/3208644925 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e68077c00 0x7f7e68078060 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:10.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.148+0000 7f7e6f400640 1 --2- 172.21.15.90:0/3208644925 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7e680769c0 0x7f7e68076de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:10.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.148+0000 7f7e6f400640 1 -- 172.21.15.90:0/3208644925 >> 172.21.15.90:0/3208644925 conn(0x7f7e680ff7f0 msgr2=0x7f7e68101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:10.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.148+0000 7f7e6f400640 1 -- 172.21.15.90:0/3208644925 shutdown_connections 2024-04-24T09:19:10.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.148+0000 7f7e6f400640 1 -- 172.21.15.90:0/3208644925 wait complete. 2024-04-24T09:19:10.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.149+0000 7f7e6f400640 1 Processor -- start 2024-04-24T09:19:10.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.149+0000 7f7e6f400640 1 -- start start 2024-04-24T09:19:10.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.149+0000 7f7e6f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e680769c0 0x7f7e6819e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:10.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.149+0000 7f7e6f400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7e68077c00 0x7f7e6819ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:10.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.149+0000 7f7e6f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e680785a0 0x7f7e681a5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:10.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.149+0000 7f7e6f400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7e68114350 con 0x7f7e680785a0 2024-04-24T09:19:10.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.149+0000 7f7e6f400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7e681141d0 con 0x7f7e68077c00 2024-04-24T09:19:10.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.149+0000 7f7e6f400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7e681144d0 con 0x7f7e680769c0 2024-04-24T09:19:10.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.149+0000 7f7e6d400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e680785a0 0x7f7e681a5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:10.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.149+0000 7f7e6d400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e680785a0 0x7f7e681a5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55730/0 (socket says 172.21.15.90:55730) 2024-04-24T09:19:10.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.149+0000 7f7e6d400640 1 -- 172.21.15.90:0/3582834435 learned_addr learned my addr 172.21.15.90:0/3582834435 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:19:10.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.149+0000 7f7e6ca00640 1 --2- 172.21.15.90:0/3582834435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e680769c0 0x7f7e6819e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:10.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.149+0000 7f7e67e00640 1 --2- 172.21.15.90:0/3582834435 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7e68077c00 0x7f7e6819ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:10.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.150+0000 7f7e6d400640 1 -- 172.21.15.90:0/3582834435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e680769c0 msgr2=0x7f7e6819e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:10.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.150+0000 7f7e6d400640 1 --2- 172.21.15.90:0/3582834435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e680769c0 0x7f7e6819e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:10.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.150+0000 7f7e6d400640 1 -- 172.21.15.90:0/3582834435 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7e68077c00 msgr2=0x7f7e6819ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:10.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.150+0000 7f7e6d400640 1 --2- 172.21.15.90:0/3582834435 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7e68077c00 0x7f7e6819ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:10.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.150+0000 7f7e6d400640 1 -- 172.21.15.90:0/3582834435 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e681a6570 con 0x7f7e680785a0 2024-04-24T09:19:10.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.150+0000 7f7e6d400640 1 --2- 172.21.15.90:0/3582834435 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e680785a0 0x7f7e681a5f00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7e6000eaa0 tx=0x7f7e6000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:10.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.150+0000 7f7e65600640 1 -- 172.21.15.90:0/3582834435 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6000cdb0 con 0x7f7e680785a0 2024-04-24T09:19:10.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.150+0000 7f7e6f400640 1 -- 172.21.15.90:0/3582834435 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e681a6860 con 0x7f7e680785a0 2024-04-24T09:19:10.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.150+0000 7f7e65600640 1 -- 172.21.15.90:0/3582834435 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e60005020 con 0x7f7e680785a0 2024-04-24T09:19:10.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.150+0000 7f7e65600640 1 -- 172.21.15.90:0/3582834435 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e600116b0 con 0x7f7e680785a0 2024-04-24T09:19:10.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.150+0000 7f7e6f400640 1 -- 172.21.15.90:0/3582834435 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e681a6d30 con 0x7f7e680785a0 2024-04-24T09:19:10.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.151+0000 7f7e6f400640 1 -- 172.21.15.90:0/3582834435 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e68110d50 con 0x7f7e680785a0 2024-04-24T09:19:10.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.153+0000 7f7e65600640 1 -- 172.21.15.90:0/3582834435 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7e60005560 con 0x7f7e680785a0 2024-04-24T09:19:10.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.155+0000 7f7e65600640 1 --2- 172.21.15.90:0/3582834435 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f7e500a3bb0 0x7f7e500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:10.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.155+0000 7f7e65600640 1 -- 172.21.15.90:0/3582834435 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f7e600d0890 con 0x7f7e680785a0 2024-04-24T09:19:10.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.155+0000 7f7e6ca00640 1 --2- 172.21.15.90:0/3582834435 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f7e500a3bb0 0x7f7e500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:10.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.156+0000 7f7e6ca00640 1 --2- 172.21.15.90:0/3582834435 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f7e500a3bb0 0x7f7e500a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c009bd0 tx=0x7f7e5c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:10.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.158+0000 7f7e65600640 1 -- 172.21.15.90:0/3582834435 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7e60016030 con 0x7f7e680785a0 2024-04-24T09:19:10.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:09 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:10.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:09 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:10.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:09 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:10.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:09 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:10.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:09 smithi114 ceph-mon[32501]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-24T09:19:10.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:09 smithi114 ceph-mon[32501]: osdmap e84: 12 total, 12 up, 12 in 2024-04-24T09:19:10.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:09 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/665273441' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-24T09:19:10.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:09 smithi114 ceph-mon[32501]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-24T09:19:10.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:09 smithi114 ceph-mon[32501]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-24T09:19:10.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:09 smithi114 ceph-mon[32501]: pgmap v320: 97 pgs: 11 creating+peering, 76 unknown, 10 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T09:19:10.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.348+0000 7f7e6f400640 1 -- 172.21.15.90:0/3582834435 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7e68062c00 con 0x7f7e680785a0 2024-04-24T09:19:10.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.352+0000 7f7e65600640 1 -- 172.21.15.90:0/3582834435 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22244 (secure 0 0 0) 0x7f7e6009ccd0 con 0x7f7e680785a0 2024-04-24T09:19:10.353 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:19:10.353 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":85,"fsid":"30ee9be2-021a-11ef-bc93-c7b262605968","created":"2024-04-24T09:09:49.496019+0000","modified":"2024-04-24T09:19:09.430952+0000","last_up_change":"2024-04-24T09:18:10.455632+0000","last_in_change":"2024-04-24T09:17:50.600366+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T09:13:45.548774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T09:19:07.185580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T09:19:08.374818+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13a59b9f-b46e-4ec4-90af-bb7a3d5c753b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6803","nonce":2864288066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6805","nonce":2864288066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6809","nonce":2864288066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6807","nonce":2864288066}]},"public_addr":"172.21.15.90:6803/2864288066","cluster_addr":"172.21.15.90:6805/2864288066","heartbeat_back_addr":"172.21.15.90:6809/2864288066","heartbeat_front_addr":"172.21.15.90:6807/2864288066","state":["exists","up"]},{"osd":1,"uuid":"1567ce8f-24e6-42cf-bf64-f6f0c534a9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6801","nonce":344222414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6803","nonce":344222414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6807","nonce":344222414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6805","nonce":344222414}]},"public_addr":"172.21.15.114:6801/344222414","cluster_addr":"172.21.15.114:6803/344222414","heartbeat_back_addr":"172.21.15.114:6807/344222414","heartbeat_front_addr":"172.21.15.114:6805/344222414","state":["exists","up"]},{"osd":2,"uuid":"f861b304-9c7d-4629-8522-37f33a747945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6801","nonce":3257067419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6803","nonce":3257067419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6807","nonce":3257067419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6805","nonce":3257067419}]},"public_addr":"172.21.15.149:6801/3257067419","cluster_addr":"172.21.15.149:6803/3257067419","heartbeat_back_addr":"172.21.15.149:6807/3257067419","heartbeat_front_addr":"172.21.15.149:6805/3257067419","state":["exists","up"]},{"osd":3,"uuid":"9377ab6e-5996-45fb-93a9-475a2e5f17a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6811","nonce":3068524970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6813","nonce":3068524970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6817","nonce":3068524970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6815","nonce":3068524970}]},"public_addr":"172.21.15.90:6811/3068524970","cluster_addr":"172.21.15.90:6813/3068524970","heartbeat_back_addr":"172.21.15.90:6817/3068524970","heartbeat_front_addr":"172.21.15.90:6815/3068524970","state":["exists","up"]},{"osd":4,"uuid":"e2ddd19e-e641-4d61-b342-02b5f12b1c4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6809","nonce":2782015522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6811","nonce":2782015522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6815","nonce":2782015522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6813","nonce":2782015522}]},"public_addr":"172.21.15.114:6809/2782015522","cluster_addr":"172.21.15.114:6811/2782015522","heartbeat_back_addr":"172.21.15.114:6815/2782015522","heartbeat_front_addr":"172.21.15.114:6813/2782015522","state":["exists","up"]},{"osd":5,"uuid":"b6ff9d10-5210-4b46-a192-6dcbf9d1f156","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6809","nonce":4266460479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6811","nonce":4266460479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6815","nonce":4266460479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6813","nonce":4266460479}]},"public_addr":"172.21.15.149:6809/4266460479","cluster_addr":"172.21.15.149:6811/4266460479","heartbeat_back_addr":"172.21.15.149:6815/4266460479","heartbeat_front_addr":"172.21.15.149:6813/4266460479","state":["exists","up"]},{"osd":6,"uuid":"4287b5a1-f953-421c-bf62-5e390bee834d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6819","nonce":1121461276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6821","nonce":1121461276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6825","nonce":1121461276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6823","nonce":1121461276}]},"public_addr":"172.21.15.90:6819/1121461276","cluster_addr":"172.21.15.90:6821/1121461276","heartbeat_back_addr":"172.21.15.90:6825/1121461276","heartbeat_front_addr":"172.21.15.90:6823/1121461276","state":["exists","up"]},{"osd":7,"uuid":"a54e507d-d6ba-4bf2-9a7f-a7b95c3859bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6817","nonce":3222546280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6819","nonce":3222546280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6823","nonce":3222546280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6821","nonce":3222546280}]},"public_addr":"172.21.15.114:6817/3222546280","cluster_addr":"172.21.15.114:6819/3222546280","heartbeat_back_addr":"172.21.15.114:6823/3222546280","heartbeat_front_addr":"172.21.15.114:6821/3222546280","state":["exists","up"]},{"osd":8,"uuid":"9950b010-5865-4109-ae37-582b1dbd2f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6817","nonce":157615149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6819","nonce":157615149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6823","nonce":157615149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6821","nonce":157615149}]},"public_addr":"172.21.15.149:6817/157615149","cluster_addr":"172.21.15.149:6819/157615149","heartbeat_back_addr":"172.21.15.149:6823/157615149","heartbeat_front_addr":"172.21.15.149:6821/157615149","state":["exists","up"]},{"osd":9,"uuid":"31e2635c-e234-421f-9757-592fd673c936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6827","nonce":1307318076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6829","nonce":1307318076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6833","nonce":1307318076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6831","nonce":1307318076}]},"public_addr":"172.21.15.90:6827/1307318076","cluster_addr":"172.21.15.90:6829/1307318076","heartbeat_back_addr":"172.21.15.90:6833/1307318076","heartbeat_front_addr":"172.21.15.90:6831/1307318076","state":["exists","up"]},{"osd":10,"uuid":"46a3b2a6-e6e5-4578-b4bb-f3e998b9bc95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6825","nonce":1719802140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6827","nonce":1719802140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6831","nonce":1719802140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6829","nonce":1719802140}]},"public_addr":"172.21.15.114:6825/1719802140","cluster_addr":"172.21.15.114:6827/1719802140","heartbeat_back_addr":"172.21.15.114:6831/1719802140","heartbeat_front_addr":"172.21.15.114:6829/1719802140","state":["exists","up"]},{"osd":11,"uuid":"ccd4456b-e947-46cf-a779-148ffa289b58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6825","nonce":1737275166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6827","nonce":1737275166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6831","nonce":1737275166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6829","nonce":1737275166}]},"public_addr":"172.21.15.149:6825/1737275166","cluster_addr":"172.21.15.149:6827/1737275166","heartbeat_back_addr":"172.21.15.149:6831/1737275166","heartbeat_front_addr":"172.21.15.149:6829/1737275166","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:12:53.015078+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:13:18.047053+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:13:41.935891+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:14:15.095388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:14:42.512355+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:15:08.113775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:15:43.828550+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:16:10.901909+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:16:36.422962+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:17:13.924446+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:17:42.814401+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:18:08.672793+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/3090852580":"2024-04-25T09:10:57.377655+0000","172.21.15.90:0/687197394":"2024-04-25T09:10:57.377655+0000","172.21.15.90:6801/1809038230":"2024-04-25T09:10:57.377655+0000","172.21.15.90:6800/1809038230":"2024-04-25T09:10:57.377655+0000","172.21.15.90:0/2964149071":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/3617388113":"2024-04-25T09:10:22.599674+0000","172.21.15.90:6801/3089140842":"2024-04-25T09:10:22.599674+0000","172.21.15.90:6800/3089140842":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/2840416149":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/305432710":"2024-04-25T09:10:57.377655+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T09:19:10.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.355+0000 7f7e6f400640 1 -- 172.21.15.90:0/3582834435 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f7e500a3bb0 msgr2=0x7f7e500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:10.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.355+0000 7f7e6f400640 1 --2- 172.21.15.90:0/3582834435 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f7e500a3bb0 0x7f7e500a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c009bd0 tx=0x7f7e5c005f90 comp rx=0 tx=0).stop 2024-04-24T09:19:10.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.355+0000 7f7e6f400640 1 -- 172.21.15.90:0/3582834435 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e680785a0 msgr2=0x7f7e681a5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:10.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.355+0000 7f7e6f400640 1 --2- 172.21.15.90:0/3582834435 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e680785a0 0x7f7e681a5f00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7e6000eaa0 tx=0x7f7e6000ef60 comp rx=0 tx=0).stop 2024-04-24T09:19:10.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.355+0000 7f7e6f400640 1 -- 172.21.15.90:0/3582834435 shutdown_connections 2024-04-24T09:19:10.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.355+0000 7f7e6f400640 1 --2- 172.21.15.90:0/3582834435 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f7e500a3bb0 0x7f7e500a6070 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:10.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.355+0000 7f7e6f400640 1 --2- 172.21.15.90:0/3582834435 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e680785a0 0x7f7e681a5f00 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:10.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.355+0000 7f7e6f400640 1 --2- 172.21.15.90:0/3582834435 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7e68077c00 0x7f7e6819ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:10.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.355+0000 7f7e6f400640 1 --2- 172.21.15.90:0/3582834435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e680769c0 0x7f7e6819e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:10.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.355+0000 7f7e6f400640 1 -- 172.21.15.90:0/3582834435 >> 172.21.15.90:0/3582834435 conn(0x7f7e680ff7f0 msgr2=0x7f7e68100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:10.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.355+0000 7f7e6f400640 1 -- 172.21.15.90:0/3582834435 shutdown_connections 2024-04-24T09:19:10.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.355+0000 7f7e6f400640 1 -- 172.21.15.90:0/3582834435 wait complete. 2024-04-24T09:19:10.373 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T09:19:10.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.564+0000 7f9aab200640 1 -- 172.21.15.90:0/2935166794 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9aac108480 msgr2=0x7f9aac1088a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:10.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.564+0000 7f9aab200640 1 --2- 172.21.15.90:0/2935166794 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9aac108480 0x7f9aac1088a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c009860 tx=0x7f9a9c02a340 comp rx=0 tx=0).stop 2024-04-24T09:19:10.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.564+0000 7f9aab200640 1 -- 172.21.15.90:0/2935166794 shutdown_connections 2024-04-24T09:19:10.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.564+0000 7f9aab200640 1 --2- 172.21.15.90:0/2935166794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9aac10a030 0x7f9aac1108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:10.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.564+0000 7f9aab200640 1 --2- 172.21.15.90:0/2935166794 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9aac109670 0x7f9aac109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:10.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.564+0000 7f9aab200640 1 --2- 172.21.15.90:0/2935166794 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9aac108480 0x7f9aac1088a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:10.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.564+0000 7f9aab200640 1 -- 172.21.15.90:0/2935166794 >> 172.21.15.90:0/2935166794 conn(0x7f9aac077440 msgr2=0x7f9aac079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:10.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.564+0000 7f9aab200640 1 -- 172.21.15.90:0/2935166794 shutdown_connections 2024-04-24T09:19:10.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.564+0000 7f9aab200640 1 -- 172.21.15.90:0/2935166794 wait complete. 2024-04-24T09:19:10.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.565+0000 7f9aab200640 1 Processor -- start 2024-04-24T09:19:10.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.565+0000 7f9aab200640 1 -- start start 2024-04-24T09:19:10.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.566+0000 7f9aab200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9aac108480 0x7f9aac19e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:10.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.566+0000 7f9aab200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9aac109670 0x7f9aac19ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:10.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.566+0000 7f9aab200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9aac10a030 0x7f9aac1a5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:10.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.566+0000 7f9aab200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9aac114110 con 0x7f9aac108480 2024-04-24T09:19:10.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.566+0000 7f9aab200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9aac113f90 con 0x7f9aac109670 2024-04-24T09:19:10.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.566+0000 7f9aab200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9aac114290 con 0x7f9aac10a030 2024-04-24T09:19:10.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.566+0000 7f9aa9e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9aac108480 0x7f9aac19e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:10.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.566+0000 7f9aa9e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9aac108480 0x7f9aac19e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55748/0 (socket says 172.21.15.90:55748) 2024-04-24T09:19:10.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.566+0000 7f9aa9e00640 1 -- 172.21.15.90:0/1642554795 learned_addr learned my addr 172.21.15.90:0/1642554795 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:19:10.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.566+0000 7f9aa1400640 1 --2- 172.21.15.90:0/1642554795 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9aac109670 0x7f9aac19ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:10.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.566+0000 7f9aaa800640 1 --2- 172.21.15.90:0/1642554795 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9aac10a030 0x7f9aac1a5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:10.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.567+0000 7f9aa1400640 1 -- 172.21.15.90:0/1642554795 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9aac10a030 msgr2=0x7f9aac1a5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:10.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.567+0000 7f9aa1400640 1 --2- 172.21.15.90:0/1642554795 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9aac10a030 0x7f9aac1a5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:10.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.567+0000 7f9aa1400640 1 -- 172.21.15.90:0/1642554795 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9aac108480 msgr2=0x7f9aac19e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:10.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.567+0000 7f9aa1400640 1 --2- 172.21.15.90:0/1642554795 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9aac108480 0x7f9aac19e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:10.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.567+0000 7f9aa1400640 1 -- 172.21.15.90:0/1642554795 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aac1a63c0 con 0x7f9aac109670 2024-04-24T09:19:10.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.567+0000 7f9aa9e00640 1 --2- 172.21.15.90:0/1642554795 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9aac108480 0x7f9aac19e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:19:10.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.567+0000 7f9aa1400640 1 --2- 172.21.15.90:0/1642554795 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9aac109670 0x7f9aac19ece0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9a9400e790 tx=0x7f9a9400ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:10.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.567+0000 7f9aa3400640 1 -- 172.21.15.90:0/1642554795 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a9400eea0 con 0x7f9aac109670 2024-04-24T09:19:10.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.567+0000 7f9aab200640 1 -- 172.21.15.90:0/1642554795 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9aac1a66b0 con 0x7f9aac109670 2024-04-24T09:19:10.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.567+0000 7f9aa3400640 1 -- 172.21.15.90:0/1642554795 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a9400ccc0 con 0x7f9aac109670 2024-04-24T09:19:10.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.567+0000 7f9aa3400640 1 -- 172.21.15.90:0/1642554795 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a94018760 con 0x7f9aac109670 2024-04-24T09:19:10.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.568+0000 7f9aab200640 1 -- 172.21.15.90:0/1642554795 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9aac1a6c50 con 0x7f9aac109670 2024-04-24T09:19:10.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.569+0000 7f9aab200640 1 -- 172.21.15.90:0/1642554795 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9aac10f7a0 con 0x7f9aac109670 2024-04-24T09:19:10.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.571+0000 7f9aa3400640 1 -- 172.21.15.90:0/1642554795 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9a94010430 con 0x7f9aac109670 2024-04-24T09:19:10.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.572+0000 7f9aa3400640 1 --2- 172.21.15.90:0/1642554795 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f9a900a3bb0 0x7f9a900a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:10.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.572+0000 7f9aa3400640 1 -- 172.21.15.90:0/1642554795 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f9a940d0980 con 0x7f9aac109670 2024-04-24T09:19:10.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.572+0000 7f9aa9e00640 1 --2- 172.21.15.90:0/1642554795 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f9a900a3bb0 0x7f9a900a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:10.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.573+0000 7f9aa9e00640 1 --2- 172.21.15.90:0/1642554795 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f9a900a3bb0 0x7f9a900a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c02a930 tx=0x7f9a9c0058b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:10.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.574+0000 7f9aa3400640 1 -- 172.21.15.90:0/1642554795 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9a9409cdc0 con 0x7f9aac109670 2024-04-24T09:19:10.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.778+0000 7f9aab200640 1 -- 172.21.15.90:0/1642554795 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9aac02d820 con 0x7f9aac109670 2024-04-24T09:19:10.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.780+0000 7f9aa3400640 1 -- 172.21.15.90:0/1642554795 <== mon.2 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22244 (secure 0 0 0) 0x7f9a940a10c0 con 0x7f9aac109670 2024-04-24T09:19:10.781 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:19:10.781 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":85,"fsid":"30ee9be2-021a-11ef-bc93-c7b262605968","created":"2024-04-24T09:09:49.496019+0000","modified":"2024-04-24T09:19:09.430952+0000","last_up_change":"2024-04-24T09:18:10.455632+0000","last_in_change":"2024-04-24T09:17:50.600366+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T09:13:45.548774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T09:19:07.185580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T09:19:08.374818+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13a59b9f-b46e-4ec4-90af-bb7a3d5c753b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6803","nonce":2864288066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6805","nonce":2864288066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6809","nonce":2864288066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6807","nonce":2864288066}]},"public_addr":"172.21.15.90:6803/2864288066","cluster_addr":"172.21.15.90:6805/2864288066","heartbeat_back_addr":"172.21.15.90:6809/2864288066","heartbeat_front_addr":"172.21.15.90:6807/2864288066","state":["exists","up"]},{"osd":1,"uuid":"1567ce8f-24e6-42cf-bf64-f6f0c534a9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6801","nonce":344222414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6803","nonce":344222414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6807","nonce":344222414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6805","nonce":344222414}]},"public_addr":"172.21.15.114:6801/344222414","cluster_addr":"172.21.15.114:6803/344222414","heartbeat_back_addr":"172.21.15.114:6807/344222414","heartbeat_front_addr":"172.21.15.114:6805/344222414","state":["exists","up"]},{"osd":2,"uuid":"f861b304-9c7d-4629-8522-37f33a747945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6801","nonce":3257067419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6803","nonce":3257067419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6807","nonce":3257067419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6805","nonce":3257067419}]},"public_addr":"172.21.15.149:6801/3257067419","cluster_addr":"172.21.15.149:6803/3257067419","heartbeat_back_addr":"172.21.15.149:6807/3257067419","heartbeat_front_addr":"172.21.15.149:6805/3257067419","state":["exists","up"]},{"osd":3,"uuid":"9377ab6e-5996-45fb-93a9-475a2e5f17a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6811","nonce":3068524970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6813","nonce":3068524970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6817","nonce":3068524970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6815","nonce":3068524970}]},"public_addr":"172.21.15.90:6811/3068524970","cluster_addr":"172.21.15.90:6813/3068524970","heartbeat_back_addr":"172.21.15.90:6817/3068524970","heartbeat_front_addr":"172.21.15.90:6815/3068524970","state":["exists","up"]},{"osd":4,"uuid":"e2ddd19e-e641-4d61-b342-02b5f12b1c4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6809","nonce":2782015522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6811","nonce":2782015522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6815","nonce":2782015522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6813","nonce":2782015522}]},"public_addr":"172.21.15.114:6809/2782015522","cluster_addr":"172.21.15.114:6811/2782015522","heartbeat_back_addr":"172.21.15.114:6815/2782015522","heartbeat_front_addr":"172.21.15.114:6813/2782015522","state":["exists","up"]},{"osd":5,"uuid":"b6ff9d10-5210-4b46-a192-6dcbf9d1f156","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6809","nonce":4266460479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6811","nonce":4266460479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6815","nonce":4266460479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6813","nonce":4266460479}]},"public_addr":"172.21.15.149:6809/4266460479","cluster_addr":"172.21.15.149:6811/4266460479","heartbeat_back_addr":"172.21.15.149:6815/4266460479","heartbeat_front_addr":"172.21.15.149:6813/4266460479","state":["exists","up"]},{"osd":6,"uuid":"4287b5a1-f953-421c-bf62-5e390bee834d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6819","nonce":1121461276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6821","nonce":1121461276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6825","nonce":1121461276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6823","nonce":1121461276}]},"public_addr":"172.21.15.90:6819/1121461276","cluster_addr":"172.21.15.90:6821/1121461276","heartbeat_back_addr":"172.21.15.90:6825/1121461276","heartbeat_front_addr":"172.21.15.90:6823/1121461276","state":["exists","up"]},{"osd":7,"uuid":"a54e507d-d6ba-4bf2-9a7f-a7b95c3859bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6817","nonce":3222546280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6819","nonce":3222546280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6823","nonce":3222546280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6821","nonce":3222546280}]},"public_addr":"172.21.15.114:6817/3222546280","cluster_addr":"172.21.15.114:6819/3222546280","heartbeat_back_addr":"172.21.15.114:6823/3222546280","heartbeat_front_addr":"172.21.15.114:6821/3222546280","state":["exists","up"]},{"osd":8,"uuid":"9950b010-5865-4109-ae37-582b1dbd2f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6817","nonce":157615149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6819","nonce":157615149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6823","nonce":157615149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6821","nonce":157615149}]},"public_addr":"172.21.15.149:6817/157615149","cluster_addr":"172.21.15.149:6819/157615149","heartbeat_back_addr":"172.21.15.149:6823/157615149","heartbeat_front_addr":"172.21.15.149:6821/157615149","state":["exists","up"]},{"osd":9,"uuid":"31e2635c-e234-421f-9757-592fd673c936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6827","nonce":1307318076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6829","nonce":1307318076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6833","nonce":1307318076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6831","nonce":1307318076}]},"public_addr":"172.21.15.90:6827/1307318076","cluster_addr":"172.21.15.90:6829/1307318076","heartbeat_back_addr":"172.21.15.90:6833/1307318076","heartbeat_front_addr":"172.21.15.90:6831/1307318076","state":["exists","up"]},{"osd":10,"uuid":"46a3b2a6-e6e5-4578-b4bb-f3e998b9bc95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6825","nonce":1719802140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6827","nonce":1719802140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6831","nonce":1719802140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6829","nonce":1719802140}]},"public_addr":"172.21.15.114:6825/1719802140","cluster_addr":"172.21.15.114:6827/1719802140","heartbeat_back_addr":"172.21.15.114:6831/1719802140","heartbeat_front_addr":"172.21.15.114:6829/1719802140","state":["exists","up"]},{"osd":11,"uuid":"ccd4456b-e947-46cf-a779-148ffa289b58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6825","nonce":1737275166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6827","nonce":1737275166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6831","nonce":1737275166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6829","nonce":1737275166}]},"public_addr":"172.21.15.149:6825/1737275166","cluster_addr":"172.21.15.149:6827/1737275166","heartbeat_back_addr":"172.21.15.149:6831/1737275166","heartbeat_front_addr":"172.21.15.149:6829/1737275166","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:12:53.015078+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:13:18.047053+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:13:41.935891+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:14:15.095388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:14:42.512355+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:15:08.113775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:15:43.828550+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:16:10.901909+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:16:36.422962+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:17:13.924446+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:17:42.814401+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:18:08.672793+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/3090852580":"2024-04-25T09:10:57.377655+0000","172.21.15.90:0/687197394":"2024-04-25T09:10:57.377655+0000","172.21.15.90:6801/1809038230":"2024-04-25T09:10:57.377655+0000","172.21.15.90:6800/1809038230":"2024-04-25T09:10:57.377655+0000","172.21.15.90:0/2964149071":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/3617388113":"2024-04-25T09:10:22.599674+0000","172.21.15.90:6801/3089140842":"2024-04-25T09:10:22.599674+0000","172.21.15.90:6800/3089140842":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/2840416149":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/305432710":"2024-04-25T09:10:57.377655+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T09:19:10.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.783+0000 7f9aab200640 1 -- 172.21.15.90:0/1642554795 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f9a900a3bb0 msgr2=0x7f9a900a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:10.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.783+0000 7f9aab200640 1 --2- 172.21.15.90:0/1642554795 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f9a900a3bb0 0x7f9a900a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c02a930 tx=0x7f9a9c0058b0 comp rx=0 tx=0).stop 2024-04-24T09:19:10.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.783+0000 7f9aab200640 1 -- 172.21.15.90:0/1642554795 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9aac109670 msgr2=0x7f9aac19ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:10.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.783+0000 7f9aab200640 1 --2- 172.21.15.90:0/1642554795 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9aac109670 0x7f9aac19ece0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9a9400e790 tx=0x7f9a9400ec50 comp rx=0 tx=0).stop 2024-04-24T09:19:10.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.783+0000 7f9aab200640 1 -- 172.21.15.90:0/1642554795 shutdown_connections 2024-04-24T09:19:10.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.783+0000 7f9aab200640 1 --2- 172.21.15.90:0/1642554795 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f9a900a3bb0 0x7f9a900a6070 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:10.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.783+0000 7f9aab200640 1 --2- 172.21.15.90:0/1642554795 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9aac10a030 0x7f9aac1a5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:10.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.783+0000 7f9aab200640 1 --2- 172.21.15.90:0/1642554795 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9aac109670 0x7f9aac19ece0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:10.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.783+0000 7f9aab200640 1 --2- 172.21.15.90:0/1642554795 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9aac108480 0x7f9aac19e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:10.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.783+0000 7f9aab200640 1 -- 172.21.15.90:0/1642554795 >> 172.21.15.90:0/1642554795 conn(0x7f9aac077440 msgr2=0x7f9aac10c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:10.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.784+0000 7f9aab200640 1 -- 172.21.15.90:0/1642554795 shutdown_connections 2024-04-24T09:19:10.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:10.784+0000 7f9aab200640 1 -- 172.21.15.90:0/1642554795 wait complete. 2024-04-24T09:19:10.803 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-24T09:19:11.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:11.003+0000 7fa28dc00640 1 -- 172.21.15.90:0/42849206 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa288104140 msgr2=0x7fa288104560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:11.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:11.003+0000 7fa28dc00640 1 --2- 172.21.15.90:0/42849206 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa288104140 0x7fa288104560 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa278009900 tx=0x7fa27802a3f0 comp rx=0 tx=0).stop 2024-04-24T09:19:11.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:11.004+0000 7fa28dc00640 1 -- 172.21.15.90:0/42849206 shutdown_connections 2024-04-24T09:19:11.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:11.004+0000 7fa28dc00640 1 --2- 172.21.15.90:0/42849206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa288105cf0 0x7fa28810c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:11.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:11.004+0000 7fa28dc00640 1 --2- 172.21.15.90:0/42849206 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa288105330 0x7fa2881057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:11.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:11.004+0000 7fa28dc00640 1 --2- 172.21.15.90:0/42849206 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa288104140 0x7fa288104560 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:11.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:11.004+0000 7fa28dc00640 1 -- 172.21.15.90:0/42849206 >> 172.21.15.90:0/42849206 conn(0x7fa2880ff6b0 msgr2=0x7fa288101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:11.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:11.004+0000 7fa28dc00640 1 -- 172.21.15.90:0/42849206 shutdown_connections 2024-04-24T09:19:11.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:11.004+0000 7fa28dc00640 1 -- 172.21.15.90:0/42849206 wait complete. 2024-04-24T09:19:11.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:11.005+0000 7fa28dc00640 1 Processor -- start 2024-04-24T09:19:11.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:11.005+0000 7fa28dc00640 1 -- start start 2024-04-24T09:19:11.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:11.005+0000 7fa28dc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa288104140 0x7fa28819a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:11.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:11.005+0000 7fa28dc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa288105330 0x7fa28819a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:11.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:11.005+0000 7fa28dc00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa288105cf0 0x7fa2881a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:11.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:11.005+0000 7fa28dc00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa28810fdd0 con 0x7fa288105330 2024-04-24T09:19:11.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:11.006+0000 7fa28dc00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa28810fc50 con 0x7fa288105cf0 2024-04-24T09:19:11.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:11.006+0000 7fa28dc00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa28810ff50 con 0x7fa288104140 2024-04-24T09:19:11.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:11.006+0000 7fa27f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa288105330 0x7fa28819a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:11.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:11.006+0000 7fa287e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa288104140 0x7fa28819a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:11.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:11.006+0000 7fa27f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa288105330 0x7fa28819a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55768/0 (socket says 172.21.15.90:55768) 2024-04-24T09:19:11.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:11.006+0000 7fa27f400640 1 -- 172.21.15.90:0/3423562913 learned_addr learned my addr 172.21.15.90:0/3423562913 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:19:11.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:11.006+0000 7fa28d200640 1 --2- 172.21.15.90:0/3423562913 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa288105cf0 0x7fa2881a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:11.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:11.006+0000 7fa27f400640 1 -- 172.21.15.90:0/3423562913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa288104140 msgr2=0x7fa28819a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:11.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:11.006+0000 7fa27f400640 1 --2- 172.21.15.90:0/3423562913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa288104140 0x7fa28819a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:11.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:11.006+0000 7fa27f400640 1 -- 172.21.15.90:0/3423562913 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa288105cf0 msgr2=0x7fa2881a1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:11.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:11.006+0000 7fa27f400640 1 --2- 172.21.15.90:0/3423562913 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa288105cf0 0x7fa2881a1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:11.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:11.006+0000 7fa27f400640 1 -- 172.21.15.90:0/3423562913 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2881a2070 con 0x7fa288105330 2024-04-24T09:19:11.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:11.006+0000 7fa27f400640 1 --2- 172.21.15.90:0/3423562913 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa288105330 0x7fa28819a990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa27000e9f0 tx=0x7fa27000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:11.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:11.007+0000 7fa285600640 1 -- 172.21.15.90:0/3423562913 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27000cd80 con 0x7fa288105330 2024-04-24T09:19:11.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:11.007+0000 7fa285600640 1 -- 172.21.15.90:0/3423562913 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa270005500 con 0x7fa288105330 2024-04-24T09:19:11.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:11.007+0000 7fa28dc00640 1 -- 172.21.15.90:0/3423562913 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2881a2360 con 0x7fa288105330 2024-04-24T09:19:11.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:11.007+0000 7fa285600640 1 -- 172.21.15.90:0/3423562913 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa270010690 con 0x7fa288105330 2024-04-24T09:19:11.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:11.007+0000 7fa28dc00640 1 -- 172.21.15.90:0/3423562913 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2881a2860 con 0x7fa288105330 2024-04-24T09:19:11.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:11.008+0000 7fa28dc00640 1 -- 172.21.15.90:0/3423562913 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa28810b530 con 0x7fa288105330 2024-04-24T09:19:11.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:11.010+0000 7fa285600640 1 -- 172.21.15.90:0/3423562913 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa2700046c0 con 0x7fa288105330 2024-04-24T09:19:11.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:11.011+0000 7fa285600640 1 --2- 172.21.15.90:0/3423562913 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa26c0a3bb0 0x7fa26c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:11.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:11.012+0000 7fa285600640 1 -- 172.21.15.90:0/3423562913 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fa2700d06b0 con 0x7fa288105330 2024-04-24T09:19:11.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:11.012+0000 7fa287e00640 1 --2- 172.21.15.90:0/3423562913 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa26c0a3bb0 0x7fa26c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:11.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:11.013+0000 7fa287e00640 1 --2- 172.21.15.90:0/3423562913 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa26c0a3bb0 0x7fa26c0a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa278005ab0 tx=0x7fa278000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:11.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:11.016+0000 7fa285600640 1 -- 172.21.15.90:0/3423562913 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa27009d1e0 con 0x7fa288105330 2024-04-24T09:19:11.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:10 smithi090 ceph-mon[27825]: osdmap e85: 12 total, 12 up, 12 in 2024-04-24T09:19:11.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:10 smithi090 ceph-mon[27825]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T09:19:11.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:10 smithi090 ceph-mon[27825]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-24T09:19:11.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:10 smithi090 ceph-mon[27825]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-24T09:19:11.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:10 smithi090 ceph-mon[27825]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-24T09:19:11.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:10 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:11.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:10 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:11.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:10 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:19:11.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:10 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:19:11.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:10 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:11.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:10 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/3582834435' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:19:11.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:10 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/1642554795' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:19:11.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:10 smithi149 ceph-mon[32592]: osdmap e85: 12 total, 12 up, 12 in 2024-04-24T09:19:11.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:10 smithi149 ceph-mon[32592]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T09:19:11.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:10 smithi149 ceph-mon[32592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-24T09:19:11.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:10 smithi149 ceph-mon[32592]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-24T09:19:11.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:10 smithi149 ceph-mon[32592]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-24T09:19:11.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:10 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:11.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:10 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:11.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:10 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:19:11.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:10 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:19:11.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:10 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:11.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:10 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/3582834435' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:19:11.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:10 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/1642554795' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:19:11.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:10 smithi114 ceph-mon[32501]: osdmap e85: 12 total, 12 up, 12 in 2024-04-24T09:19:11.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:10 smithi114 ceph-mon[32501]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T09:19:11.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:10 smithi114 ceph-mon[32501]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-24T09:19:11.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:10 smithi114 ceph-mon[32501]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-24T09:19:11.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:10 smithi114 ceph-mon[32501]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-24T09:19:11.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:10 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:11.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:10 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:11.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:10 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:19:11.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:10 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:19:11.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:10 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:19:11.177 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:10 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/3582834435' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:19:11.177 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:10 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/1642554795' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:19:11.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:11.280+0000 7fa28dc00640 1 -- 172.21.15.90:0/3423562913 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fa28802d620 con 0x7fa288105330 2024-04-24T09:19:12.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:11 smithi090 ceph-mon[27825]: osdmap e86: 12 total, 12 up, 12 in 2024-04-24T09:19:12.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:11 smithi090 ceph-mon[27825]: mds.? [v2:172.21.15.114:6838/1025403710,v1:172.21.15.114:6839/1025403710] up:active 2024-04-24T09:19:12.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:11 smithi090 ceph-mon[27825]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T09:19:12.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:11 smithi090 ceph-mon[27825]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T09:19:12.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:11 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/3423562913' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-24T09:19:12.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:11 smithi090 ceph-mon[27825]: pgmap v323: 97 pgs: 15 creating+peering, 32 unknown, 50 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-04-24T09:19:12.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:11 smithi149 ceph-mon[32592]: osdmap e86: 12 total, 12 up, 12 in 2024-04-24T09:19:12.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:11 smithi149 ceph-mon[32592]: mds.? [v2:172.21.15.114:6838/1025403710,v1:172.21.15.114:6839/1025403710] up:active 2024-04-24T09:19:12.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:11 smithi149 ceph-mon[32592]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T09:19:12.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:11 smithi149 ceph-mon[32592]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T09:19:12.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:11 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/3423562913' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-24T09:19:12.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:11 smithi149 ceph-mon[32592]: pgmap v323: 97 pgs: 15 creating+peering, 32 unknown, 50 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-04-24T09:19:12.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:11 smithi114 ceph-mon[32501]: osdmap e86: 12 total, 12 up, 12 in 2024-04-24T09:19:12.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:11 smithi114 ceph-mon[32501]: mds.? [v2:172.21.15.114:6838/1025403710,v1:172.21.15.114:6839/1025403710] up:active 2024-04-24T09:19:12.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:11 smithi114 ceph-mon[32501]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T09:19:12.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:11 smithi114 ceph-mon[32501]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T09:19:12.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:11 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/3423562913' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-24T09:19:12.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:11 smithi114 ceph-mon[32501]: pgmap v323: 97 pgs: 15 creating+peering, 32 unknown, 50 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-04-24T09:19:12.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.250+0000 7fa285600640 1 -- 172.21.15.90:0/3423562913 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7fa27009cae0 con 0x7fa288105330 2024-04-24T09:19:12.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.254+0000 7fa28dc00640 1 -- 172.21.15.90:0/3423562913 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa26c0a3bb0 msgr2=0x7fa26c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:12.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.254+0000 7fa28dc00640 1 --2- 172.21.15.90:0/3423562913 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa26c0a3bb0 0x7fa26c0a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa278005ab0 tx=0x7fa278000bc0 comp rx=0 tx=0).stop 2024-04-24T09:19:12.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.254+0000 7fa28dc00640 1 -- 172.21.15.90:0/3423562913 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa288105330 msgr2=0x7fa28819a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:12.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.254+0000 7fa28dc00640 1 --2- 172.21.15.90:0/3423562913 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa288105330 0x7fa28819a990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa27000e9f0 tx=0x7fa27000edf0 comp rx=0 tx=0).stop 2024-04-24T09:19:12.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.254+0000 7fa28dc00640 1 -- 172.21.15.90:0/3423562913 shutdown_connections 2024-04-24T09:19:12.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.254+0000 7fa28dc00640 1 --2- 172.21.15.90:0/3423562913 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa26c0a3bb0 0x7fa26c0a6070 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:12.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.254+0000 7fa28dc00640 1 --2- 172.21.15.90:0/3423562913 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa288105cf0 0x7fa2881a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:12.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.254+0000 7fa28dc00640 1 --2- 172.21.15.90:0/3423562913 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa288105330 0x7fa28819a990 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:12.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.254+0000 7fa28dc00640 1 --2- 172.21.15.90:0/3423562913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa288104140 0x7fa28819a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:12.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.254+0000 7fa28dc00640 1 -- 172.21.15.90:0/3423562913 >> 172.21.15.90:0/3423562913 conn(0x7fa2880ff6b0 msgr2=0x7fa288108700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:12.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.255+0000 7fa28dc00640 1 -- 172.21.15.90:0/3423562913 shutdown_connections 2024-04-24T09:19:12.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.255+0000 7fa28dc00640 1 -- 172.21.15.90:0/3423562913 wait complete. 2024-04-24T09:19:12.279 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-04-24T09:19:12.279 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-04-24T09:19:12.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.476+0000 7f396be00640 1 -- 172.21.15.90:0/4117521766 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f396c107c60 msgr2=0x7f396c10a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:12.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.476+0000 7f396be00640 1 --2- 172.21.15.90:0/4117521766 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f396c107c60 0x7f396c10a050 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3964009c90 tx=0x7f396402a860 comp rx=0 tx=0).stop 2024-04-24T09:19:12.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.476+0000 7f396be00640 1 -- 172.21.15.90:0/4117521766 shutdown_connections 2024-04-24T09:19:12.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.476+0000 7f396be00640 1 --2- 172.21.15.90:0/4117521766 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f396c107c60 0x7f396c10a050 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:12.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.476+0000 7f396be00640 1 --2- 172.21.15.90:0/4117521766 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f396c105330 0x7f396c107720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:12.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.476+0000 7f396be00640 1 --2- 172.21.15.90:0/4117521766 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f396c102a00 0x7f396c104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:12.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.476+0000 7f396be00640 1 -- 172.21.15.90:0/4117521766 >> 172.21.15.90:0/4117521766 conn(0x7f396c0fc3b0 msgr2=0x7f396c0fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:12.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.476+0000 7f396be00640 1 -- 172.21.15.90:0/4117521766 shutdown_connections 2024-04-24T09:19:12.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.476+0000 7f396be00640 1 -- 172.21.15.90:0/4117521766 wait complete. 2024-04-24T09:19:12.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.477+0000 7f396be00640 1 Processor -- start 2024-04-24T09:19:12.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.477+0000 7f396be00640 1 -- start start 2024-04-24T09:19:12.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.478+0000 7f396be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f396c102a00 0x7f396c19a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:12.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.478+0000 7f396be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f396c105330 0x7f396c19a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:12.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.478+0000 7f396be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f396c107c60 0x7f396c1a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:12.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.478+0000 7f396be00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f396c10dc00 con 0x7f396c107c60 2024-04-24T09:19:12.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.478+0000 7f396be00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f396c10da80 con 0x7f396c102a00 2024-04-24T09:19:12.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.478+0000 7f396be00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f396c10dd80 con 0x7f396c105330 2024-04-24T09:19:12.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.478+0000 7f396b400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f396c107c60 0x7f396c1a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:12.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.478+0000 7f3963e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f396c105330 0x7f396c19a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:12.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.478+0000 7f3963e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f396c105330 0x7f396c19a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:39354/0 (socket says 172.21.15.90:39354) 2024-04-24T09:19:12.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.478+0000 7f3963e00640 1 -- 172.21.15.90:0/2956389731 learned_addr learned my addr 172.21.15.90:0/2956389731 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:19:12.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.478+0000 7f396aa00640 1 --2- 172.21.15.90:0/2956389731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f396c102a00 0x7f396c19a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:12.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.478+0000 7f3963e00640 1 -- 172.21.15.90:0/2956389731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f396c102a00 msgr2=0x7f396c19a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:12.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.478+0000 7f3963e00640 1 --2- 172.21.15.90:0/2956389731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f396c102a00 0x7f396c19a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:12.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.478+0000 7f3963e00640 1 -- 172.21.15.90:0/2956389731 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f396c107c60 msgr2=0x7f396c1a1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:12.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.478+0000 7f3963e00640 1 --2- 172.21.15.90:0/2956389731 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f396c107c60 0x7f396c1a1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:12.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.478+0000 7f3963e00640 1 -- 172.21.15.90:0/2956389731 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f396c1a2070 con 0x7f396c105330 2024-04-24T09:19:12.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.479+0000 7f3963e00640 1 --2- 172.21.15.90:0/2956389731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f396c105330 0x7f396c19a990 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f395400d740 tx=0x7f395400dc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:12.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.479+0000 7f3963400640 1 -- 172.21.15.90:0/2956389731 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3954017070 con 0x7f396c105330 2024-04-24T09:19:12.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.479+0000 7f3963400640 1 -- 172.21.15.90:0/2956389731 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f395400de50 con 0x7f396c105330 2024-04-24T09:19:12.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.479+0000 7f396be00640 1 -- 172.21.15.90:0/2956389731 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f396c1a2360 con 0x7f396c105330 2024-04-24T09:19:12.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.479+0000 7f3963400640 1 -- 172.21.15.90:0/2956389731 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3954004e00 con 0x7f396c105330 2024-04-24T09:19:12.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.479+0000 7f396be00640 1 -- 172.21.15.90:0/2956389731 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f396c1a28b0 con 0x7f396c105330 2024-04-24T09:19:12.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.480+0000 7f396be00640 1 -- 172.21.15.90:0/2956389731 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f396c109750 con 0x7f396c105330 2024-04-24T09:19:12.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.482+0000 7f3963400640 1 -- 172.21.15.90:0/2956389731 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3954004fa0 con 0x7f396c105330 2024-04-24T09:19:12.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.483+0000 7f3963400640 1 --2- 172.21.15.90:0/2956389731 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f39500a3c30 0x7f39500a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:12.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.484+0000 7f3963400640 1 -- 172.21.15.90:0/2956389731 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f39540cfbd0 con 0x7f396c105330 2024-04-24T09:19:12.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.484+0000 7f396aa00640 1 --2- 172.21.15.90:0/2956389731 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f39500a3c30 0x7f39500a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:12.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.484+0000 7f396aa00640 1 --2- 172.21.15.90:0/2956389731 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f39500a3c30 0x7f39500a60f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f395c009bd0 tx=0x7f395c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:12.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.488+0000 7f3963400640 1 -- 172.21.15.90:0/2956389731 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f395409c010 con 0x7f396c105330 2024-04-24T09:19:12.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:12.733+0000 7f396be00640 1 -- 172.21.15.90:0/2956389731 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f396c062c00 con 0x7f396c105330 2024-04-24T09:19:13.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:12 smithi090 ceph-mon[27825]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-24T09:19:13.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:12 smithi090 ceph-mon[27825]: Cluster is now healthy 2024-04-24T09:19:13.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:12 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/3423562913' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-24T09:19:13.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:12 smithi090 ceph-mon[27825]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T09:19:13.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:12 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/2956389731' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-24T09:19:13.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:12 smithi090 ceph-mon[27825]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-24T09:19:13.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:12 smithi149 ceph-mon[32592]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-24T09:19:13.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:12 smithi149 ceph-mon[32592]: Cluster is now healthy 2024-04-24T09:19:13.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:12 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/3423562913' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-24T09:19:13.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:12 smithi149 ceph-mon[32592]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T09:19:13.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:12 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/2956389731' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-24T09:19:13.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:12 smithi149 ceph-mon[32592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-24T09:19:13.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:12 smithi114 ceph-mon[32501]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-24T09:19:13.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:12 smithi114 ceph-mon[32501]: Cluster is now healthy 2024-04-24T09:19:13.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:12 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/3423562913' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-24T09:19:13.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:12 smithi114 ceph-mon[32501]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T09:19:13.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:12 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/2956389731' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-24T09:19:13.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:12 smithi114 ceph-mon[32501]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-24T09:19:13.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.256+0000 7f3963400640 1 -- 172.21.15.90:0/2956389731 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f39540a0310 con 0x7f396c105330 2024-04-24T09:19:13.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.262+0000 7f396be00640 1 -- 172.21.15.90:0/2956389731 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f39500a3c30 msgr2=0x7f39500a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:13.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.262+0000 7f396be00640 1 --2- 172.21.15.90:0/2956389731 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f39500a3c30 0x7f39500a60f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f395c009bd0 tx=0x7f395c005f90 comp rx=0 tx=0).stop 2024-04-24T09:19:13.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.262+0000 7f396be00640 1 -- 172.21.15.90:0/2956389731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f396c105330 msgr2=0x7f396c19a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:13.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.262+0000 7f396be00640 1 --2- 172.21.15.90:0/2956389731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f396c105330 0x7f396c19a990 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f395400d740 tx=0x7f395400dc00 comp rx=0 tx=0).stop 2024-04-24T09:19:13.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.262+0000 7f396be00640 1 -- 172.21.15.90:0/2956389731 shutdown_connections 2024-04-24T09:19:13.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.262+0000 7f396be00640 1 --2- 172.21.15.90:0/2956389731 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f39500a3c30 0x7f39500a60f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:13.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.262+0000 7f396be00640 1 --2- 172.21.15.90:0/2956389731 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f396c107c60 0x7f396c1a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:13.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.262+0000 7f396be00640 1 --2- 172.21.15.90:0/2956389731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f396c105330 0x7f396c19a990 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:13.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.262+0000 7f396be00640 1 --2- 172.21.15.90:0/2956389731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f396c102a00 0x7f396c19a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:13.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.262+0000 7f396be00640 1 -- 172.21.15.90:0/2956389731 >> 172.21.15.90:0/2956389731 conn(0x7f396c0fc3b0 msgr2=0x7f396c101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:13.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.262+0000 7f396be00640 1 -- 172.21.15.90:0/2956389731 shutdown_connections 2024-04-24T09:19:13.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.263+0000 7f396be00640 1 -- 172.21.15.90:0/2956389731 wait complete. 2024-04-24T09:19:13.289 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-24T09:19:13.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.479+0000 7faa7e600640 1 -- 172.21.15.90:0/3108043703 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa78108540 msgr2=0x7faa78108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:13.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.479+0000 7faa7e600640 1 --2- 172.21.15.90:0/3108043703 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa78108540 0x7faa78108960 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7faa60009900 tx=0x7faa6002a410 comp rx=0 tx=0).stop 2024-04-24T09:19:13.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.479+0000 7faa7e600640 1 -- 172.21.15.90:0/3108043703 shutdown_connections 2024-04-24T09:19:13.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.479+0000 7faa7e600640 1 --2- 172.21.15.90:0/3108043703 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa7810a0f0 0x7faa78110980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:13.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.479+0000 7faa7e600640 1 --2- 172.21.15.90:0/3108043703 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faa78109730 0x7faa78109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:13.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.479+0000 7faa7e600640 1 --2- 172.21.15.90:0/3108043703 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa78108540 0x7faa78108960 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:13.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.479+0000 7faa7e600640 1 -- 172.21.15.90:0/3108043703 >> 172.21.15.90:0/3108043703 conn(0x7faa780774d0 msgr2=0x7faa78079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:13.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.479+0000 7faa7e600640 1 -- 172.21.15.90:0/3108043703 shutdown_connections 2024-04-24T09:19:13.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.480+0000 7faa7e600640 1 -- 172.21.15.90:0/3108043703 wait complete. 2024-04-24T09:19:13.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.481+0000 7faa7e600640 1 Processor -- start 2024-04-24T09:19:13.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.481+0000 7faa7e600640 1 -- start start 2024-04-24T09:19:13.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.481+0000 7faa7e600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa78108540 0x7faa7819e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:13.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.481+0000 7faa7e600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faa78109730 0x7faa7819ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:13.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.481+0000 7faa7e600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa7810a0f0 0x7faa781a5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:13.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.481+0000 7faa7e600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7faa78114220 con 0x7faa7810a0f0 2024-04-24T09:19:13.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.481+0000 7faa7e600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7faa781140a0 con 0x7faa78109730 2024-04-24T09:19:13.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.481+0000 7faa7e600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7faa781143a0 con 0x7faa78108540 2024-04-24T09:19:13.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.481+0000 7faa7d200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa78108540 0x7faa7819e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:13.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.481+0000 7faa7d200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa78108540 0x7faa7819e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:39368/0 (socket says 172.21.15.90:39368) 2024-04-24T09:19:13.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.481+0000 7faa7d200640 1 -- 172.21.15.90:0/2721891135 learned_addr learned my addr 172.21.15.90:0/2721891135 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:19:13.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.481+0000 7faa7dc00640 1 --2- 172.21.15.90:0/2721891135 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa7810a0f0 0x7faa781a5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:13.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.481+0000 7faa6fe00640 1 --2- 172.21.15.90:0/2721891135 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faa78109730 0x7faa7819ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:13.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.482+0000 7faa7dc00640 1 -- 172.21.15.90:0/2721891135 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa78108540 msgr2=0x7faa7819e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:13.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.482+0000 7faa7dc00640 1 --2- 172.21.15.90:0/2721891135 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa78108540 0x7faa7819e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:13.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.482+0000 7faa7dc00640 1 -- 172.21.15.90:0/2721891135 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faa78109730 msgr2=0x7faa7819ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:13.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.482+0000 7faa7dc00640 1 --2- 172.21.15.90:0/2721891135 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faa78109730 0x7faa7819ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:13.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.482+0000 7faa7dc00640 1 -- 172.21.15.90:0/2721891135 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa781a6410 con 0x7faa7810a0f0 2024-04-24T09:19:13.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.482+0000 7faa6fe00640 1 --2- 172.21.15.90:0/2721891135 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faa78109730 0x7faa7819ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:19:13.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.482+0000 7faa7dc00640 1 --2- 172.21.15.90:0/2721891135 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa7810a0f0 0x7faa781a5da0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7faa70009fd0 tx=0x7faa7000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:13.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.482+0000 7faa76000640 1 -- 172.21.15.90:0/2721891135 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa7000ce20 con 0x7faa7810a0f0 2024-04-24T09:19:13.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.482+0000 7faa76000640 1 -- 172.21.15.90:0/2721891135 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa70005020 con 0x7faa7810a0f0 2024-04-24T09:19:13.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.482+0000 7faa76000640 1 -- 172.21.15.90:0/2721891135 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa70010720 con 0x7faa7810a0f0 2024-04-24T09:19:13.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.482+0000 7faa7e600640 1 -- 172.21.15.90:0/2721891135 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa781a6700 con 0x7faa7810a0f0 2024-04-24T09:19:13.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.483+0000 7faa7e600640 1 -- 172.21.15.90:0/2721891135 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa781a6c00 con 0x7faa7810a0f0 2024-04-24T09:19:13.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.484+0000 7faa7e600640 1 -- 172.21.15.90:0/2721891135 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa7810f800 con 0x7faa7810a0f0 2024-04-24T09:19:13.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.486+0000 7faa76000640 1 -- 172.21.15.90:0/2721891135 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7faa70005540 con 0x7faa7810a0f0 2024-04-24T09:19:13.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.487+0000 7faa76000640 1 --2- 172.21.15.90:0/2721891135 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7faa580a3bb0 0x7faa580a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:13.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.487+0000 7faa76000640 1 -- 172.21.15.90:0/2721891135 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7faa700d0a00 con 0x7faa7810a0f0 2024-04-24T09:19:13.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.487+0000 7faa7d200640 1 --2- 172.21.15.90:0/2721891135 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7faa580a3bb0 0x7faa580a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:13.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.488+0000 7faa7d200640 1 --2- 172.21.15.90:0/2721891135 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7faa580a3bb0 0x7faa580a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7faa60000c00 tx=0x7faa600315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:13.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.491+0000 7faa76000640 1 -- 172.21.15.90:0/2721891135 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faa7009d490 con 0x7faa7810a0f0 2024-04-24T09:19:13.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:13.737+0000 7faa7e600640 1 -- 172.21.15.90:0/2721891135 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7faa78062c00 con 0x7faa7810a0f0 2024-04-24T09:19:14.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:13 smithi090 ceph-mon[27825]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-24T09:19:14.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:13 smithi090 ceph-mon[27825]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-24T09:19:14.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:13 smithi090 ceph-mon[27825]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-24T09:19:14.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:13 smithi090 ceph-mon[27825]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T09:19:14.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:13 smithi090 ceph-mon[27825]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-24T09:19:14.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:13 smithi090 ceph-mon[27825]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-24T09:19:14.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:13 smithi090 ceph-mon[27825]: pgmap v324: 97 pgs: 15 creating+peering, 8 unknown, 74 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 550 B/s wr, 2 op/s 2024-04-24T09:19:14.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:13 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/2721891135' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-24T09:19:14.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:13 smithi149 ceph-mon[32592]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-24T09:19:14.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:13 smithi149 ceph-mon[32592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-24T09:19:14.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:13 smithi149 ceph-mon[32592]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-24T09:19:14.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:13 smithi149 ceph-mon[32592]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T09:19:14.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:13 smithi149 ceph-mon[32592]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-24T09:19:14.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:13 smithi149 ceph-mon[32592]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-24T09:19:14.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:13 smithi149 ceph-mon[32592]: pgmap v324: 97 pgs: 15 creating+peering, 8 unknown, 74 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 550 B/s wr, 2 op/s 2024-04-24T09:19:14.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:13 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/2721891135' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-24T09:19:14.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:13 smithi114 ceph-mon[32501]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-24T09:19:14.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:13 smithi114 ceph-mon[32501]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-24T09:19:14.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:13 smithi114 ceph-mon[32501]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-24T09:19:14.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:13 smithi114 ceph-mon[32501]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T09:19:14.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:13 smithi114 ceph-mon[32501]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-24T09:19:14.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:13 smithi114 ceph-mon[32501]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-24T09:19:14.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:13 smithi114 ceph-mon[32501]: pgmap v324: 97 pgs: 15 creating+peering, 8 unknown, 74 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 550 B/s wr, 2 op/s 2024-04-24T09:19:14.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:13 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/2721891135' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-24T09:19:14.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.273+0000 7faa76000640 1 -- 172.21.15.90:0/2721891135 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) ==== 121+0+0 (secure 0 0 0) 0x7faa7009cd90 con 0x7faa7810a0f0 2024-04-24T09:19:14.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.277+0000 7faa7e600640 1 -- 172.21.15.90:0/2721891135 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7faa580a3bb0 msgr2=0x7faa580a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:14.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.277+0000 7faa7e600640 1 --2- 172.21.15.90:0/2721891135 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7faa580a3bb0 0x7faa580a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7faa60000c00 tx=0x7faa600315b0 comp rx=0 tx=0).stop 2024-04-24T09:19:14.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.277+0000 7faa7e600640 1 -- 172.21.15.90:0/2721891135 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa7810a0f0 msgr2=0x7faa781a5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:14.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.277+0000 7faa7e600640 1 --2- 172.21.15.90:0/2721891135 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa7810a0f0 0x7faa781a5da0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7faa70009fd0 tx=0x7faa7000ef10 comp rx=0 tx=0).stop 2024-04-24T09:19:14.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.277+0000 7faa7e600640 1 -- 172.21.15.90:0/2721891135 shutdown_connections 2024-04-24T09:19:14.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.277+0000 7faa7e600640 1 --2- 172.21.15.90:0/2721891135 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7faa580a3bb0 0x7faa580a6070 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:14.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.277+0000 7faa7e600640 1 --2- 172.21.15.90:0/2721891135 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa7810a0f0 0x7faa781a5da0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:14.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.277+0000 7faa7e600640 1 --2- 172.21.15.90:0/2721891135 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faa78109730 0x7faa7819ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:14.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.277+0000 7faa7e600640 1 --2- 172.21.15.90:0/2721891135 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa78108540 0x7faa7819e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:14.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.277+0000 7faa7e600640 1 -- 172.21.15.90:0/2721891135 >> 172.21.15.90:0/2721891135 conn(0x7faa780774d0 msgr2=0x7faa7810c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:14.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.277+0000 7faa7e600640 1 -- 172.21.15.90:0/2721891135 shutdown_connections 2024-04-24T09:19:14.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.278+0000 7faa7e600640 1 -- 172.21.15.90:0/2721891135 wait complete. 2024-04-24T09:19:14.307 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-24T09:19:14.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.493+0000 7f93a8200640 1 -- 172.21.15.90:0/2496498401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93a0105e10 msgr2=0x7f93a010c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:14.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.493+0000 7f93a8200640 1 --2- 172.21.15.90:0/2496498401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93a0105e10 0x7f93a010c6a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9398009be0 tx=0x7f939802a6c0 comp rx=0 tx=0).stop 2024-04-24T09:19:14.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.493+0000 7f93a8200640 1 -- 172.21.15.90:0/2496498401 shutdown_connections 2024-04-24T09:19:14.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.493+0000 7f93a8200640 1 --2- 172.21.15.90:0/2496498401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93a0105e10 0x7f93a010c6a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:14.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.493+0000 7f93a8200640 1 --2- 172.21.15.90:0/2496498401 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93a0105450 0x7f93a01058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:14.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.493+0000 7f93a8200640 1 --2- 172.21.15.90:0/2496498401 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93a0104260 0x7f93a0104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:14.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.493+0000 7f93a8200640 1 -- 172.21.15.90:0/2496498401 >> 172.21.15.90:0/2496498401 conn(0x7f93a00ff7f0 msgr2=0x7f93a0101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:14.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.493+0000 7f93a8200640 1 -- 172.21.15.90:0/2496498401 shutdown_connections 2024-04-24T09:19:14.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.493+0000 7f93a8200640 1 -- 172.21.15.90:0/2496498401 wait complete. 2024-04-24T09:19:14.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.494+0000 7f93a8200640 1 Processor -- start 2024-04-24T09:19:14.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.494+0000 7f93a8200640 1 -- start start 2024-04-24T09:19:14.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.494+0000 7f93a8200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93a0104260 0x7f93a019a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:14.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.494+0000 7f93a8200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93a0105450 0x7f93a019aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:14.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.494+0000 7f93a8200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93a0105e10 0x7f93a01a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:14.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.494+0000 7f93a8200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f93a010fef0 con 0x7f93a0105e10 2024-04-24T09:19:14.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.494+0000 7f93a8200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f93a010fd70 con 0x7f93a0105450 2024-04-24T09:19:14.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.494+0000 7f93a8200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f93a0110070 con 0x7f93a0104260 2024-04-24T09:19:14.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.495+0000 7f93a5800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93a0104260 0x7f93a019a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:14.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.495+0000 7f93a4e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93a0105450 0x7f93a019aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:14.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.495+0000 7f93a5800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93a0104260 0x7f93a019a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:39382/0 (socket says 172.21.15.90:39382) 2024-04-24T09:19:14.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.495+0000 7f93a6200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93a0105e10 0x7f93a01a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:14.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.495+0000 7f93a5800640 1 -- 172.21.15.90:0/671970035 learned_addr learned my addr 172.21.15.90:0/671970035 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:19:14.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.495+0000 7f93a4e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93a0105450 0x7f93a019aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.90:51654/0 (socket says 172.21.15.90:51654) 2024-04-24T09:19:14.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.495+0000 7f93a6200640 1 -- 172.21.15.90:0/671970035 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93a0104260 msgr2=0x7f93a019a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:14.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.495+0000 7f93a6200640 1 --2- 172.21.15.90:0/671970035 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93a0104260 0x7f93a019a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:14.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.495+0000 7f93a6200640 1 -- 172.21.15.90:0/671970035 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93a0105450 msgr2=0x7f93a019aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:14.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.495+0000 7f93a6200640 1 --2- 172.21.15.90:0/671970035 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93a0105450 0x7f93a019aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:14.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.495+0000 7f93a6200640 1 -- 172.21.15.90:0/671970035 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93a01a2130 con 0x7f93a0105e10 2024-04-24T09:19:14.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.495+0000 7f93a4e00640 1 --2- 172.21.15.90:0/671970035 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93a0105450 0x7f93a019aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:19:14.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.495+0000 7f93a6200640 1 --2- 172.21.15.90:0/671970035 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93a0105e10 0x7f93a01a1ac0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9398034040 tx=0x7f93980305d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:14.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.496+0000 7f939e000640 1 -- 172.21.15.90:0/671970035 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9398041020 con 0x7f93a0105e10 2024-04-24T09:19:14.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.496+0000 7f939e000640 1 -- 172.21.15.90:0/671970035 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9398030740 con 0x7f93a0105e10 2024-04-24T09:19:14.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.496+0000 7f93a8200640 1 -- 172.21.15.90:0/671970035 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93a01a23c0 con 0x7f93a0105e10 2024-04-24T09:19:14.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.496+0000 7f939e000640 1 -- 172.21.15.90:0/671970035 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9398036040 con 0x7f93a0105e10 2024-04-24T09:19:14.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.496+0000 7f93a8200640 1 -- 172.21.15.90:0/671970035 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93a01a2880 con 0x7f93a0105e10 2024-04-24T09:19:14.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.497+0000 7f93a8200640 1 -- 172.21.15.90:0/671970035 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93a010b580 con 0x7f93a0105e10 2024-04-24T09:19:14.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.499+0000 7f939e000640 1 -- 172.21.15.90:0/671970035 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9398044020 con 0x7f93a0105e10 2024-04-24T09:19:14.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.501+0000 7f939e000640 1 --2- 172.21.15.90:0/671970035 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f93880a3bb0 0x7f93880a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:14.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.501+0000 7f939e000640 1 -- 172.21.15.90:0/671970035 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f93980f0fb0 con 0x7f93a0105e10 2024-04-24T09:19:14.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.501+0000 7f93a5800640 1 --2- 172.21.15.90:0/671970035 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f93880a3bb0 0x7f93880a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:14.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.502+0000 7f93a5800640 1 --2- 172.21.15.90:0/671970035 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f93880a3bb0 0x7f93880a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9394009c60 tx=0x7f9394009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:14.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.505+0000 7f939e000640 1 -- 172.21.15.90:0/671970035 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f939804b020 con 0x7f93a0105e10 2024-04-24T09:19:14.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:14.753+0000 7f93a8200640 1 -- 172.21.15.90:0/671970035 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f93a0062c00 con 0x7f93a0105e10 2024-04-24T09:19:15.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.286+0000 7f939e000640 1 -- 172.21.15.90:0/671970035 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) ==== 115+0+0 (secure 0 0 0) 0x7f93980bda40 con 0x7f93a0105e10 2024-04-24T09:19:15.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.290+0000 7f93a8200640 1 -- 172.21.15.90:0/671970035 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f93880a3bb0 msgr2=0x7f93880a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:15.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.290+0000 7f93a8200640 1 --2- 172.21.15.90:0/671970035 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f93880a3bb0 0x7f93880a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9394009c60 tx=0x7f9394009260 comp rx=0 tx=0).stop 2024-04-24T09:19:15.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.290+0000 7f93a8200640 1 -- 172.21.15.90:0/671970035 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93a0105e10 msgr2=0x7f93a01a1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:15.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.290+0000 7f93a8200640 1 --2- 172.21.15.90:0/671970035 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93a0105e10 0x7f93a01a1ac0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9398034040 tx=0x7f93980305d0 comp rx=0 tx=0).stop 2024-04-24T09:19:15.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.290+0000 7f93a8200640 1 -- 172.21.15.90:0/671970035 shutdown_connections 2024-04-24T09:19:15.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.290+0000 7f93a8200640 1 --2- 172.21.15.90:0/671970035 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f93880a3bb0 0x7f93880a6070 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:15.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.290+0000 7f93a8200640 1 --2- 172.21.15.90:0/671970035 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93a0105e10 0x7f93a01a1ac0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:15.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.290+0000 7f93a8200640 1 --2- 172.21.15.90:0/671970035 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93a0105450 0x7f93a019aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:15.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.290+0000 7f93a8200640 1 --2- 172.21.15.90:0/671970035 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93a0104260 0x7f93a019a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:15.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.290+0000 7f93a8200640 1 -- 172.21.15.90:0/671970035 >> 172.21.15.90:0/671970035 conn(0x7f93a00ff7f0 msgr2=0x7f93a0108750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:15.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.290+0000 7f93a8200640 1 -- 172.21.15.90:0/671970035 shutdown_connections 2024-04-24T09:19:15.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.291+0000 7f93a8200640 1 -- 172.21.15.90:0/671970035 wait complete. 2024-04-24T09:19:15.314 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-04-24T09:19:15.314 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-24T09:19:15.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.494+0000 7f69fcc00640 1 -- 172.21.15.90:0/2636540479 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69f80769c0 msgr2=0x7f69f8076de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:15.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.494+0000 7f69fcc00640 1 --2- 172.21.15.90:0/2636540479 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69f80769c0 0x7f69f8076de0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f69e8009900 tx=0x7f69e802a3f0 comp rx=0 tx=0).stop 2024-04-24T09:19:15.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.494+0000 7f69fcc00640 1 -- 172.21.15.90:0/2636540479 shutdown_connections 2024-04-24T09:19:15.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.494+0000 7f69fcc00640 1 --2- 172.21.15.90:0/2636540479 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69f80785a0 0x7f69f8078e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:15.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.494+0000 7f69fcc00640 1 --2- 172.21.15.90:0/2636540479 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69f8077c00 0x7f69f8078060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:15.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.494+0000 7f69fcc00640 1 --2- 172.21.15.90:0/2636540479 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69f80769c0 0x7f69f8076de0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:15.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.494+0000 7f69fcc00640 1 -- 172.21.15.90:0/2636540479 >> 172.21.15.90:0/2636540479 conn(0x7f69f80ff7f0 msgr2=0x7f69f8101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:15.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.494+0000 7f69fcc00640 1 -- 172.21.15.90:0/2636540479 shutdown_connections 2024-04-24T09:19:15.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.494+0000 7f69fcc00640 1 -- 172.21.15.90:0/2636540479 wait complete. 2024-04-24T09:19:15.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.495+0000 7f69fcc00640 1 Processor -- start 2024-04-24T09:19:15.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.495+0000 7f69fcc00640 1 -- start start 2024-04-24T09:19:15.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.496+0000 7f69fcc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69f80769c0 0x7f69f819e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:15.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.496+0000 7f69fcc00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69f8077c00 0x7f69f819eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:15.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.496+0000 7f69fcc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69f80785a0 0x7f69f81a5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:15.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.496+0000 7f69fcc00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f69f8114350 con 0x7f69f80769c0 2024-04-24T09:19:15.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.496+0000 7f69fcc00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f69f81141d0 con 0x7f69f8077c00 2024-04-24T09:19:15.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.496+0000 7f69fcc00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f69f81144d0 con 0x7f69f80785a0 2024-04-24T09:19:15.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.496+0000 7f69f5e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69f80769c0 0x7f69f819e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:15.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.496+0000 7f69f5e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69f80769c0 0x7f69f819e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55842/0 (socket says 172.21.15.90:55842) 2024-04-24T09:19:15.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.496+0000 7f69f5e00640 1 -- 172.21.15.90:0/2857610957 learned_addr learned my addr 172.21.15.90:0/2857610957 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:19:15.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.496+0000 7f69f5400640 1 --2- 172.21.15.90:0/2857610957 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69f8077c00 0x7f69f819eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:15.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.496+0000 7f69f6800640 1 --2- 172.21.15.90:0/2857610957 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69f80785a0 0x7f69f81a5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:15.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.497+0000 7f69f5e00640 1 -- 172.21.15.90:0/2857610957 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69f80785a0 msgr2=0x7f69f81a5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:15.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.497+0000 7f69f5e00640 1 --2- 172.21.15.90:0/2857610957 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69f80785a0 0x7f69f81a5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:15.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.497+0000 7f69f5e00640 1 -- 172.21.15.90:0/2857610957 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69f8077c00 msgr2=0x7f69f819eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:15.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.497+0000 7f69f5e00640 1 --2- 172.21.15.90:0/2857610957 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69f8077c00 0x7f69f819eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:15.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.497+0000 7f69f5e00640 1 -- 172.21.15.90:0/2857610957 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69f81a65b0 con 0x7f69f80769c0 2024-04-24T09:19:15.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.497+0000 7f69f5400640 1 --2- 172.21.15.90:0/2857610957 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69f8077c00 0x7f69f819eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:19:15.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.497+0000 7f69f5e00640 1 --2- 172.21.15.90:0/2857610957 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69f80769c0 0x7f69f819e990 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f69e8031440 tx=0x7f69e8032e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:15.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.497+0000 7f69eea00640 1 -- 172.21.15.90:0/2857610957 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e803f030 con 0x7f69f80769c0 2024-04-24T09:19:15.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.497+0000 7f69eea00640 1 -- 172.21.15.90:0/2857610957 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69e8003ce0 con 0x7f69f80769c0 2024-04-24T09:19:15.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.497+0000 7f69fcc00640 1 -- 172.21.15.90:0/2857610957 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69f81a6840 con 0x7f69f80769c0 2024-04-24T09:19:15.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.497+0000 7f69eea00640 1 -- 172.21.15.90:0/2857610957 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e8035070 con 0x7f69f80769c0 2024-04-24T09:19:15.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.497+0000 7f69fcc00640 1 -- 172.21.15.90:0/2857610957 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69f81a6d30 con 0x7f69f80769c0 2024-04-24T09:19:15.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.498+0000 7f69fcc00640 1 -- 172.21.15.90:0/2857610957 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69f8110bf0 con 0x7f69f80769c0 2024-04-24T09:19:15.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.501+0000 7f69eea00640 1 -- 172.21.15.90:0/2857610957 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f69e8042030 con 0x7f69f80769c0 2024-04-24T09:19:15.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.503+0000 7f69eea00640 1 --2- 172.21.15.90:0/2857610957 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f69dc0a3bb0 0x7f69dc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:15.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.503+0000 7f69eea00640 1 -- 172.21.15.90:0/2857610957 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f69e80f0cf0 con 0x7f69f80769c0 2024-04-24T09:19:15.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.503+0000 7f69f5400640 1 --2- 172.21.15.90:0/2857610957 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f69dc0a3bb0 0x7f69dc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:15.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.504+0000 7f69f5400640 1 --2- 172.21.15.90:0/2857610957 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f69dc0a3bb0 0x7f69dc0a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f69f819fea0 tx=0x7f69e400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:15.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.506+0000 7f69eea00640 1 -- 172.21.15.90:0/2857610957 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f69e80bd130 con 0x7f69f80769c0 2024-04-24T09:19:15.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:15 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/2721891135' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-24T09:19:15.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:15 smithi090 ceph-mon[27825]: mds.? [v2:172.21.15.149:6838/652734263,v1:172.21.15.149:6839/652734263] up:active 2024-04-24T09:19:15.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:15 smithi090 ceph-mon[27825]: daemon mds.a assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-24T09:19:15.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:15 smithi090 ceph-mon[27825]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-24T09:19:15.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:15 smithi090 ceph-mon[27825]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=a=up:creating} 9 up:standby 2024-04-24T09:19:15.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:15 smithi090 ceph-mon[27825]: daemon mds.a is now active in filesystem cephfs as rank 2 2024-04-24T09:19:15.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:15 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/671970035' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-24T09:19:15.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:15 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/2721891135' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-24T09:19:15.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:15 smithi149 ceph-mon[32592]: mds.? [v2:172.21.15.149:6838/652734263,v1:172.21.15.149:6839/652734263] up:active 2024-04-24T09:19:15.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:15 smithi149 ceph-mon[32592]: daemon mds.a assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-24T09:19:15.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:15 smithi149 ceph-mon[32592]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-24T09:19:15.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:15 smithi149 ceph-mon[32592]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=a=up:creating} 9 up:standby 2024-04-24T09:19:15.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:15 smithi149 ceph-mon[32592]: daemon mds.a is now active in filesystem cephfs as rank 2 2024-04-24T09:19:15.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:15 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/671970035' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-24T09:19:15.676 INFO:journalctl@ceph.mds.b.smithi114.stdout:Apr 24 09:19:15 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-b[63608]: 2024-04-24T09:19:15.216+0000 7fda95279700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T09:19:15.676 INFO:journalctl@ceph.mds.b.smithi114.stdout:Apr 24 09:19:15 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-b[63608]: 2024-04-24T09:19:15.216+0000 7fda95279700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-24T09:19:15.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:15 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/2721891135' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-24T09:19:15.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:15 smithi114 ceph-mon[32501]: mds.? [v2:172.21.15.149:6838/652734263,v1:172.21.15.149:6839/652734263] up:active 2024-04-24T09:19:15.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:15 smithi114 ceph-mon[32501]: daemon mds.a assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-24T09:19:15.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:15 smithi114 ceph-mon[32501]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-24T09:19:15.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:15 smithi114 ceph-mon[32501]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=a=up:creating} 9 up:standby 2024-04-24T09:19:15.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:15 smithi114 ceph-mon[32501]: daemon mds.a is now active in filesystem cephfs as rank 2 2024-04-24T09:19:15.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:15 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/671970035' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-24T09:19:15.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:15.721+0000 7f69fcc00640 1 -- 172.21.15.90:0/2857610957 --> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f69f8062440 con 0x7f69dc0a3bb0 2024-04-24T09:19:16.102 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:19:15 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:19:15.731+0000 7f2d4fe00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T09:19:16.103 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:19:15 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:19:15.731+0000 7f2d4fe00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T09:19:16.103 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:19:15 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:19:15.731+0000 7f2d4fe00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T09:19:16.103 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:19:15 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:19:15.731+0000 7f2d4fe00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T09:19:16.103 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:19:15 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:19:15.731+0000 7f2d4fe00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T09:19:16.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.272+0000 7f69eea00640 1 -- 172.21.15.90:0/2857610957 <== mgr.14152 v2:172.21.15.90:6800/4261662956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f69f8062440 con 0x7f69dc0a3bb0 2024-04-24T09:19:16.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.277+0000 7f69fcc00640 1 -- 172.21.15.90:0/2857610957 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f69dc0a3bb0 msgr2=0x7f69dc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:16.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.277+0000 7f69fcc00640 1 --2- 172.21.15.90:0/2857610957 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f69dc0a3bb0 0x7f69dc0a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f69f819fea0 tx=0x7f69e400b390 comp rx=0 tx=0).stop 2024-04-24T09:19:16.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.277+0000 7f69fcc00640 1 -- 172.21.15.90:0/2857610957 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69f80769c0 msgr2=0x7f69f819e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:16.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.277+0000 7f69fcc00640 1 --2- 172.21.15.90:0/2857610957 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69f80769c0 0x7f69f819e990 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f69e8031440 tx=0x7f69e8032e20 comp rx=0 tx=0).stop 2024-04-24T09:19:16.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.277+0000 7f69fcc00640 1 -- 172.21.15.90:0/2857610957 shutdown_connections 2024-04-24T09:19:16.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.277+0000 7f69fcc00640 1 --2- 172.21.15.90:0/2857610957 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f69dc0a3bb0 0x7f69dc0a6070 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:16.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.277+0000 7f69fcc00640 1 --2- 172.21.15.90:0/2857610957 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69f80785a0 0x7f69f81a5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:16.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.277+0000 7f69fcc00640 1 --2- 172.21.15.90:0/2857610957 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69f8077c00 0x7f69f819eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:16.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.277+0000 7f69fcc00640 1 --2- 172.21.15.90:0/2857610957 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69f80769c0 0x7f69f819e990 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:16.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.277+0000 7f69fcc00640 1 -- 172.21.15.90:0/2857610957 >> 172.21.15.90:0/2857610957 conn(0x7f69f80ff7f0 msgr2=0x7f69f81012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:16.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.277+0000 7f69fcc00640 1 -- 172.21.15.90:0/2857610957 shutdown_connections 2024-04-24T09:19:16.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.277+0000 7f69fcc00640 1 -- 172.21.15.90:0/2857610957 wait complete. 2024-04-24T09:19:16.299 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-24T09:19:16.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.488+0000 7f20c6200640 1 -- 172.21.15.90:0/554018934 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20c010a210 msgr2=0x7f20c0110aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:16.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.488+0000 7f20c6200640 1 --2- 172.21.15.90:0/554018934 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20c010a210 0x7f20c0110aa0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f20b8009ca0 tx=0x7f20b802a8c0 comp rx=0 tx=0).stop 2024-04-24T09:19:16.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.488+0000 7f20c6200640 1 -- 172.21.15.90:0/554018934 shutdown_connections 2024-04-24T09:19:16.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.488+0000 7f20c6200640 1 --2- 172.21.15.90:0/554018934 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20c010a210 0x7f20c0110aa0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:16.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.488+0000 7f20c6200640 1 --2- 172.21.15.90:0/554018934 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20c0109850 0x7f20c0109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:16.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.488+0000 7f20c6200640 1 --2- 172.21.15.90:0/554018934 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20c0108660 0x7f20c0108a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:16.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.488+0000 7f20c6200640 1 -- 172.21.15.90:0/554018934 >> 172.21.15.90:0/554018934 conn(0x7f20c0077600 msgr2=0x7f20c0079a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:16.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.488+0000 7f20c6200640 1 -- 172.21.15.90:0/554018934 shutdown_connections 2024-04-24T09:19:16.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.488+0000 7f20c6200640 1 -- 172.21.15.90:0/554018934 wait complete. 2024-04-24T09:19:16.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.489+0000 7f20c6200640 1 Processor -- start 2024-04-24T09:19:16.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.489+0000 7f20c6200640 1 -- start start 2024-04-24T09:19:16.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.490+0000 7f20c6200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20c0108660 0x7f20c019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:16.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.490+0000 7f20c6200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20c0109850 0x7f20c019ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:16.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.490+0000 7f20c6200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20c010a210 0x7f20c01a5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:16.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.490+0000 7f20c6200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f20c0114310 con 0x7f20c010a210 2024-04-24T09:19:16.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.490+0000 7f20c6200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f20c0114190 con 0x7f20c0108660 2024-04-24T09:19:16.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.490+0000 7f20c6200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f20c0114490 con 0x7f20c0109850 2024-04-24T09:19:16.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.490+0000 7f20bf400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20c0108660 0x7f20c019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:16.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.490+0000 7f20bf400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20c0108660 0x7f20c019e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.90:47080/0 (socket says 172.21.15.90:47080) 2024-04-24T09:19:16.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.490+0000 7f20bf400640 1 -- 172.21.15.90:0/2180213077 learned_addr learned my addr 172.21.15.90:0/2180213077 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:19:16.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.490+0000 7f20bfe00640 1 --2- 172.21.15.90:0/2180213077 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20c010a210 0x7f20c01a5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:16.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.490+0000 7f20b6a00640 1 --2- 172.21.15.90:0/2180213077 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20c0109850 0x7f20c019ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:16.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.490+0000 7f20bf400640 1 -- 172.21.15.90:0/2180213077 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20c0109850 msgr2=0x7f20c019ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:16.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.490+0000 7f20bf400640 1 --2- 172.21.15.90:0/2180213077 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20c0109850 0x7f20c019ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:16.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.490+0000 7f20bf400640 1 -- 172.21.15.90:0/2180213077 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20c010a210 msgr2=0x7f20c01a5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:16.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.490+0000 7f20bf400640 1 --2- 172.21.15.90:0/2180213077 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20c010a210 0x7f20c01a5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:16.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.490+0000 7f20bf400640 1 -- 172.21.15.90:0/2180213077 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20c01a6540 con 0x7f20c0108660 2024-04-24T09:19:16.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.490+0000 7f20bfe00640 1 --2- 172.21.15.90:0/2180213077 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20c010a210 0x7f20c01a5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:19:16.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.491+0000 7f20bf400640 1 --2- 172.21.15.90:0/2180213077 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20c0108660 0x7f20c019e920 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f20b000e9c0 tx=0x7f20b000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:16.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.491+0000 7f20bcc00640 1 -- 172.21.15.90:0/2180213077 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b000cd80 con 0x7f20c0108660 2024-04-24T09:19:16.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.491+0000 7f20c6200640 1 -- 172.21.15.90:0/2180213077 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20c01a6830 con 0x7f20c0108660 2024-04-24T09:19:16.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.491+0000 7f20bcc00640 1 -- 172.21.15.90:0/2180213077 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20b00035f0 con 0x7f20c0108660 2024-04-24T09:19:16.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.491+0000 7f20bcc00640 1 -- 172.21.15.90:0/2180213077 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b0010690 con 0x7f20c0108660 2024-04-24T09:19:16.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.491+0000 7f20c6200640 1 -- 172.21.15.90:0/2180213077 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20c01a6d80 con 0x7f20c0108660 2024-04-24T09:19:16.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.492+0000 7f20c6200640 1 -- 172.21.15.90:0/2180213077 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20c010f920 con 0x7f20c0108660 2024-04-24T09:19:16.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.494+0000 7f20bcc00640 1 -- 172.21.15.90:0/2180213077 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f20b00037a0 con 0x7f20c0108660 2024-04-24T09:19:16.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.495+0000 7f20bcc00640 1 --2- 172.21.15.90:0/2180213077 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f20a40a3bb0 0x7f20a40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:16.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.495+0000 7f20bcc00640 1 -- 172.21.15.90:0/2180213077 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f20b00d0730 con 0x7f20c0108660 2024-04-24T09:19:16.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.495+0000 7f20b6a00640 1 --2- 172.21.15.90:0/2180213077 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f20a40a3bb0 0x7f20a40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:16.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.496+0000 7f20b6a00640 1 --2- 172.21.15.90:0/2180213077 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f20a40a3bb0 0x7f20a40a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f20a80095f0 tx=0x7f20a8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:16.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.499+0000 7f20bcc00640 1 -- 172.21.15.90:0/2180213077 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f20b009d1c0 con 0x7f20c0108660 2024-04-24T09:19:16.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:16 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/671970035' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-24T09:19:16.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:16 smithi090 ceph-mon[27825]: mds.? [v2:172.21.15.90:6840/397593398,v1:172.21.15.90:6841/397593398] up:active 2024-04-24T09:19:16.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:16 smithi090 ceph-mon[27825]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-24T09:19:16.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:16 smithi090 ceph-mon[27825]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=a=up:active} 9 up:standby 2024-04-24T09:19:16.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:16 smithi090 ceph-mon[27825]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=a=up:active,3=c=up:creating} 8 up:standby 2024-04-24T09:19:16.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:16 smithi090 ceph-mon[27825]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-04-24T09:19:16.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:16 smithi090 ceph-mon[27825]: pgmap v325: 97 pgs: 97 active+clean; 580 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 10 op/s 2024-04-24T09:19:16.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:16 smithi090 ceph-mon[27825]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:19:16.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:16 smithi090 ceph-mon[27825]: mds.? [v2:172.21.15.149:6836/2250310587,v1:172.21.15.149:6837/2250310587] up:active 2024-04-24T09:19:16.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:16 smithi090 ceph-mon[27825]: daemon mds.j assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-24T09:19:16.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:16 smithi090 ceph-mon[27825]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-24T09:19:16.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:16 smithi090 ceph-mon[27825]: Cluster is now healthy 2024-04-24T09:19:16.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:16 smithi090 ceph-mon[27825]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=a=up:active,3=c=up:active} 4 up:standby-replay 4 up:standby 2024-04-24T09:19:16.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:16 smithi090 ceph-mon[27825]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-04-24T09:19:16.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:16 smithi090 ceph-mon[27825]: daemon mds.j is now active in filesystem cephfs as rank 4 2024-04-24T09:19:16.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:16 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/671970035' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-24T09:19:16.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:16 smithi149 ceph-mon[32592]: mds.? [v2:172.21.15.90:6840/397593398,v1:172.21.15.90:6841/397593398] up:active 2024-04-24T09:19:16.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:16 smithi149 ceph-mon[32592]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-24T09:19:16.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:16 smithi149 ceph-mon[32592]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=a=up:active} 9 up:standby 2024-04-24T09:19:16.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:16 smithi149 ceph-mon[32592]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=a=up:active,3=c=up:creating} 8 up:standby 2024-04-24T09:19:16.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:16 smithi149 ceph-mon[32592]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-04-24T09:19:16.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:16 smithi149 ceph-mon[32592]: pgmap v325: 97 pgs: 97 active+clean; 580 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 10 op/s 2024-04-24T09:19:16.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:16 smithi149 ceph-mon[32592]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:19:16.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:16 smithi149 ceph-mon[32592]: mds.? [v2:172.21.15.149:6836/2250310587,v1:172.21.15.149:6837/2250310587] up:active 2024-04-24T09:19:16.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:16 smithi149 ceph-mon[32592]: daemon mds.j assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-24T09:19:16.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:16 smithi149 ceph-mon[32592]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-24T09:19:16.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:16 smithi149 ceph-mon[32592]: Cluster is now healthy 2024-04-24T09:19:16.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:16 smithi149 ceph-mon[32592]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=a=up:active,3=c=up:active} 4 up:standby-replay 4 up:standby 2024-04-24T09:19:16.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:16 smithi149 ceph-mon[32592]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-04-24T09:19:16.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:16 smithi149 ceph-mon[32592]: daemon mds.j is now active in filesystem cephfs as rank 4 2024-04-24T09:19:16.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:16 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/671970035' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-24T09:19:16.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:16 smithi114 ceph-mon[32501]: mds.? [v2:172.21.15.90:6840/397593398,v1:172.21.15.90:6841/397593398] up:active 2024-04-24T09:19:16.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:16 smithi114 ceph-mon[32501]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-24T09:19:16.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:16 smithi114 ceph-mon[32501]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=a=up:active} 9 up:standby 2024-04-24T09:19:16.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:16 smithi114 ceph-mon[32501]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=a=up:active,3=c=up:creating} 8 up:standby 2024-04-24T09:19:16.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:16 smithi114 ceph-mon[32501]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-04-24T09:19:16.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:16 smithi114 ceph-mon[32501]: pgmap v325: 97 pgs: 97 active+clean; 580 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 10 op/s 2024-04-24T09:19:16.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:16 smithi114 ceph-mon[32501]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:19:16.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:16 smithi114 ceph-mon[32501]: mds.? [v2:172.21.15.149:6836/2250310587,v1:172.21.15.149:6837/2250310587] up:active 2024-04-24T09:19:16.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:16 smithi114 ceph-mon[32501]: daemon mds.j assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-24T09:19:16.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:16 smithi114 ceph-mon[32501]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-24T09:19:16.677 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:16 smithi114 ceph-mon[32501]: Cluster is now healthy 2024-04-24T09:19:16.677 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:16 smithi114 ceph-mon[32501]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=a=up:active,3=c=up:active} 4 up:standby-replay 4 up:standby 2024-04-24T09:19:16.677 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:16 smithi114 ceph-mon[32501]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-04-24T09:19:16.677 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:16 smithi114 ceph-mon[32501]: daemon mds.j is now active in filesystem cephfs as rank 4 2024-04-24T09:19:16.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.714+0000 7f20c6200640 1 -- 172.21.15.90:0/2180213077 --> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f20c0062440 con 0x7f20a40a3bb0 2024-04-24T09:19:16.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.752+0000 7f20bcc00640 1 -- 172.21.15.90:0/2180213077 <== mgr.14152 v2:172.21.15.90:6800/4261662956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f20c0062440 con 0x7f20a40a3bb0 2024-04-24T09:19:16.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.755+0000 7f20c6200640 1 -- 172.21.15.90:0/2180213077 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f20a40a3bb0 msgr2=0x7f20a40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:16.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.755+0000 7f20c6200640 1 --2- 172.21.15.90:0/2180213077 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f20a40a3bb0 0x7f20a40a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f20a80095f0 tx=0x7f20a8005f90 comp rx=0 tx=0).stop 2024-04-24T09:19:16.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.755+0000 7f20c6200640 1 -- 172.21.15.90:0/2180213077 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20c0108660 msgr2=0x7f20c019e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:16.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.755+0000 7f20c6200640 1 --2- 172.21.15.90:0/2180213077 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20c0108660 0x7f20c019e920 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f20b000e9c0 tx=0x7f20b000ee80 comp rx=0 tx=0).stop 2024-04-24T09:19:16.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.755+0000 7f20c6200640 1 -- 172.21.15.90:0/2180213077 shutdown_connections 2024-04-24T09:19:16.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.755+0000 7f20c6200640 1 --2- 172.21.15.90:0/2180213077 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f20a40a3bb0 0x7f20a40a6070 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:16.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.755+0000 7f20c6200640 1 --2- 172.21.15.90:0/2180213077 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20c010a210 0x7f20c01a5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:16.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.755+0000 7f20c6200640 1 --2- 172.21.15.90:0/2180213077 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20c0109850 0x7f20c019ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:16.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.755+0000 7f20c6200640 1 --2- 172.21.15.90:0/2180213077 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20c0108660 0x7f20c019e920 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:16.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.755+0000 7f20c6200640 1 -- 172.21.15.90:0/2180213077 >> 172.21.15.90:0/2180213077 conn(0x7f20c0077600 msgr2=0x7f20c010caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:16.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.755+0000 7f20c6200640 1 -- 172.21.15.90:0/2180213077 shutdown_connections 2024-04-24T09:19:16.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.755+0000 7f20c6200640 1 -- 172.21.15.90:0/2180213077 wait complete. 2024-04-24T09:19:16.773 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-24T09:19:16.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.966+0000 7f7b4a600640 1 -- 172.21.15.90:0/979414517 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b44074a00 msgr2=0x7f7b44074e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:16.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.966+0000 7f7b4a600640 1 --2- 172.21.15.90:0/979414517 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b44074a00 0x7f7b44074e60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7b30009900 tx=0x7f7b3002a480 comp rx=0 tx=0).stop 2024-04-24T09:19:16.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.966+0000 7f7b4a600640 1 -- 172.21.15.90:0/979414517 shutdown_connections 2024-04-24T09:19:16.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.966+0000 7f7b4a600640 1 --2- 172.21.15.90:0/979414517 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b44109dc0 0x7f7b4410c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:16.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.966+0000 7f7b4a600640 1 --2- 172.21.15.90:0/979414517 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b44074a00 0x7f7b44074e60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:16.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.966+0000 7f7b4a600640 1 --2- 172.21.15.90:0/979414517 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b44075fe0 0x7f7b44074430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:16.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.966+0000 7f7b4a600640 1 -- 172.21.15.90:0/979414517 >> 172.21.15.90:0/979414517 conn(0x7f7b440fd4a0 msgr2=0x7f7b440ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:16.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.966+0000 7f7b4a600640 1 -- 172.21.15.90:0/979414517 shutdown_connections 2024-04-24T09:19:16.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.966+0000 7f7b4a600640 1 -- 172.21.15.90:0/979414517 wait complete. 2024-04-24T09:19:16.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.967+0000 7f7b4a600640 1 Processor -- start 2024-04-24T09:19:16.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.967+0000 7f7b4a600640 1 -- start start 2024-04-24T09:19:16.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.967+0000 7f7b4a600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b44074a00 0x7f7b4419a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:16.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.967+0000 7f7b4a600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b44075fe0 0x7f7b4419a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:16.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.967+0000 7f7b4a600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b44109dc0 0x7f7b441a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:16.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.967+0000 7f7b4a600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4410fe10 con 0x7f7b44074a00 2024-04-24T09:19:16.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.967+0000 7f7b4a600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4410fc90 con 0x7f7b44075fe0 2024-04-24T09:19:16.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.967+0000 7f7b4a600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4410ff90 con 0x7f7b44109dc0 2024-04-24T09:19:16.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.967+0000 7f7b43e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b44075fe0 0x7f7b4419a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:16.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.968+0000 7f7b43e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b44075fe0 0x7f7b4419a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.90:47110/0 (socket says 172.21.15.90:47110) 2024-04-24T09:19:16.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.968+0000 7f7b43e00640 1 -- 172.21.15.90:0/791603933 learned_addr learned my addr 172.21.15.90:0/791603933 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:19:16.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.968+0000 7f7b49200640 1 --2- 172.21.15.90:0/791603933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b44074a00 0x7f7b4419a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:16.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.968+0000 7f7b49c00640 1 --2- 172.21.15.90:0/791603933 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b44109dc0 0x7f7b441a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:16.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.968+0000 7f7b43e00640 1 -- 172.21.15.90:0/791603933 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b44109dc0 msgr2=0x7f7b441a19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:16.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.968+0000 7f7b43e00640 1 --2- 172.21.15.90:0/791603933 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b44109dc0 0x7f7b441a19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:16.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.968+0000 7f7b43e00640 1 -- 172.21.15.90:0/791603933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b44074a00 msgr2=0x7f7b4419a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:16.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.968+0000 7f7b43e00640 1 --2- 172.21.15.90:0/791603933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b44074a00 0x7f7b4419a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:16.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.968+0000 7f7b43e00640 1 -- 172.21.15.90:0/791603933 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b441a2050 con 0x7f7b44075fe0 2024-04-24T09:19:16.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.968+0000 7f7b49200640 1 --2- 172.21.15.90:0/791603933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b44074a00 0x7f7b4419a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:19:16.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.968+0000 7f7b43e00640 1 --2- 172.21.15.90:0/791603933 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b44075fe0 0x7f7b4419a970 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7b30031440 tx=0x7f7b30032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:16.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.969+0000 7f7b41600640 1 -- 172.21.15.90:0/791603933 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3003f030 con 0x7f7b44075fe0 2024-04-24T09:19:16.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.969+0000 7f7b41600640 1 -- 172.21.15.90:0/791603933 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b30003ce0 con 0x7f7b44075fe0 2024-04-24T09:19:16.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.969+0000 7f7b41600640 1 -- 172.21.15.90:0/791603933 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b30035070 con 0x7f7b44075fe0 2024-04-24T09:19:16.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.969+0000 7f7b4a600640 1 -- 172.21.15.90:0/791603933 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b441a22e0 con 0x7f7b44075fe0 2024-04-24T09:19:16.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.969+0000 7f7b4a600640 1 -- 172.21.15.90:0/791603933 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b441a2780 con 0x7f7b44075fe0 2024-04-24T09:19:16.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.970+0000 7f7b4a600640 1 -- 172.21.15.90:0/791603933 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b44108b80 con 0x7f7b44075fe0 2024-04-24T09:19:16.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.972+0000 7f7b41600640 1 -- 172.21.15.90:0/791603933 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7b30042030 con 0x7f7b44075fe0 2024-04-24T09:19:16.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.974+0000 7f7b41600640 1 --2- 172.21.15.90:0/791603933 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f7b2c0a3c30 0x7f7b2c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:16.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.974+0000 7f7b41600640 1 -- 172.21.15.90:0/791603933 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f7b300f0750 con 0x7f7b44075fe0 2024-04-24T09:19:16.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.975+0000 7f7b49200640 1 --2- 172.21.15.90:0/791603933 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f7b2c0a3c30 0x7f7b2c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:16.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.975+0000 7f7b49200640 1 --2- 172.21.15.90:0/791603933 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f7b2c0a3c30 0x7f7b2c0a60f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7b38009fc0 tx=0x7f7b38005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:16.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:16.978+0000 7f7b41600640 1 -- 172.21.15.90:0/791603933 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7b300bcb90 con 0x7f7b44075fe0 2024-04-24T09:19:17.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.208+0000 7f7b4a600640 1 -- 172.21.15.90:0/791603933 --> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f7b440624d0 con 0x7f7b2c0a3c30 2024-04-24T09:19:17.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.215+0000 7f7b41600640 1 -- 172.21.15.90:0/791603933 <== mgr.14152 v2:172.21.15.90:6800/4261662956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f7b440624d0 con 0x7f7b2c0a3c30 2024-04-24T09:19:17.215 INFO:teuthology.orchestra.run.smithi090.stdout:/volumes/qa/sv_0/c012e7ac-f5f2-4c87-82d4-d29adf7deece 2024-04-24T09:19:17.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.217+0000 7f7b4a600640 1 -- 172.21.15.90:0/791603933 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f7b2c0a3c30 msgr2=0x7f7b2c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:17.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.217+0000 7f7b4a600640 1 --2- 172.21.15.90:0/791603933 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f7b2c0a3c30 0x7f7b2c0a60f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7b38009fc0 tx=0x7f7b38005f90 comp rx=0 tx=0).stop 2024-04-24T09:19:17.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.217+0000 7f7b4a600640 1 -- 172.21.15.90:0/791603933 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b44075fe0 msgr2=0x7f7b4419a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:17.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.217+0000 7f7b4a600640 1 --2- 172.21.15.90:0/791603933 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b44075fe0 0x7f7b4419a970 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7b30031440 tx=0x7f7b30032ce0 comp rx=0 tx=0).stop 2024-04-24T09:19:17.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.218+0000 7f7b4a600640 1 -- 172.21.15.90:0/791603933 shutdown_connections 2024-04-24T09:19:17.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.218+0000 7f7b4a600640 1 --2- 172.21.15.90:0/791603933 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f7b2c0a3c30 0x7f7b2c0a60f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:17.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.218+0000 7f7b4a600640 1 --2- 172.21.15.90:0/791603933 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b44109dc0 0x7f7b441a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:17.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.218+0000 7f7b4a600640 1 --2- 172.21.15.90:0/791603933 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b44075fe0 0x7f7b4419a970 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:17.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.218+0000 7f7b4a600640 1 --2- 172.21.15.90:0/791603933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b44074a00 0x7f7b4419a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:17.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.218+0000 7f7b4a600640 1 -- 172.21.15.90:0/791603933 >> 172.21.15.90:0/791603933 conn(0x7f7b440fd4a0 msgr2=0x7f7b44105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:17.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.218+0000 7f7b4a600640 1 -- 172.21.15.90:0/791603933 shutdown_connections 2024-04-24T09:19:17.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.218+0000 7f7b4a600640 1 -- 172.21.15.90:0/791603933 wait complete. 2024-04-24T09:19:17.235 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-24T09:19:17.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.426+0000 7f2587c00640 1 -- 172.21.15.90:0/741446345 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2580103240 msgr2=0x7f25801036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:17.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.426+0000 7f2587c00640 1 --2- 172.21.15.90:0/741446345 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2580103240 0x7f25801036c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2568009ca0 tx=0x7f256802a8c0 comp rx=0 tx=0).stop 2024-04-24T09:19:17.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.426+0000 7f2587c00640 1 -- 172.21.15.90:0/741446345 shutdown_connections 2024-04-24T09:19:17.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.426+0000 7f2587c00640 1 --2- 172.21.15.90:0/741446345 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2580103c00 0x7f258010a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:17.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.426+0000 7f2587c00640 1 --2- 172.21.15.90:0/741446345 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2580103240 0x7f25801036c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:17.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.426+0000 7f2587c00640 1 --2- 172.21.15.90:0/741446345 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2580102050 0x7f2580102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:17.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.426+0000 7f2587c00640 1 -- 172.21.15.90:0/741446345 >> 172.21.15.90:0/741446345 conn(0x7f25800fd5e0 msgr2=0x7f25800ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:17.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.426+0000 7f2587c00640 1 -- 172.21.15.90:0/741446345 shutdown_connections 2024-04-24T09:19:17.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.426+0000 7f2587c00640 1 -- 172.21.15.90:0/741446345 wait complete. 2024-04-24T09:19:17.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.427+0000 7f2587c00640 1 Processor -- start 2024-04-24T09:19:17.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.427+0000 7f2587c00640 1 -- start start 2024-04-24T09:19:17.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.428+0000 7f2587c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2580102050 0x7f258019a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:17.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.428+0000 7f2587c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2580103240 0x7f258019ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:17.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.428+0000 7f2587c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2580103c00 0x7f25801a1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:17.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.428+0000 7f2587c00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f258010dd30 con 0x7f2580103c00 2024-04-24T09:19:17.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.428+0000 7f2587c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f258010dbb0 con 0x7f2580102050 2024-04-24T09:19:17.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.428+0000 7f2587c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f258010deb0 con 0x7f2580103240 2024-04-24T09:19:17.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.428+0000 7f2585200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2580102050 0x7f258019a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:17.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.428+0000 7f2585200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2580102050 0x7f258019a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.90:47132/0 (socket says 172.21.15.90:47132) 2024-04-24T09:19:17.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.428+0000 7f2585200640 1 -- 172.21.15.90:0/1772920102 learned_addr learned my addr 172.21.15.90:0/1772920102 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:19:17.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.428+0000 7f2585c00640 1 --2- 172.21.15.90:0/1772920102 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2580103c00 0x7f25801a1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:17.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.428+0000 7f257fe00640 1 --2- 172.21.15.90:0/1772920102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2580103240 0x7f258019ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:17.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.428+0000 7f2585200640 1 -- 172.21.15.90:0/1772920102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2580103240 msgr2=0x7f258019ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:17.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.428+0000 7f2585200640 1 --2- 172.21.15.90:0/1772920102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2580103240 0x7f258019ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:17.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.428+0000 7f2585200640 1 -- 172.21.15.90:0/1772920102 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2580103c00 msgr2=0x7f25801a1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:17.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.428+0000 7f2585200640 1 --2- 172.21.15.90:0/1772920102 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2580103c00 0x7f25801a1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:17.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.428+0000 7f2585200640 1 -- 172.21.15.90:0/1772920102 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25801a21f0 con 0x7f2580102050 2024-04-24T09:19:17.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.428+0000 7f2585c00640 1 --2- 172.21.15.90:0/1772920102 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2580103c00 0x7f25801a1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:19:17.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.429+0000 7f2585200640 1 --2- 172.21.15.90:0/1772920102 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2580102050 0x7f258019a5d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f257000e9c0 tx=0x7f257000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:17.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.429+0000 7f257d600640 1 -- 172.21.15.90:0/1772920102 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257000cd80 con 0x7f2580102050 2024-04-24T09:19:17.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.429+0000 7f257d600640 1 -- 172.21.15.90:0/1772920102 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25700035f0 con 0x7f2580102050 2024-04-24T09:19:17.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.429+0000 7f257d600640 1 -- 172.21.15.90:0/1772920102 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2570010690 con 0x7f2580102050 2024-04-24T09:19:17.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.429+0000 7f2587c00640 1 -- 172.21.15.90:0/1772920102 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25801a24e0 con 0x7f2580102050 2024-04-24T09:19:17.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.429+0000 7f2587c00640 1 -- 172.21.15.90:0/1772920102 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25801a29e0 con 0x7f2580102050 2024-04-24T09:19:17.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.430+0000 7f2587c00640 1 -- 172.21.15.90:0/1772920102 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25801a2ca0 con 0x7f2580102050 2024-04-24T09:19:17.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.431+0000 7f257d600640 1 -- 172.21.15.90:0/1772920102 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f25700037a0 con 0x7f2580102050 2024-04-24T09:19:17.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.433+0000 7f257d600640 1 --2- 172.21.15.90:0/1772920102 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f25640a3c30 0x7f25640a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:17.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.433+0000 7f257d600640 1 -- 172.21.15.90:0/1772920102 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f25700d07e0 con 0x7f2580102050 2024-04-24T09:19:17.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.433+0000 7f257fe00640 1 --2- 172.21.15.90:0/1772920102 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f25640a3c30 0x7f25640a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:17.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.434+0000 7f257fe00640 1 --2- 172.21.15.90:0/1772920102 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f25640a3c30 0x7f25640a60f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f258019bae0 tx=0x7f2568031bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:17.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.437+0000 7f257d600640 1 -- 172.21.15.90:0/1772920102 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f257009d270 con 0x7f2580102050 2024-04-24T09:19:17.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:17 smithi090 ceph-mon[27825]: from='client.24653 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:19:17.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:17 smithi090 ceph-mon[27825]: mds.? [v2:172.21.15.90:6834/2396995634,v1:172.21.15.90:6835/2396995634] up:active 2024-04-24T09:19:17.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:17 smithi090 ceph-mon[27825]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-24T09:19:17.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:17 smithi090 ceph-mon[27825]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:19:17.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:17 smithi149 ceph-mon[32592]: from='client.24653 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:19:17.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:17 smithi149 ceph-mon[32592]: mds.? [v2:172.21.15.90:6834/2396995634,v1:172.21.15.90:6835/2396995634] up:active 2024-04-24T09:19:17.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:17 smithi149 ceph-mon[32592]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-24T09:19:17.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:17 smithi149 ceph-mon[32592]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:19:17.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.669+0000 7f2587c00640 1 -- 172.21.15.90:0/1772920102 --> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f2580075950 con 0x7f25640a3c30 2024-04-24T09:19:17.675 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:17 smithi114 ceph-mon[32501]: from='client.24653 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:19:17.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:17 smithi114 ceph-mon[32501]: mds.? [v2:172.21.15.90:6834/2396995634,v1:172.21.15.90:6835/2396995634] up:active 2024-04-24T09:19:17.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:17 smithi114 ceph-mon[32501]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-24T09:19:17.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:17 smithi114 ceph-mon[32501]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:19:17.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.705+0000 7f257d600640 1 -- 172.21.15.90:0/1772920102 <== mgr.14152 v2:172.21.15.90:6800/4261662956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2580075950 con 0x7f25640a3c30 2024-04-24T09:19:17.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.708+0000 7f2587c00640 1 -- 172.21.15.90:0/1772920102 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f25640a3c30 msgr2=0x7f25640a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:17.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.708+0000 7f2587c00640 1 --2- 172.21.15.90:0/1772920102 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f25640a3c30 0x7f25640a60f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f258019bae0 tx=0x7f2568031bd0 comp rx=0 tx=0).stop 2024-04-24T09:19:17.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.708+0000 7f2587c00640 1 -- 172.21.15.90:0/1772920102 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2580102050 msgr2=0x7f258019a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:17.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.708+0000 7f2587c00640 1 --2- 172.21.15.90:0/1772920102 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2580102050 0x7f258019a5d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f257000e9c0 tx=0x7f257000ee80 comp rx=0 tx=0).stop 2024-04-24T09:19:17.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.708+0000 7f2587c00640 1 -- 172.21.15.90:0/1772920102 shutdown_connections 2024-04-24T09:19:17.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.708+0000 7f2587c00640 1 --2- 172.21.15.90:0/1772920102 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f25640a3c30 0x7f25640a60f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:17.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.708+0000 7f2587c00640 1 --2- 172.21.15.90:0/1772920102 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2580103c00 0x7f25801a1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:17.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.708+0000 7f2587c00640 1 --2- 172.21.15.90:0/1772920102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2580103240 0x7f258019ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:17.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.708+0000 7f2587c00640 1 --2- 172.21.15.90:0/1772920102 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2580102050 0x7f258019a5d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:17.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.708+0000 7f2587c00640 1 -- 172.21.15.90:0/1772920102 >> 172.21.15.90:0/1772920102 conn(0x7f25800fd5e0 msgr2=0x7f2580074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:17.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.709+0000 7f2587c00640 1 -- 172.21.15.90:0/1772920102 shutdown_connections 2024-04-24T09:19:17.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.709+0000 7f2587c00640 1 -- 172.21.15.90:0/1772920102 wait complete. 2024-04-24T09:19:17.727 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-24T09:19:17.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.939+0000 7fbeb0600640 1 -- 172.21.15.90:0/3425771896 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbea8077c00 msgr2=0x7fbea8078060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:17.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.939+0000 7fbeb0600640 1 --2- 172.21.15.90:0/3425771896 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbea8077c00 0x7fbea8078060 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbe94009900 tx=0x7fbe9402a3f0 comp rx=0 tx=0).stop 2024-04-24T09:19:17.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.939+0000 7fbeb0600640 1 -- 172.21.15.90:0/3425771896 shutdown_connections 2024-04-24T09:19:17.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.939+0000 7fbeb0600640 1 --2- 172.21.15.90:0/3425771896 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbea80785a0 0x7fbea8078e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:17.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.939+0000 7fbeb0600640 1 --2- 172.21.15.90:0/3425771896 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbea8077c00 0x7fbea8078060 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:17.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.939+0000 7fbeb0600640 1 --2- 172.21.15.90:0/3425771896 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbea80769c0 0x7fbea8076de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:17.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.939+0000 7fbeb0600640 1 -- 172.21.15.90:0/3425771896 >> 172.21.15.90:0/3425771896 conn(0x7fbea80ff7f0 msgr2=0x7fbea8101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:17.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.939+0000 7fbeb0600640 1 -- 172.21.15.90:0/3425771896 shutdown_connections 2024-04-24T09:19:17.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.939+0000 7fbeb0600640 1 -- 172.21.15.90:0/3425771896 wait complete. 2024-04-24T09:19:17.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.940+0000 7fbeb0600640 1 Processor -- start 2024-04-24T09:19:17.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.940+0000 7fbeb0600640 1 -- start start 2024-04-24T09:19:17.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.940+0000 7fbeb0600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbea80769c0 0x7fbea819e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:17.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.940+0000 7fbeb0600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbea8077c00 0x7fbea819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:17.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.940+0000 7fbeb0600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbea80785a0 0x7fbea81a5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:17.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.940+0000 7fbeb0600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fbea8114350 con 0x7fbea8077c00 2024-04-24T09:19:17.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.940+0000 7fbeb0600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fbea81141d0 con 0x7fbea80769c0 2024-04-24T09:19:17.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.940+0000 7fbeb0600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbea81144d0 con 0x7fbea80785a0 2024-04-24T09:19:17.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.940+0000 7fbeadc00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbea80769c0 0x7fbea819e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:17.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.940+0000 7fbead200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbea8077c00 0x7fbea819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:17.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.940+0000 7fbeadc00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbea80769c0 0x7fbea819e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.90:47156/0 (socket says 172.21.15.90:47156) 2024-04-24T09:19:17.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.940+0000 7fbeadc00640 1 -- 172.21.15.90:0/715066153 learned_addr learned my addr 172.21.15.90:0/715066153 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:19:17.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.940+0000 7fbeae600640 1 --2- 172.21.15.90:0/715066153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbea80785a0 0x7fbea81a5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:17.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.941+0000 7fbeadc00640 1 -- 172.21.15.90:0/715066153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbea80785a0 msgr2=0x7fbea81a5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:17.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.941+0000 7fbeadc00640 1 --2- 172.21.15.90:0/715066153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbea80785a0 0x7fbea81a5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:17.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.941+0000 7fbeadc00640 1 -- 172.21.15.90:0/715066153 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbea8077c00 msgr2=0x7fbea819eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:17.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.941+0000 7fbeadc00640 1 --2- 172.21.15.90:0/715066153 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbea8077c00 0x7fbea819eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:17.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.941+0000 7fbeadc00640 1 -- 172.21.15.90:0/715066153 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbea81a65a0 con 0x7fbea80769c0 2024-04-24T09:19:17.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.941+0000 7fbeadc00640 1 --2- 172.21.15.90:0/715066153 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbea80769c0 0x7fbea819e980 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c00e9f0 tx=0x7fbe9c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:17.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.941+0000 7fbea6000640 1 -- 172.21.15.90:0/715066153 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c00cd80 con 0x7fbea80769c0 2024-04-24T09:19:17.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.941+0000 7fbea6000640 1 -- 172.21.15.90:0/715066153 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe9c005500 con 0x7fbea80769c0 2024-04-24T09:19:17.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.941+0000 7fbeb0600640 1 -- 172.21.15.90:0/715066153 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbea81a6890 con 0x7fbea80769c0 2024-04-24T09:19:17.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.941+0000 7fbea6000640 1 -- 172.21.15.90:0/715066153 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c010690 con 0x7fbea80769c0 2024-04-24T09:19:17.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.941+0000 7fbeb0600640 1 -- 172.21.15.90:0/715066153 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbea81a6db0 con 0x7fbea80769c0 2024-04-24T09:19:17.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.942+0000 7fbeb0600640 1 -- 172.21.15.90:0/715066153 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbea8110bf0 con 0x7fbea80769c0 2024-04-24T09:19:17.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.944+0000 7fbea6000640 1 -- 172.21.15.90:0/715066153 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbe9c0046c0 con 0x7fbea80769c0 2024-04-24T09:19:17.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.946+0000 7fbea6000640 1 --2- 172.21.15.90:0/715066153 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fbe900a3c30 0x7fbe900a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:17.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.946+0000 7fbea6000640 1 -- 172.21.15.90:0/715066153 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fbe9c0d06d0 con 0x7fbea80769c0 2024-04-24T09:19:17.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.946+0000 7fbead200640 1 --2- 172.21.15.90:0/715066153 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fbe900a3c30 0x7fbe900a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:17.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.946+0000 7fbead200640 1 --2- 172.21.15.90:0/715066153 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fbe900a3c30 0x7fbe900a60f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbe94031440 tx=0x7fbe940315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:17.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:17.949+0000 7fbea6000640 1 -- 172.21.15.90:0/715066153 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbe9c09d160 con 0x7fbea80769c0 2024-04-24T09:19:18.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.184+0000 7fbeb0600640 1 -- 172.21.15.90:0/715066153 --> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fbea8062440 con 0x7fbe900a3c30 2024-04-24T09:19:18.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.196+0000 7fbea6000640 1 -- 172.21.15.90:0/715066153 <== mgr.14152 v2:172.21.15.90:6800/4261662956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fbea8062440 con 0x7fbe900a3c30 2024-04-24T09:19:18.196 INFO:teuthology.orchestra.run.smithi090.stdout:/volumes/qa/sv_1/af9733a5-fcb7-4ed2-92a8-e15d22975ff5 2024-04-24T09:19:18.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.198+0000 7fbeb0600640 1 -- 172.21.15.90:0/715066153 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fbe900a3c30 msgr2=0x7fbe900a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:18.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.198+0000 7fbeb0600640 1 --2- 172.21.15.90:0/715066153 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fbe900a3c30 0x7fbe900a60f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbe94031440 tx=0x7fbe940315b0 comp rx=0 tx=0).stop 2024-04-24T09:19:18.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.198+0000 7fbeb0600640 1 -- 172.21.15.90:0/715066153 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbea80769c0 msgr2=0x7fbea819e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:18.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.198+0000 7fbeb0600640 1 --2- 172.21.15.90:0/715066153 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbea80769c0 0x7fbea819e980 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c00e9f0 tx=0x7fbe9c00edf0 comp rx=0 tx=0).stop 2024-04-24T09:19:18.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.199+0000 7fbeb0600640 1 -- 172.21.15.90:0/715066153 shutdown_connections 2024-04-24T09:19:18.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.199+0000 7fbeb0600640 1 --2- 172.21.15.90:0/715066153 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fbe900a3c30 0x7fbe900a60f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:18.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.199+0000 7fbeb0600640 1 --2- 172.21.15.90:0/715066153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbea80785a0 0x7fbea81a5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:18.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.199+0000 7fbeb0600640 1 --2- 172.21.15.90:0/715066153 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbea8077c00 0x7fbea819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:18.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.199+0000 7fbeb0600640 1 --2- 172.21.15.90:0/715066153 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbea80769c0 0x7fbea819e980 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:18.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.199+0000 7fbeb0600640 1 -- 172.21.15.90:0/715066153 >> 172.21.15.90:0/715066153 conn(0x7fbea80ff7f0 msgr2=0x7fbea81012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:18.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.199+0000 7fbeb0600640 1 -- 172.21.15.90:0/715066153 shutdown_connections 2024-04-24T09:19:18.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.199+0000 7fbeb0600640 1 -- 172.21.15.90:0/715066153 wait complete. 2024-04-24T09:19:18.219 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:19:18.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.431+0000 7f46f2200640 1 -- 172.21.15.90:0/3177738449 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46ec10a030 msgr2=0x7f46ec1108c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:18.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.431+0000 7f46f2200640 1 --2- 172.21.15.90:0/3177738449 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46ec10a030 0x7f46ec1108c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f46e4009ca0 tx=0x7f46e402a8c0 comp rx=0 tx=0).stop 2024-04-24T09:19:18.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.431+0000 7f46f2200640 1 -- 172.21.15.90:0/3177738449 shutdown_connections 2024-04-24T09:19:18.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.431+0000 7f46f2200640 1 --2- 172.21.15.90:0/3177738449 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46ec10a030 0x7f46ec1108c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:18.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.431+0000 7f46f2200640 1 --2- 172.21.15.90:0/3177738449 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46ec109670 0x7f46ec109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:18.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.431+0000 7f46f2200640 1 --2- 172.21.15.90:0/3177738449 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46ec108480 0x7f46ec1088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:18.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.431+0000 7f46f2200640 1 -- 172.21.15.90:0/3177738449 >> 172.21.15.90:0/3177738449 conn(0x7f46ec077440 msgr2=0x7f46ec079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:18.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.431+0000 7f46f2200640 1 -- 172.21.15.90:0/3177738449 shutdown_connections 2024-04-24T09:19:18.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.431+0000 7f46f2200640 1 -- 172.21.15.90:0/3177738449 wait complete. 2024-04-24T09:19:18.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.432+0000 7f46f2200640 1 Processor -- start 2024-04-24T09:19:18.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.433+0000 7f46f2200640 1 -- start start 2024-04-24T09:19:18.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.433+0000 7f46f2200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46ec108480 0x7f46ec19e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:18.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.433+0000 7f46f2200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46ec109670 0x7f46ec19ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:18.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.433+0000 7f46f2200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46ec10a030 0x7f46ec1a5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:18.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.433+0000 7f46f2200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f46ec1140e0 con 0x7f46ec10a030 2024-04-24T09:19:18.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.433+0000 7f46f2200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f46ec113f60 con 0x7f46ec109670 2024-04-24T09:19:18.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.433+0000 7f46f2200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f46ec114260 con 0x7f46ec108480 2024-04-24T09:19:18.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.433+0000 7f46f0e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46ec108480 0x7f46ec19e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:18.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.433+0000 7f46f1800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46ec10a030 0x7f46ec1a5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:18.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.433+0000 7f46f0e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46ec108480 0x7f46ec19e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:43044/0 (socket says 172.21.15.90:43044) 2024-04-24T09:19:18.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.433+0000 7f46ebe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46ec109670 0x7f46ec19ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:18.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.433+0000 7f46f0e00640 1 -- 172.21.15.90:0/636796812 learned_addr learned my addr 172.21.15.90:0/636796812 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:19:18.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.434+0000 7f46f0e00640 1 -- 172.21.15.90:0/636796812 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46ec109670 msgr2=0x7f46ec19ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:18.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.434+0000 7f46f0e00640 1 --2- 172.21.15.90:0/636796812 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46ec109670 0x7f46ec19ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:18.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.434+0000 7f46f0e00640 1 -- 172.21.15.90:0/636796812 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46ec10a030 msgr2=0x7f46ec1a5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:18.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.434+0000 7f46f0e00640 1 --2- 172.21.15.90:0/636796812 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46ec10a030 0x7f46ec1a5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:18.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.434+0000 7f46f0e00640 1 -- 172.21.15.90:0/636796812 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46ec1a6340 con 0x7f46ec108480 2024-04-24T09:19:18.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.434+0000 7f46f1800640 1 --2- 172.21.15.90:0/636796812 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46ec10a030 0x7f46ec1a5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:19:18.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.434+0000 7f46f0e00640 1 --2- 172.21.15.90:0/636796812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46ec108480 0x7f46ec19e720 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f46e000e9c0 tx=0x7f46e000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:18.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.434+0000 7f46e9600640 1 -- 172.21.15.90:0/636796812 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e000cd80 con 0x7f46ec108480 2024-04-24T09:19:18.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.434+0000 7f46e9600640 1 -- 172.21.15.90:0/636796812 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46e00035f0 con 0x7f46ec108480 2024-04-24T09:19:18.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.434+0000 7f46e9600640 1 -- 172.21.15.90:0/636796812 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e0010690 con 0x7f46ec108480 2024-04-24T09:19:18.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.434+0000 7f46f2200640 1 -- 172.21.15.90:0/636796812 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46ec1a6630 con 0x7f46ec108480 2024-04-24T09:19:18.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.434+0000 7f46f2200640 1 -- 172.21.15.90:0/636796812 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46ec1a6b80 con 0x7f46ec108480 2024-04-24T09:19:18.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.436+0000 7f46f2200640 1 -- 172.21.15.90:0/636796812 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46ec10f820 con 0x7f46ec108480 2024-04-24T09:19:18.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.437+0000 7f46e9600640 1 -- 172.21.15.90:0/636796812 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f46e00037a0 con 0x7f46ec108480 2024-04-24T09:19:18.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.439+0000 7f46e9600640 1 --2- 172.21.15.90:0/636796812 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f46d40a3c40 0x7f46d40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:18.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.439+0000 7f46e9600640 1 -- 172.21.15.90:0/636796812 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f46e00d08e0 con 0x7f46ec108480 2024-04-24T09:19:18.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.439+0000 7f46ebe00640 1 --2- 172.21.15.90:0/636796812 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f46d40a3c40 0x7f46d40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:18.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.439+0000 7f46ebe00640 1 --2- 172.21.15.90:0/636796812 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f46d40a3c40 0x7f46d40a6100 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f46ec19fc30 tx=0x7f46d8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:18.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.443+0000 7f46e9600640 1 -- 172.21.15.90:0/636796812 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f46e009d210 con 0x7f46ec108480 2024-04-24T09:19:18.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:18 smithi090 ceph-mon[27825]: from='client.24659 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:19:18.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:18 smithi090 ceph-mon[27825]: pgmap v326: 97 pgs: 97 active+clean; 582 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s rd, 5.9 KiB/s wr, 18 op/s 2024-04-24T09:19:18.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:18 smithi090 ceph-mon[27825]: from='client.24665 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:19:18.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:18 smithi149 ceph-mon[32592]: from='client.24659 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:19:18.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:18 smithi149 ceph-mon[32592]: pgmap v326: 97 pgs: 97 active+clean; 582 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s rd, 5.9 KiB/s wr, 18 op/s 2024-04-24T09:19:18.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:18 smithi149 ceph-mon[32592]: from='client.24665 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:19:18.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:18 smithi114 ceph-mon[32501]: from='client.24659 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:19:18.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:18 smithi114 ceph-mon[32501]: pgmap v326: 97 pgs: 97 active+clean; 582 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s rd, 5.9 KiB/s wr, 18 op/s 2024-04-24T09:19:18.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:18 smithi114 ceph-mon[32501]: from='client.24665 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:19:18.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.689+0000 7f46f2200640 1 -- 172.21.15.90:0/636796812 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46ec02d620 con 0x7f46ec108480 2024-04-24T09:19:18.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.691+0000 7f46e9600640 1 -- 172.21.15.90:0/636796812 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11928 (secure 0 0 0) 0x7f46e009cb10 con 0x7f46ec108480 2024-04-24T09:19:18.691 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:19:18.691 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/227251251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":227251251},{"type":"v1","addr":"172.21.15.90:6837","nonce":227251251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/1904617971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":1904617971},{"type":"v1","addr":"172.21.15.90:6839","nonce":1904617971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T09:19:09.430875+0000","modified":"2024-04-24T09:19:17.259940+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24547,"mds_2":24545,"mds_3":24526,"mds_4":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6833/219377141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":219377141},{"type":"v1","addr":"172.21.15.114:6833","nonce":219377141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"j","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.90:6835/2396995634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2396995634},{"type":"v1","addr":"172.21.15.90:6835","nonce":2396995634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/779642417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":779642417},{"type":"v1","addr":"172.21.15.149:6833","nonce":779642417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6835/2565340800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":2565340800},{"type":"v1","addr":"172.21.15.114:6835","nonce":2565340800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6837/875324280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":875324280},{"type":"v1","addr":"172.21.15.114:6837","nonce":875324280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/3157375624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3157375624},{"type":"v1","addr":"172.21.15.149:6835","nonce":3157375624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.149:6837/2250310587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2250310587},{"type":"v1","addr":"172.21.15.149:6837","nonce":2250310587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.90:6841/397593398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":397593398},{"type":"v1","addr":"172.21.15.90:6841","nonce":397593398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.149:6839/652734263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":652734263},{"type":"v1","addr":"172.21.15.149:6839","nonce":652734263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6839/1025403710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":1025403710},{"type":"v1","addr":"172.21.15.114:6839","nonce":1025403710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24545,24557,24446,24526,24547]},"id":1}]} 2024-04-24T09:19:18.691 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 27 2024-04-24T09:19:18.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.693+0000 7f46f2200640 1 -- 172.21.15.90:0/636796812 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f46d40a3c40 msgr2=0x7f46d40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:18.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.693+0000 7f46f2200640 1 --2- 172.21.15.90:0/636796812 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f46d40a3c40 0x7f46d40a6100 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f46ec19fc30 tx=0x7f46d8005f90 comp rx=0 tx=0).stop 2024-04-24T09:19:18.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.693+0000 7f46f2200640 1 -- 172.21.15.90:0/636796812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46ec108480 msgr2=0x7f46ec19e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:18.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.693+0000 7f46f2200640 1 --2- 172.21.15.90:0/636796812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46ec108480 0x7f46ec19e720 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f46e000e9c0 tx=0x7f46e000ee80 comp rx=0 tx=0).stop 2024-04-24T09:19:18.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.693+0000 7f46f2200640 1 -- 172.21.15.90:0/636796812 shutdown_connections 2024-04-24T09:19:18.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.693+0000 7f46f2200640 1 --2- 172.21.15.90:0/636796812 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f46d40a3c40 0x7f46d40a6100 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:18.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.693+0000 7f46f2200640 1 --2- 172.21.15.90:0/636796812 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46ec10a030 0x7f46ec1a5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:18.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.693+0000 7f46f2200640 1 --2- 172.21.15.90:0/636796812 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46ec109670 0x7f46ec19ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:18.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.693+0000 7f46f2200640 1 --2- 172.21.15.90:0/636796812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46ec108480 0x7f46ec19e720 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:18.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.693+0000 7f46f2200640 1 -- 172.21.15.90:0/636796812 >> 172.21.15.90:0/636796812 conn(0x7f46ec077440 msgr2=0x7f46ec10c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:18.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.693+0000 7f46f2200640 1 -- 172.21.15.90:0/636796812 shutdown_connections 2024-04-24T09:19:18.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.694+0000 7f46f2200640 1 -- 172.21.15.90:0/636796812 wait complete. 2024-04-24T09:19:18.712 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T09:19:18.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.910+0000 7f2d52c00640 1 -- 172.21.15.90:0/2228071517 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d4c109850 msgr2=0x7f2d4c109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:18.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.910+0000 7f2d52c00640 1 --2- 172.21.15.90:0/2228071517 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d4c109850 0x7f2d4c109cd0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2d34009ca0 tx=0x7f2d3402a8c0 comp rx=0 tx=0).stop 2024-04-24T09:19:18.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.910+0000 7f2d52c00640 1 -- 172.21.15.90:0/2228071517 shutdown_connections 2024-04-24T09:19:18.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.910+0000 7f2d52c00640 1 --2- 172.21.15.90:0/2228071517 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2d4c10a210 0x7f2d4c110aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:18.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.910+0000 7f2d52c00640 1 --2- 172.21.15.90:0/2228071517 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d4c109850 0x7f2d4c109cd0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:18.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.910+0000 7f2d52c00640 1 --2- 172.21.15.90:0/2228071517 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d4c108660 0x7f2d4c108a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:18.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.910+0000 7f2d52c00640 1 -- 172.21.15.90:0/2228071517 >> 172.21.15.90:0/2228071517 conn(0x7f2d4c077620 msgr2=0x7f2d4c079a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:18.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.910+0000 7f2d52c00640 1 -- 172.21.15.90:0/2228071517 shutdown_connections 2024-04-24T09:19:18.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.910+0000 7f2d52c00640 1 -- 172.21.15.90:0/2228071517 wait complete. 2024-04-24T09:19:18.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.911+0000 7f2d52c00640 1 Processor -- start 2024-04-24T09:19:18.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.911+0000 7f2d52c00640 1 -- start start 2024-04-24T09:19:18.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.912+0000 7f2d52c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d4c108660 0x7f2d4c19e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:18.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.912+0000 7f2d52c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d4c109850 0x7f2d4c19eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:18.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.912+0000 7f2d52c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2d4c10a210 0x7f2d4c1a5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:18.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.912+0000 7f2d52c00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4c1142f0 con 0x7f2d4c108660 2024-04-24T09:19:18.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.912+0000 7f2d52c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4c114170 con 0x7f2d4c10a210 2024-04-24T09:19:18.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.912+0000 7f2d52c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4c114470 con 0x7f2d4c109850 2024-04-24T09:19:18.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.912+0000 7f2d4be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d4c108660 0x7f2d4c19e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:18.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.912+0000 7f2d4b400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d4c109850 0x7f2d4c19eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:18.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.912+0000 7f2d4b400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d4c109850 0x7f2d4c19eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:43066/0 (socket says 172.21.15.90:43066) 2024-04-24T09:19:18.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.912+0000 7f2d4b400640 1 -- 172.21.15.90:0/401846159 learned_addr learned my addr 172.21.15.90:0/401846159 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:19:18.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.912+0000 7f2d50c00640 1 --2- 172.21.15.90:0/401846159 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2d4c10a210 0x7f2d4c1a5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:18.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.912+0000 7f2d50c00640 1 -- 172.21.15.90:0/401846159 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d4c109850 msgr2=0x7f2d4c19eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:18.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.912+0000 7f2d50c00640 1 --2- 172.21.15.90:0/401846159 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d4c109850 0x7f2d4c19eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:18.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.912+0000 7f2d50c00640 1 -- 172.21.15.90:0/401846159 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d4c108660 msgr2=0x7f2d4c19e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:18.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.912+0000 7f2d50c00640 1 --2- 172.21.15.90:0/401846159 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d4c108660 0x7f2d4c19e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:18.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.912+0000 7f2d50c00640 1 -- 172.21.15.90:0/401846159 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d4c1a65b0 con 0x7f2d4c10a210 2024-04-24T09:19:18.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.913+0000 7f2d50c00640 1 --2- 172.21.15.90:0/401846159 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2d4c10a210 0x7f2d4c1a5f40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2d4400e970 tx=0x7f2d4400ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:18.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.913+0000 7f2d48c00640 1 -- 172.21.15.90:0/401846159 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4400ccc0 con 0x7f2d4c10a210 2024-04-24T09:19:18.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.913+0000 7f2d48c00640 1 -- 172.21.15.90:0/401846159 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d44005020 con 0x7f2d4c10a210 2024-04-24T09:19:18.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.913+0000 7f2d52c00640 1 -- 172.21.15.90:0/401846159 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d4c1a68a0 con 0x7f2d4c10a210 2024-04-24T09:19:18.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.913+0000 7f2d48c00640 1 -- 172.21.15.90:0/401846159 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d44004a00 con 0x7f2d4c10a210 2024-04-24T09:19:18.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.913+0000 7f2d52c00640 1 -- 172.21.15.90:0/401846159 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d4c1a6df0 con 0x7f2d4c10a210 2024-04-24T09:19:18.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.914+0000 7f2d52c00640 1 -- 172.21.15.90:0/401846159 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d4c10f9d0 con 0x7f2d4c10a210 2024-04-24T09:19:18.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.916+0000 7f2d48c00640 1 -- 172.21.15.90:0/401846159 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2d440051d0 con 0x7f2d4c10a210 2024-04-24T09:19:18.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.918+0000 7f2d48c00640 1 --2- 172.21.15.90:0/401846159 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f2d300a3c40 0x7f2d300a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:18.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.918+0000 7f2d48c00640 1 -- 172.21.15.90:0/401846159 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f2d440d0430 con 0x7f2d4c10a210 2024-04-24T09:19:18.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.918+0000 7f2d4be00640 1 --2- 172.21.15.90:0/401846159 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f2d300a3c40 0x7f2d300a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:18.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.918+0000 7f2d4be00640 1 --2- 172.21.15.90:0/401846159 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f2d300a3c40 0x7f2d300a6100 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c1096b0 tx=0x7f2d3c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:18.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:18.922+0000 7f2d48c00640 1 -- 172.21.15.90:0/401846159 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2d4409cd60 con 0x7f2d4c10a210 2024-04-24T09:19:19.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.129+0000 7f2d52c00640 1 -- 172.21.15.90:0/401846159 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2d4c02d820 con 0x7f2d4c10a210 2024-04-24T09:19:19.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.131+0000 7f2d48c00640 1 -- 172.21.15.90:0/401846159 <== mon.2 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22231 (secure 0 0 0) 0x7f2d4409c660 con 0x7f2d4c10a210 2024-04-24T09:19:19.131 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:19:19.131 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":86,"fsid":"30ee9be2-021a-11ef-bc93-c7b262605968","created":"2024-04-24T09:09:49.496019+0000","modified":"2024-04-24T09:19:10.816601+0000","last_up_change":"2024-04-24T09:18:10.455632+0000","last_in_change":"2024-04-24T09:17:50.600366+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T09:13:45.548774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T09:19:07.185580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T09:19:08.374818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13a59b9f-b46e-4ec4-90af-bb7a3d5c753b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6803","nonce":2864288066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6805","nonce":2864288066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6809","nonce":2864288066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6807","nonce":2864288066}]},"public_addr":"172.21.15.90:6803/2864288066","cluster_addr":"172.21.15.90:6805/2864288066","heartbeat_back_addr":"172.21.15.90:6809/2864288066","heartbeat_front_addr":"172.21.15.90:6807/2864288066","state":["exists","up"]},{"osd":1,"uuid":"1567ce8f-24e6-42cf-bf64-f6f0c534a9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6801","nonce":344222414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6803","nonce":344222414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6807","nonce":344222414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6805","nonce":344222414}]},"public_addr":"172.21.15.114:6801/344222414","cluster_addr":"172.21.15.114:6803/344222414","heartbeat_back_addr":"172.21.15.114:6807/344222414","heartbeat_front_addr":"172.21.15.114:6805/344222414","state":["exists","up"]},{"osd":2,"uuid":"f861b304-9c7d-4629-8522-37f33a747945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6801","nonce":3257067419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6803","nonce":3257067419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6807","nonce":3257067419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6805","nonce":3257067419}]},"public_addr":"172.21.15.149:6801/3257067419","cluster_addr":"172.21.15.149:6803/3257067419","heartbeat_back_addr":"172.21.15.149:6807/3257067419","heartbeat_front_addr":"172.21.15.149:6805/3257067419","state":["exists","up"]},{"osd":3,"uuid":"9377ab6e-5996-45fb-93a9-475a2e5f17a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6811","nonce":3068524970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6813","nonce":3068524970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6817","nonce":3068524970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6815","nonce":3068524970}]},"public_addr":"172.21.15.90:6811/3068524970","cluster_addr":"172.21.15.90:6813/3068524970","heartbeat_back_addr":"172.21.15.90:6817/3068524970","heartbeat_front_addr":"172.21.15.90:6815/3068524970","state":["exists","up"]},{"osd":4,"uuid":"e2ddd19e-e641-4d61-b342-02b5f12b1c4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6809","nonce":2782015522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6811","nonce":2782015522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6815","nonce":2782015522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6813","nonce":2782015522}]},"public_addr":"172.21.15.114:6809/2782015522","cluster_addr":"172.21.15.114:6811/2782015522","heartbeat_back_addr":"172.21.15.114:6815/2782015522","heartbeat_front_addr":"172.21.15.114:6813/2782015522","state":["exists","up"]},{"osd":5,"uuid":"b6ff9d10-5210-4b46-a192-6dcbf9d1f156","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6809","nonce":4266460479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6811","nonce":4266460479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6815","nonce":4266460479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6813","nonce":4266460479}]},"public_addr":"172.21.15.149:6809/4266460479","cluster_addr":"172.21.15.149:6811/4266460479","heartbeat_back_addr":"172.21.15.149:6815/4266460479","heartbeat_front_addr":"172.21.15.149:6813/4266460479","state":["exists","up"]},{"osd":6,"uuid":"4287b5a1-f953-421c-bf62-5e390bee834d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6819","nonce":1121461276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6821","nonce":1121461276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6825","nonce":1121461276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6823","nonce":1121461276}]},"public_addr":"172.21.15.90:6819/1121461276","cluster_addr":"172.21.15.90:6821/1121461276","heartbeat_back_addr":"172.21.15.90:6825/1121461276","heartbeat_front_addr":"172.21.15.90:6823/1121461276","state":["exists","up"]},{"osd":7,"uuid":"a54e507d-d6ba-4bf2-9a7f-a7b95c3859bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6817","nonce":3222546280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6819","nonce":3222546280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6823","nonce":3222546280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6821","nonce":3222546280}]},"public_addr":"172.21.15.114:6817/3222546280","cluster_addr":"172.21.15.114:6819/3222546280","heartbeat_back_addr":"172.21.15.114:6823/3222546280","heartbeat_front_addr":"172.21.15.114:6821/3222546280","state":["exists","up"]},{"osd":8,"uuid":"9950b010-5865-4109-ae37-582b1dbd2f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6817","nonce":157615149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6819","nonce":157615149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6823","nonce":157615149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6821","nonce":157615149}]},"public_addr":"172.21.15.149:6817/157615149","cluster_addr":"172.21.15.149:6819/157615149","heartbeat_back_addr":"172.21.15.149:6823/157615149","heartbeat_front_addr":"172.21.15.149:6821/157615149","state":["exists","up"]},{"osd":9,"uuid":"31e2635c-e234-421f-9757-592fd673c936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6827","nonce":1307318076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6829","nonce":1307318076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6833","nonce":1307318076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6831","nonce":1307318076}]},"public_addr":"172.21.15.90:6827/1307318076","cluster_addr":"172.21.15.90:6829/1307318076","heartbeat_back_addr":"172.21.15.90:6833/1307318076","heartbeat_front_addr":"172.21.15.90:6831/1307318076","state":["exists","up"]},{"osd":10,"uuid":"46a3b2a6-e6e5-4578-b4bb-f3e998b9bc95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6825","nonce":1719802140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6827","nonce":1719802140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6831","nonce":1719802140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6829","nonce":1719802140}]},"public_addr":"172.21.15.114:6825/1719802140","cluster_addr":"172.21.15.114:6827/1719802140","heartbeat_back_addr":"172.21.15.114:6831/1719802140","heartbeat_front_addr":"172.21.15.114:6829/1719802140","state":["exists","up"]},{"osd":11,"uuid":"ccd4456b-e947-46cf-a779-148ffa289b58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6825","nonce":1737275166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6827","nonce":1737275166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6831","nonce":1737275166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6829","nonce":1737275166}]},"public_addr":"172.21.15.149:6825/1737275166","cluster_addr":"172.21.15.149:6827/1737275166","heartbeat_back_addr":"172.21.15.149:6831/1737275166","heartbeat_front_addr":"172.21.15.149:6829/1737275166","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:12:53.015078+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:13:18.047053+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:13:41.935891+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:14:15.095388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:14:42.512355+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:15:08.113775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:15:43.828550+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:16:10.901909+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:16:36.422962+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:17:13.924446+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:17:42.814401+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:18:08.672793+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/3090852580":"2024-04-25T09:10:57.377655+0000","172.21.15.90:0/687197394":"2024-04-25T09:10:57.377655+0000","172.21.15.90:6801/1809038230":"2024-04-25T09:10:57.377655+0000","172.21.15.90:6800/1809038230":"2024-04-25T09:10:57.377655+0000","172.21.15.90:0/2964149071":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/3617388113":"2024-04-25T09:10:22.599674+0000","172.21.15.90:6801/3089140842":"2024-04-25T09:10:22.599674+0000","172.21.15.90:6800/3089140842":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/2840416149":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/305432710":"2024-04-25T09:10:57.377655+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T09:19:19.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.133+0000 7f2d52c00640 1 -- 172.21.15.90:0/401846159 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f2d300a3c40 msgr2=0x7f2d300a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:19.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.133+0000 7f2d52c00640 1 --2- 172.21.15.90:0/401846159 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f2d300a3c40 0x7f2d300a6100 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c1096b0 tx=0x7f2d3c005f90 comp rx=0 tx=0).stop 2024-04-24T09:19:19.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.133+0000 7f2d52c00640 1 -- 172.21.15.90:0/401846159 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2d4c10a210 msgr2=0x7f2d4c1a5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:19.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.133+0000 7f2d52c00640 1 --2- 172.21.15.90:0/401846159 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2d4c10a210 0x7f2d4c1a5f40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2d4400e970 tx=0x7f2d4400ee30 comp rx=0 tx=0).stop 2024-04-24T09:19:19.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.134+0000 7f2d52c00640 1 -- 172.21.15.90:0/401846159 shutdown_connections 2024-04-24T09:19:19.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.134+0000 7f2d52c00640 1 --2- 172.21.15.90:0/401846159 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f2d300a3c40 0x7f2d300a6100 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:19.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.134+0000 7f2d52c00640 1 --2- 172.21.15.90:0/401846159 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2d4c10a210 0x7f2d4c1a5f40 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:19.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.134+0000 7f2d52c00640 1 --2- 172.21.15.90:0/401846159 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d4c109850 0x7f2d4c19eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:19.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.134+0000 7f2d52c00640 1 --2- 172.21.15.90:0/401846159 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d4c108660 0x7f2d4c19e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:19.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.134+0000 7f2d52c00640 1 -- 172.21.15.90:0/401846159 >> 172.21.15.90:0/401846159 conn(0x7f2d4c077620 msgr2=0x7f2d4c10cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:19.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.134+0000 7f2d52c00640 1 -- 172.21.15.90:0/401846159 shutdown_connections 2024-04-24T09:19:19.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.134+0000 7f2d52c00640 1 -- 172.21.15.90:0/401846159 wait complete. 2024-04-24T09:19:19.154 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-24T09:19:19.154 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T09:19:19.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.355+0000 7fcb60a00640 1 -- 172.21.15.90:0/124081732 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb5c10a210 msgr2=0x7fcb5c110aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:19.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.355+0000 7fcb60a00640 1 --2- 172.21.15.90:0/124081732 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb5c10a210 0x7fcb5c110aa0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcb54009ca0 tx=0x7fcb5402a8c0 comp rx=0 tx=0).stop 2024-04-24T09:19:19.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.356+0000 7fcb60a00640 1 -- 172.21.15.90:0/124081732 shutdown_connections 2024-04-24T09:19:19.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.356+0000 7fcb60a00640 1 --2- 172.21.15.90:0/124081732 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb5c10a210 0x7fcb5c110aa0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:19.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.356+0000 7fcb60a00640 1 --2- 172.21.15.90:0/124081732 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb5c109850 0x7fcb5c109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:19.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.356+0000 7fcb60a00640 1 --2- 172.21.15.90:0/124081732 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb5c108660 0x7fcb5c108a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:19.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.356+0000 7fcb60a00640 1 -- 172.21.15.90:0/124081732 >> 172.21.15.90:0/124081732 conn(0x7fcb5c077620 msgr2=0x7fcb5c079a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:19.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.356+0000 7fcb60a00640 1 -- 172.21.15.90:0/124081732 shutdown_connections 2024-04-24T09:19:19.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.356+0000 7fcb60a00640 1 -- 172.21.15.90:0/124081732 wait complete. 2024-04-24T09:19:19.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.357+0000 7fcb60a00640 1 Processor -- start 2024-04-24T09:19:19.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.357+0000 7fcb60a00640 1 -- start start 2024-04-24T09:19:19.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.357+0000 7fcb60a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb5c108660 0x7fcb5c19e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:19.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.357+0000 7fcb60a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb5c109850 0x7fcb5c19ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:19.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.357+0000 7fcb60a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb5c10a210 0x7fcb5c1a5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:19.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.357+0000 7fcb60a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5c1142c0 con 0x7fcb5c109850 2024-04-24T09:19:19.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.357+0000 7fcb60a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5c114140 con 0x7fcb5c10a210 2024-04-24T09:19:19.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.357+0000 7fcb60a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5c114440 con 0x7fcb5c108660 2024-04-24T09:19:19.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.357+0000 7fcb59e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb5c108660 0x7fcb5c19e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:19.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.357+0000 7fcb59400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb5c109850 0x7fcb5c19ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:19.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.357+0000 7fcb59e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb5c108660 0x7fcb5c19e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:43090/0 (socket says 172.21.15.90:43090) 2024-04-24T09:19:19.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.357+0000 7fcb59e00640 1 -- 172.21.15.90:0/565880364 learned_addr learned my addr 172.21.15.90:0/565880364 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:19:19.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.358+0000 7fcb5a800640 1 --2- 172.21.15.90:0/565880364 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb5c10a210 0x7fcb5c1a5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:19.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.358+0000 7fcb59e00640 1 -- 172.21.15.90:0/565880364 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb5c10a210 msgr2=0x7fcb5c1a5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:19.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.358+0000 7fcb59e00640 1 --2- 172.21.15.90:0/565880364 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb5c10a210 0x7fcb5c1a5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:19.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.358+0000 7fcb59e00640 1 -- 172.21.15.90:0/565880364 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb5c109850 msgr2=0x7fcb5c19ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:19.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.358+0000 7fcb59e00640 1 --2- 172.21.15.90:0/565880364 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb5c109850 0x7fcb5c19ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:19.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.358+0000 7fcb59e00640 1 -- 172.21.15.90:0/565880364 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb5c1a64f0 con 0x7fcb5c108660 2024-04-24T09:19:19.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.358+0000 7fcb59e00640 1 --2- 172.21.15.90:0/565880364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb5c108660 0x7fcb5c19e8d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c00e9c0 tx=0x7fcb4c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:19.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.358+0000 7fcb52a00640 1 -- 172.21.15.90:0/565880364 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c00cd80 con 0x7fcb5c108660 2024-04-24T09:19:19.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.358+0000 7fcb52a00640 1 -- 172.21.15.90:0/565880364 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb4c0035f0 con 0x7fcb5c108660 2024-04-24T09:19:19.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.358+0000 7fcb52a00640 1 -- 172.21.15.90:0/565880364 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c010690 con 0x7fcb5c108660 2024-04-24T09:19:19.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.358+0000 7fcb60a00640 1 -- 172.21.15.90:0/565880364 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb5c1a67e0 con 0x7fcb5c108660 2024-04-24T09:19:19.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.359+0000 7fcb60a00640 1 -- 172.21.15.90:0/565880364 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb5c1a6d80 con 0x7fcb5c108660 2024-04-24T09:19:19.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.360+0000 7fcb60a00640 1 -- 172.21.15.90:0/565880364 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb5c10f980 con 0x7fcb5c108660 2024-04-24T09:19:19.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.361+0000 7fcb52a00640 1 -- 172.21.15.90:0/565880364 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcb4c010830 con 0x7fcb5c108660 2024-04-24T09:19:19.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.363+0000 7fcb52a00640 1 --2- 172.21.15.90:0/565880364 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fcb400a3cc0 0x7fcb400a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:19.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.363+0000 7fcb52a00640 1 -- 172.21.15.90:0/565880364 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fcb4c0d0810 con 0x7fcb5c108660 2024-04-24T09:19:19.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.363+0000 7fcb59400640 1 --2- 172.21.15.90:0/565880364 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fcb400a3cc0 0x7fcb400a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:19.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.364+0000 7fcb59400640 1 --2- 172.21.15.90:0/565880364 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fcb400a3cc0 0x7fcb400a6180 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcb480095f0 tx=0x7fcb48005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:19.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.367+0000 7fcb52a00640 1 -- 172.21.15.90:0/565880364 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcb4c09d180 con 0x7fcb5c108660 2024-04-24T09:19:19.403 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:19 smithi090 ceph-mon[27825]: from='client.24671 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:19:19.403 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:19 smithi090 ceph-mon[27825]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-24T09:19:19.403 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:19 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/636796812' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:19:19.403 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:19 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/401846159' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:19:19.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.567+0000 7fcb60a00640 1 -- 172.21.15.90:0/565880364 --> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcb5c062440 con 0x7fcb400a3cc0 2024-04-24T09:19:19.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.577+0000 7fcb52a00640 1 -- 172.21.15.90:0/565880364 <== mgr.14152 v2:172.21.15.90:6800/4261662956 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258947 (secure 0 0 0) 0x7fcb5c062440 con 0x7fcb400a3cc0 2024-04-24T09:19:19.580 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:19:19.582 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-24T09:19:19.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.582+0000 7fcb60a00640 1 -- 172.21.15.90:0/565880364 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fcb400a3cc0 msgr2=0x7fcb400a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:19.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.582+0000 7fcb60a00640 1 --2- 172.21.15.90:0/565880364 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fcb400a3cc0 0x7fcb400a6180 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcb480095f0 tx=0x7fcb48005f90 comp rx=0 tx=0).stop 2024-04-24T09:19:19.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.582+0000 7fcb60a00640 1 -- 172.21.15.90:0/565880364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb5c108660 msgr2=0x7fcb5c19e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:19.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.582+0000 7fcb60a00640 1 --2- 172.21.15.90:0/565880364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb5c108660 0x7fcb5c19e8d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c00e9c0 tx=0x7fcb4c00ee80 comp rx=0 tx=0).stop 2024-04-24T09:19:19.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.582+0000 7fcb60a00640 1 -- 172.21.15.90:0/565880364 shutdown_connections 2024-04-24T09:19:19.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.582+0000 7fcb60a00640 1 --2- 172.21.15.90:0/565880364 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fcb400a3cc0 0x7fcb400a6180 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:19.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.582+0000 7fcb60a00640 1 --2- 172.21.15.90:0/565880364 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb5c10a210 0x7fcb5c1a5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:19.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.582+0000 7fcb60a00640 1 --2- 172.21.15.90:0/565880364 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb5c109850 0x7fcb5c19ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:19.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.582+0000 7fcb60a00640 1 --2- 172.21.15.90:0/565880364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcb5c108660 0x7fcb5c19e8d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:19.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.582+0000 7fcb60a00640 1 -- 172.21.15.90:0/565880364 >> 172.21.15.90:0/565880364 conn(0x7fcb5c077620 msgr2=0x7fcb5c10cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:19.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.582+0000 7fcb60a00640 1 -- 172.21.15.90:0/565880364 shutdown_connections 2024-04-24T09:19:19.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.582+0000 7fcb60a00640 1 -- 172.21.15.90:0/565880364 wait complete. 2024-04-24T09:19:19.601 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2024-04-24T09:19:19.519971+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":639745,"num_objects":94,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":155,"num_read_kb":168,"num_write":388,"num_write_kb":3043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":61,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":316,"ondisk_log_size":316,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368772,"kb_used_data":11292,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124573052,"statfs":{"total":1151940427776,"available":1151562805248,"internally_reserved":0,"allocated":11563008,"data_stored":7131504,"data_compressed":109099,"data_compressed_allocated":139264,"data_compressed_original":1777664,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":49377,"num_objects":92,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":0,"num_read":29,"num_read_kb":59,"num_write":207,"num_write_kb":105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":61,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.701890"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.837342+0000","last_change":"2024-04-24T09:19:09.860043+0000","last_active":"2024-04-24T09:19:10.837342+0000","last_peered":"2024-04-24T09:19:10.837342+0000","last_clean":"2024-04-24T09:19:10.837342+0000","last_became_active":"2024-04-24T09:19:09.859766+0000","last_became_peered":"2024-04-24T09:19:09.859766+0000","last_unstale":"2024-04-24T09:19:10.837342+0000","last_undegraded":"2024-04-24T09:19:10.837342+0000","last_fullsized":"2024-04-24T09:19:10.837342+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:40:46.241239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:12.505761+0000","last_change":"2024-04-24T09:19:09.847293+0000","last_active":"2024-04-24T09:19:12.505761+0000","last_peered":"2024-04-24T09:19:12.505761+0000","last_clean":"2024-04-24T09:19:12.505761+0000","last_became_active":"2024-04-24T09:19:09.846955+0000","last_became_peered":"2024-04-24T09:19:09.846955+0000","last_unstale":"2024-04-24T09:19:12.505761+0000","last_undegraded":"2024-04-24T09:19:12.505761+0000","last_fullsized":"2024-04-24T09:19:12.505761+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:38:10.062654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:12.505624+0000","last_change":"2024-04-24T09:19:09.858954+0000","last_active":"2024-04-24T09:19:12.505624+0000","last_peered":"2024-04-24T09:19:12.505624+0000","last_clean":"2024-04-24T09:19:12.505624+0000","last_became_active":"2024-04-24T09:19:09.855915+0000","last_became_peered":"2024-04-24T09:19:09.855915+0000","last_unstale":"2024-04-24T09:19:12.505624+0000","last_undegraded":"2024-04-24T09:19:12.505624+0000","last_fullsized":"2024-04-24T09:19:12.505624+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:12:44.214700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:11.347658+0000","last_change":"2024-04-24T09:19:09.848582+0000","last_active":"2024-04-24T09:19:11.347658+0000","last_peered":"2024-04-24T09:19:11.347658+0000","last_clean":"2024-04-24T09:19:11.347658+0000","last_became_active":"2024-04-24T09:19:09.848163+0000","last_became_peered":"2024-04-24T09:19:09.848163+0000","last_unstale":"2024-04-24T09:19:11.347658+0000","last_undegraded":"2024-04-24T09:19:11.347658+0000","last_fullsized":"2024-04-24T09:19:11.347658+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:33:46.106081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:11.950548+0000","last_change":"2024-04-24T09:19:09.864608+0000","last_active":"2024-04-24T09:19:11.950548+0000","last_peered":"2024-04-24T09:19:11.950548+0000","last_clean":"2024-04-24T09:19:11.950548+0000","last_became_active":"2024-04-24T09:19:09.864271+0000","last_became_peered":"2024-04-24T09:19:09.864271+0000","last_unstale":"2024-04-24T09:19:11.950548+0000","last_undegraded":"2024-04-24T09:19:11.950548+0000","last_fullsized":"2024-04-24T09:19:11.950548+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:08:57.535882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:12.501605+0000","last_change":"2024-04-24T09:19:09.838589+0000","last_active":"2024-04-24T09:19:12.501605+0000","last_peered":"2024-04-24T09:19:12.501605+0000","last_clean":"2024-04-24T09:19:12.501605+0000","last_became_active":"2024-04-24T09:19:09.837628+0000","last_became_peered":"2024-04-24T09:19:09.837628+0000","last_unstale":"2024-04-24T09:19:12.501605+0000","last_undegraded":"2024-04-24T09:19:12.501605+0000","last_fullsized":"2024-04-24T09:19:12.501605+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:29:56.967777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.821798+0000","last_change":"2024-04-24T09:19:09.846296+0000","last_active":"2024-04-24T09:19:10.821798+0000","last_peered":"2024-04-24T09:19:10.821798+0000","last_clean":"2024-04-24T09:19:10.821798+0000","last_became_active":"2024-04-24T09:19:09.846010+0000","last_became_peered":"2024-04-24T09:19:09.846010+0000","last_unstale":"2024-04-24T09:19:10.821798+0000","last_undegraded":"2024-04-24T09:19:10.821798+0000","last_fullsized":"2024-04-24T09:19:10.821798+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:29:38.507747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.822766+0000","last_change":"2024-04-24T09:19:09.866821+0000","last_active":"2024-04-24T09:19:10.822766+0000","last_peered":"2024-04-24T09:19:10.822766+0000","last_clean":"2024-04-24T09:19:10.822766+0000","last_became_active":"2024-04-24T09:19:09.861793+0000","last_became_peered":"2024-04-24T09:19:09.861793+0000","last_unstale":"2024-04-24T09:19:10.822766+0000","last_undegraded":"2024-04-24T09:19:10.822766+0000","last_fullsized":"2024-04-24T09:19:10.822766+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:25:25.095642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:12.500884+0000","last_change":"2024-04-24T09:19:09.846433+0000","last_active":"2024-04-24T09:19:12.500884+0000","last_peered":"2024-04-24T09:19:12.500884+0000","last_clean":"2024-04-24T09:19:12.500884+0000","last_became_active":"2024-04-24T09:19:09.846044+0000","last_became_peered":"2024-04-24T09:19:09.846044+0000","last_unstale":"2024-04-24T09:19:12.500884+0000","last_undegraded":"2024-04-24T09:19:12.500884+0000","last_fullsized":"2024-04-24T09:19:12.500884+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:02:41.566626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:12.502075+0000","last_change":"2024-04-24T09:19:09.836662+0000","last_active":"2024-04-24T09:19:12.502075+0000","last_peered":"2024-04-24T09:19:12.502075+0000","last_clean":"2024-04-24T09:19:12.502075+0000","last_became_active":"2024-04-24T09:19:09.836283+0000","last_became_peered":"2024-04-24T09:19:09.836283+0000","last_unstale":"2024-04-24T09:19:12.502075+0000","last_undegraded":"2024-04-24T09:19:12.502075+0000","last_fullsized":"2024-04-24T09:19:12.502075+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:16:16.503469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.822274+0000","last_change":"2024-04-24T09:19:09.844369+0000","last_active":"2024-04-24T09:19:10.822274+0000","last_peered":"2024-04-24T09:19:10.822274+0000","last_clean":"2024-04-24T09:19:10.822274+0000","last_became_active":"2024-04-24T09:19:09.844044+0000","last_became_peered":"2024-04-24T09:19:09.844044+0000","last_unstale":"2024-04-24T09:19:10.822274+0000","last_undegraded":"2024-04-24T09:19:10.822274+0000","last_fullsized":"2024-04-24T09:19:10.822274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:51:45.861678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:11.347581+0000","last_change":"2024-04-24T09:19:09.846589+0000","last_active":"2024-04-24T09:19:11.347581+0000","last_peered":"2024-04-24T09:19:11.347581+0000","last_clean":"2024-04-24T09:19:11.347581+0000","last_became_active":"2024-04-24T09:19:09.844074+0000","last_became_peered":"2024-04-24T09:19:09.844074+0000","last_unstale":"2024-04-24T09:19:11.347581+0000","last_undegraded":"2024-04-24T09:19:11.347581+0000","last_fullsized":"2024-04-24T09:19:11.347581+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:47:12.002280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.828968+0000","last_change":"2024-04-24T09:19:09.847860+0000","last_active":"2024-04-24T09:19:10.828968+0000","last_peered":"2024-04-24T09:19:10.828968+0000","last_clean":"2024-04-24T09:19:10.828968+0000","last_became_active":"2024-04-24T09:19:09.847477+0000","last_became_peered":"2024-04-24T09:19:09.847477+0000","last_unstale":"2024-04-24T09:19:10.828968+0000","last_undegraded":"2024-04-24T09:19:10.828968+0000","last_fullsized":"2024-04-24T09:19:10.828968+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:39:44.655033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:12.505404+0000","last_change":"2024-04-24T09:19:09.854561+0000","last_active":"2024-04-24T09:19:12.505404+0000","last_peered":"2024-04-24T09:19:12.505404+0000","last_clean":"2024-04-24T09:19:12.505404+0000","last_became_active":"2024-04-24T09:19:09.849681+0000","last_became_peered":"2024-04-24T09:19:09.849681+0000","last_unstale":"2024-04-24T09:19:12.505404+0000","last_undegraded":"2024-04-24T09:19:12.505404+0000","last_fullsized":"2024-04-24T09:19:12.505404+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:46:36.572491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:12.505341+0000","last_change":"2024-04-24T09:19:09.859518+0000","last_active":"2024-04-24T09:19:12.505341+0000","last_peered":"2024-04-24T09:19:12.505341+0000","last_clean":"2024-04-24T09:19:12.505341+0000","last_became_active":"2024-04-24T09:19:09.859211+0000","last_became_peered":"2024-04-24T09:19:09.859211+0000","last_unstale":"2024-04-24T09:19:12.505341+0000","last_undegraded":"2024-04-24T09:19:12.505341+0000","last_fullsized":"2024-04-24T09:19:12.505341+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:20:04.273210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:12.500374+0000","last_change":"2024-04-24T09:19:09.837995+0000","last_active":"2024-04-24T09:19:12.500374+0000","last_peered":"2024-04-24T09:19:12.500374+0000","last_clean":"2024-04-24T09:19:12.500374+0000","last_became_active":"2024-04-24T09:19:09.837625+0000","last_became_peered":"2024-04-24T09:19:09.837625+0000","last_unstale":"2024-04-24T09:19:12.500374+0000","last_undegraded":"2024-04-24T09:19:12.500374+0000","last_fullsized":"2024-04-24T09:19:12.500374+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:14:30.602137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.837461+0000","last_change":"2024-04-24T09:19:09.855596+0000","last_active":"2024-04-24T09:19:10.837461+0000","last_peered":"2024-04-24T09:19:10.837461+0000","last_clean":"2024-04-24T09:19:10.837461+0000","last_became_active":"2024-04-24T09:19:09.855250+0000","last_became_peered":"2024-04-24T09:19:09.855250+0000","last_unstale":"2024-04-24T09:19:10.837461+0000","last_undegraded":"2024-04-24T09:19:10.837461+0000","last_fullsized":"2024-04-24T09:19:10.837461+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:48:58.862920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:11.116162+0000","last_change":"2024-04-24T09:19:09.844898+0000","last_active":"2024-04-24T09:19:11.116162+0000","last_peered":"2024-04-24T09:19:11.116162+0000","last_clean":"2024-04-24T09:19:11.116162+0000","last_became_active":"2024-04-24T09:19:09.844566+0000","last_became_peered":"2024-04-24T09:19:09.844566+0000","last_unstale":"2024-04-24T09:19:11.116162+0000","last_undegraded":"2024-04-24T09:19:11.116162+0000","last_fullsized":"2024-04-24T09:19:11.116162+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:15:34.145221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:11.235234+0000","last_change":"2024-04-24T09:19:09.856299+0000","last_active":"2024-04-24T09:19:11.235234+0000","last_peered":"2024-04-24T09:19:11.235234+0000","last_clean":"2024-04-24T09:19:11.235234+0000","last_became_active":"2024-04-24T09:19:09.855963+0000","last_became_peered":"2024-04-24T09:19:09.855963+0000","last_unstale":"2024-04-24T09:19:11.235234+0000","last_undegraded":"2024-04-24T09:19:11.235234+0000","last_fullsized":"2024-04-24T09:19:11.235234+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:10:27.372894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:16.267602+0000","last_change":"2024-04-24T09:19:08.973018+0000","last_active":"2024-04-24T09:19:16.267602+0000","last_peered":"2024-04-24T09:19:16.267602+0000","last_clean":"2024-04-24T09:19:16.267602+0000","last_became_active":"2024-04-24T09:19:08.972637+0000","last_became_peered":"2024-04-24T09:19:08.972637+0000","last_unstale":"2024-04-24T09:19:16.267602+0000","last_undegraded":"2024-04-24T09:19:16.267602+0000","last_fullsized":"2024-04-24T09:19:16.267602+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:11:10.890872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.823426+0000","last_change":"2024-04-24T09:19:09.862806+0000","last_active":"2024-04-24T09:19:10.823426+0000","last_peered":"2024-04-24T09:19:10.823426+0000","last_clean":"2024-04-24T09:19:10.823426+0000","last_became_active":"2024-04-24T09:19:09.862499+0000","last_became_peered":"2024-04-24T09:19:09.862499+0000","last_unstale":"2024-04-24T09:19:10.823426+0000","last_undegraded":"2024-04-24T09:19:10.823426+0000","last_fullsized":"2024-04-24T09:19:10.823426+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:13:19.494260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:12.103116+0000","last_change":"2024-04-24T09:19:08.973130+0000","last_active":"2024-04-24T09:19:12.103116+0000","last_peered":"2024-04-24T09:19:12.103116+0000","last_clean":"2024-04-24T09:19:12.103116+0000","last_became_active":"2024-04-24T09:19:08.972797+0000","last_became_peered":"2024-04-24T09:19:08.972797+0000","last_unstale":"2024-04-24T09:19:12.103116+0000","last_undegraded":"2024-04-24T09:19:12.103116+0000","last_fullsized":"2024-04-24T09:19:12.103116+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:21:49.236525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:12.505889+0000","last_change":"2024-04-24T09:19:09.860809+0000","last_active":"2024-04-24T09:19:12.505889+0000","last_peered":"2024-04-24T09:19:12.505889+0000","last_clean":"2024-04-24T09:19:12.505889+0000","last_became_active":"2024-04-24T09:19:09.860480+0000","last_became_peered":"2024-04-24T09:19:09.860480+0000","last_unstale":"2024-04-24T09:19:12.505889+0000","last_undegraded":"2024-04-24T09:19:12.505889+0000","last_fullsized":"2024-04-24T09:19:12.505889+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:39:21.114116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:16.270214+0000","last_change":"2024-04-24T09:19:08.950145+0000","last_active":"2024-04-24T09:19:16.270214+0000","last_peered":"2024-04-24T09:19:16.270214+0000","last_clean":"2024-04-24T09:19:16.270214+0000","last_became_active":"2024-04-24T09:19:08.949808+0000","last_became_peered":"2024-04-24T09:19:08.949808+0000","last_unstale":"2024-04-24T09:19:16.270214+0000","last_undegraded":"2024-04-24T09:19:16.270214+0000","last_fullsized":"2024-04-24T09:19:16.270214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:15:36.962348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:12.500706+0000","last_change":"2024-04-24T09:19:09.850123+0000","last_active":"2024-04-24T09:19:12.500706+0000","last_peered":"2024-04-24T09:19:12.500706+0000","last_clean":"2024-04-24T09:19:12.500706+0000","last_became_active":"2024-04-24T09:19:09.849770+0000","last_became_peered":"2024-04-24T09:19:09.849770+0000","last_unstale":"2024-04-24T09:19:12.500706+0000","last_undegraded":"2024-04-24T09:19:12.500706+0000","last_fullsized":"2024-04-24T09:19:12.500706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:10:00.374950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:16.269407+0000","last_change":"2024-04-24T09:19:08.952190+0000","last_active":"2024-04-24T09:19:16.269407+0000","last_peered":"2024-04-24T09:19:16.269407+0000","last_clean":"2024-04-24T09:19:16.269407+0000","last_became_active":"2024-04-24T09:19:08.951848+0000","last_became_peered":"2024-04-24T09:19:08.951848+0000","last_unstale":"2024-04-24T09:19:16.269407+0000","last_undegraded":"2024-04-24T09:19:16.269407+0000","last_fullsized":"2024-04-24T09:19:16.269407+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:02:54.892658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:11.404960+0000","last_change":"2024-04-24T09:19:09.845083+0000","last_active":"2024-04-24T09:19:11.404960+0000","last_peered":"2024-04-24T09:19:11.404960+0000","last_clean":"2024-04-24T09:19:11.404960+0000","last_became_active":"2024-04-24T09:19:09.844516+0000","last_became_peered":"2024-04-24T09:19:09.844516+0000","last_unstale":"2024-04-24T09:19:11.404960+0000","last_undegraded":"2024-04-24T09:19:11.404960+0000","last_fullsized":"2024-04-24T09:19:11.404960+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:10:41.995167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:13.273666+0000","last_change":"2024-04-24T09:19:08.950498+0000","last_active":"2024-04-24T09:19:13.273666+0000","last_peered":"2024-04-24T09:19:13.273666+0000","last_clean":"2024-04-24T09:19:13.273666+0000","last_became_active":"2024-04-24T09:19:08.950189+0000","last_became_peered":"2024-04-24T09:19:08.950189+0000","last_unstale":"2024-04-24T09:19:13.273666+0000","last_undegraded":"2024-04-24T09:19:13.273666+0000","last_fullsized":"2024-04-24T09:19:13.273666+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:40:41.342349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:11.950403+0000","last_change":"2024-04-24T09:19:09.847388+0000","last_active":"2024-04-24T09:19:11.950403+0000","last_peered":"2024-04-24T09:19:11.950403+0000","last_clean":"2024-04-24T09:19:11.950403+0000","last_became_active":"2024-04-24T09:19:09.846999+0000","last_became_peered":"2024-04-24T09:19:09.846999+0000","last_unstale":"2024-04-24T09:19:11.950403+0000","last_undegraded":"2024-04-24T09:19:11.950403+0000","last_fullsized":"2024-04-24T09:19:11.950403+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:49:09.814212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:15.305362+0000","last_change":"2024-04-24T09:19:08.965244+0000","last_active":"2024-04-24T09:19:15.305362+0000","last_peered":"2024-04-24T09:19:15.305362+0000","last_clean":"2024-04-24T09:19:15.305362+0000","last_became_active":"2024-04-24T09:19:08.964899+0000","last_became_peered":"2024-04-24T09:19:08.964899+0000","last_unstale":"2024-04-24T09:19:15.305362+0000","last_undegraded":"2024-04-24T09:19:15.305362+0000","last_fullsized":"2024-04-24T09:19:15.305362+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:53:28.035567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:11.404721+0000","last_change":"2024-04-24T09:19:09.864902+0000","last_active":"2024-04-24T09:19:11.404721+0000","last_peered":"2024-04-24T09:19:11.404721+0000","last_clean":"2024-04-24T09:19:11.404721+0000","last_became_active":"2024-04-24T09:19:09.864500+0000","last_became_peered":"2024-04-24T09:19:09.864500+0000","last_unstale":"2024-04-24T09:19:11.404721+0000","last_undegraded":"2024-04-24T09:19:11.404721+0000","last_fullsized":"2024-04-24T09:19:11.404721+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:37:31.810050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:14.303865+0000","last_change":"2024-04-24T09:19:08.977987+0000","last_active":"2024-04-24T09:19:14.303865+0000","last_peered":"2024-04-24T09:19:14.303865+0000","last_clean":"2024-04-24T09:19:14.303865+0000","last_became_active":"2024-04-24T09:19:08.977666+0000","last_became_peered":"2024-04-24T09:19:08.977666+0000","last_unstale":"2024-04-24T09:19:14.303865+0000","last_undegraded":"2024-04-24T09:19:14.303865+0000","last_fullsized":"2024-04-24T09:19:14.303865+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:21:21.328463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.822595+0000","last_change":"2024-04-24T09:19:09.863990+0000","last_active":"2024-04-24T09:19:10.822595+0000","last_peered":"2024-04-24T09:19:10.822595+0000","last_clean":"2024-04-24T09:19:10.822595+0000","last_became_active":"2024-04-24T09:19:09.854938+0000","last_became_peered":"2024-04-24T09:19:09.854938+0000","last_unstale":"2024-04-24T09:19:10.822595+0000","last_undegraded":"2024-04-24T09:19:10.822595+0000","last_fullsized":"2024-04-24T09:19:10.822595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:41:55.054373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:13.273611+0000","last_change":"2024-04-24T09:19:08.949938+0000","last_active":"2024-04-24T09:19:13.273611+0000","last_peered":"2024-04-24T09:19:13.273611+0000","last_clean":"2024-04-24T09:19:13.273611+0000","last_became_active":"2024-04-24T09:19:08.949561+0000","last_became_peered":"2024-04-24T09:19:08.949561+0000","last_unstale":"2024-04-24T09:19:13.273611+0000","last_undegraded":"2024-04-24T09:19:13.273611+0000","last_fullsized":"2024-04-24T09:19:13.273611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:55:50.488129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:11.115600+0000","last_change":"2024-04-24T09:19:09.841782+0000","last_active":"2024-04-24T09:19:11.115600+0000","last_peered":"2024-04-24T09:19:11.115600+0000","last_clean":"2024-04-24T09:19:11.115600+0000","last_became_active":"2024-04-24T09:19:09.841452+0000","last_became_peered":"2024-04-24T09:19:09.841452+0000","last_unstale":"2024-04-24T09:19:11.115600+0000","last_undegraded":"2024-04-24T09:19:11.115600+0000","last_fullsized":"2024-04-24T09:19:11.115600+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:58:19.618999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:18.254031+0000","last_change":"2024-04-24T09:19:08.973169+0000","last_active":"2024-04-24T09:19:18.254031+0000","last_peered":"2024-04-24T09:19:18.254031+0000","last_clean":"2024-04-24T09:19:18.254031+0000","last_became_active":"2024-04-24T09:19:08.972835+0000","last_became_peered":"2024-04-24T09:19:08.972835+0000","last_unstale":"2024-04-24T09:19:18.254031+0000","last_undegraded":"2024-04-24T09:19:18.254031+0000","last_fullsized":"2024-04-24T09:19:18.254031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:34:04.156280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:12.500895+0000","last_change":"2024-04-24T09:19:09.854957+0000","last_active":"2024-04-24T09:19:12.500895+0000","last_peered":"2024-04-24T09:19:12.500895+0000","last_clean":"2024-04-24T09:19:12.500895+0000","last_became_active":"2024-04-24T09:19:09.854574+0000","last_became_peered":"2024-04-24T09:19:09.854574+0000","last_unstale":"2024-04-24T09:19:12.500895+0000","last_undegraded":"2024-04-24T09:19:12.500895+0000","last_fullsized":"2024-04-24T09:19:12.500895+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:07:28.296335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:16.267920+0000","last_change":"2024-04-24T09:19:08.953179+0000","last_active":"2024-04-24T09:19:16.267920+0000","last_peered":"2024-04-24T09:19:16.267920+0000","last_clean":"2024-04-24T09:19:16.267920+0000","last_became_active":"2024-04-24T09:19:08.952864+0000","last_became_peered":"2024-04-24T09:19:08.952864+0000","last_unstale":"2024-04-24T09:19:16.267920+0000","last_undegraded":"2024-04-24T09:19:16.267920+0000","last_fullsized":"2024-04-24T09:19:16.267920+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:22:31.470593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:12.505883+0000","last_change":"2024-04-24T09:19:09.850855+0000","last_active":"2024-04-24T09:19:12.505883+0000","last_peered":"2024-04-24T09:19:12.505883+0000","last_clean":"2024-04-24T09:19:12.505883+0000","last_became_active":"2024-04-24T09:19:09.850535+0000","last_became_peered":"2024-04-24T09:19:09.850535+0000","last_unstale":"2024-04-24T09:19:12.505883+0000","last_undegraded":"2024-04-24T09:19:12.505883+0000","last_fullsized":"2024-04-24T09:19:12.505883+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:27:23.294935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:13.278571+0000","last_change":"2024-04-24T09:19:08.967882+0000","last_active":"2024-04-24T09:19:13.278571+0000","last_peered":"2024-04-24T09:19:13.278571+0000","last_clean":"2024-04-24T09:19:13.278571+0000","last_became_active":"2024-04-24T09:19:08.967625+0000","last_became_peered":"2024-04-24T09:19:08.967625+0000","last_unstale":"2024-04-24T09:19:13.278571+0000","last_undegraded":"2024-04-24T09:19:13.278571+0000","last_fullsized":"2024-04-24T09:19:13.278571+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:39:57.802717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:12.500765+0000","last_change":"2024-04-24T09:19:09.844721+0000","last_active":"2024-04-24T09:19:12.500765+0000","last_peered":"2024-04-24T09:19:12.500765+0000","last_clean":"2024-04-24T09:19:12.500765+0000","last_became_active":"2024-04-24T09:19:09.844365+0000","last_became_peered":"2024-04-24T09:19:09.844365+0000","last_unstale":"2024-04-24T09:19:12.500765+0000","last_undegraded":"2024-04-24T09:19:12.500765+0000","last_fullsized":"2024-04-24T09:19:12.500765+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:11:11.215015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'22","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:18.261921+0000","last_change":"2024-04-24T09:19:08.955409+0000","last_active":"2024-04-24T09:19:18.261921+0000","last_peered":"2024-04-24T09:19:18.261921+0000","last_clean":"2024-04-24T09:19:18.261921+0000","last_became_active":"2024-04-24T09:19:08.955065+0000","last_became_peered":"2024-04-24T09:19:08.955065+0000","last_unstale":"2024-04-24T09:19:18.261921+0000","last_undegraded":"2024-04-24T09:19:18.261921+0000","last_fullsized":"2024-04-24T09:19:18.261921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:28:34.411386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43377,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":10,"num_read_kb":42,"num_write":24,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:11.950698+0000","last_change":"2024-04-24T09:19:09.846521+0000","last_active":"2024-04-24T09:19:11.950698+0000","last_peered":"2024-04-24T09:19:11.950698+0000","last_clean":"2024-04-24T09:19:11.950698+0000","last_became_active":"2024-04-24T09:19:09.846162+0000","last_became_peered":"2024-04-24T09:19:09.846162+0000","last_unstale":"2024-04-24T09:19:11.950698+0000","last_undegraded":"2024-04-24T09:19:11.950698+0000","last_fullsized":"2024-04-24T09:19:11.950698+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:05:50.383432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:14.303266+0000","last_change":"2024-04-24T09:19:08.963754+0000","last_active":"2024-04-24T09:19:14.303266+0000","last_peered":"2024-04-24T09:19:14.303266+0000","last_clean":"2024-04-24T09:19:14.303266+0000","last_became_active":"2024-04-24T09:19:08.963379+0000","last_became_peered":"2024-04-24T09:19:08.963379+0000","last_unstale":"2024-04-24T09:19:14.303266+0000","last_undegraded":"2024-04-24T09:19:14.303266+0000","last_fullsized":"2024-04-24T09:19:14.303266+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:25:01.538860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.828763+0000","last_change":"2024-04-24T09:19:09.863649+0000","last_active":"2024-04-24T09:19:10.828763+0000","last_peered":"2024-04-24T09:19:10.828763+0000","last_clean":"2024-04-24T09:19:10.828763+0000","last_became_active":"2024-04-24T09:19:09.863295+0000","last_became_peered":"2024-04-24T09:19:09.863295+0000","last_unstale":"2024-04-24T09:19:10.828763+0000","last_undegraded":"2024-04-24T09:19:10.828763+0000","last_fullsized":"2024-04-24T09:19:10.828763+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:06:56.404909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:13.276677+0000","last_change":"2024-04-24T09:19:08.961409+0000","last_active":"2024-04-24T09:19:13.276677+0000","last_peered":"2024-04-24T09:19:13.276677+0000","last_clean":"2024-04-24T09:19:13.276677+0000","last_became_active":"2024-04-24T09:19:08.961085+0000","last_became_peered":"2024-04-24T09:19:08.961085+0000","last_unstale":"2024-04-24T09:19:13.276677+0000","last_undegraded":"2024-04-24T09:19:13.276677+0000","last_fullsized":"2024-04-24T09:19:13.276677+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:10:32.816204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.822226+0000","last_change":"2024-04-24T09:19:09.845992+0000","last_active":"2024-04-24T09:19:10.822226+0000","last_peered":"2024-04-24T09:19:10.822226+0000","last_clean":"2024-04-24T09:19:10.822226+0000","last_became_active":"2024-04-24T09:19:09.845673+0000","last_became_peered":"2024-04-24T09:19:09.845673+0000","last_unstale":"2024-04-24T09:19:10.822226+0000","last_undegraded":"2024-04-24T09:19:10.822226+0000","last_fullsized":"2024-04-24T09:19:10.822226+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:42:53.136476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:15.303603+0000","last_change":"2024-04-24T09:19:08.954872+0000","last_active":"2024-04-24T09:19:15.303603+0000","last_peered":"2024-04-24T09:19:15.303603+0000","last_clean":"2024-04-24T09:19:15.303603+0000","last_became_active":"2024-04-24T09:19:08.954540+0000","last_became_peered":"2024-04-24T09:19:08.954540+0000","last_unstale":"2024-04-24T09:19:15.303603+0000","last_undegraded":"2024-04-24T09:19:15.303603+0000","last_fullsized":"2024-04-24T09:19:15.303603+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:32:15.637337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.821813+0000","last_change":"2024-04-24T09:19:09.841735+0000","last_active":"2024-04-24T09:19:10.821813+0000","last_peered":"2024-04-24T09:19:10.821813+0000","last_clean":"2024-04-24T09:19:10.821813+0000","last_became_active":"2024-04-24T09:19:09.841422+0000","last_became_peered":"2024-04-24T09:19:09.841422+0000","last_unstale":"2024-04-24T09:19:10.821813+0000","last_undegraded":"2024-04-24T09:19:10.821813+0000","last_fullsized":"2024-04-24T09:19:10.821813+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:37:46.128344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:16.248975+0000","last_change":"2024-04-24T09:19:08.971108+0000","last_active":"2024-04-24T09:19:16.248975+0000","last_peered":"2024-04-24T09:19:16.248975+0000","last_clean":"2024-04-24T09:19:16.248975+0000","last_became_active":"2024-04-24T09:19:08.970808+0000","last_became_peered":"2024-04-24T09:19:08.970808+0000","last_unstale":"2024-04-24T09:19:16.248975+0000","last_undegraded":"2024-04-24T09:19:16.248975+0000","last_fullsized":"2024-04-24T09:19:16.248975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:37:26.311351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:11.404455+0000","last_change":"2024-04-24T09:19:09.848041+0000","last_active":"2024-04-24T09:19:11.404455+0000","last_peered":"2024-04-24T09:19:11.404455+0000","last_clean":"2024-04-24T09:19:11.404455+0000","last_became_active":"2024-04-24T09:19:09.847090+0000","last_became_peered":"2024-04-24T09:19:09.847090+0000","last_unstale":"2024-04-24T09:19:11.404455+0000","last_undegraded":"2024-04-24T09:19:11.404455+0000","last_fullsized":"2024-04-24T09:19:11.404455+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:45:13.652402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:15.293217+0000","last_change":"2024-04-24T09:19:08.952061+0000","last_active":"2024-04-24T09:19:15.293217+0000","last_peered":"2024-04-24T09:19:15.293217+0000","last_clean":"2024-04-24T09:19:15.293217+0000","last_became_active":"2024-04-24T09:19:08.950549+0000","last_became_peered":"2024-04-24T09:19:08.950549+0000","last_unstale":"2024-04-24T09:19:15.293217+0000","last_undegraded":"2024-04-24T09:19:15.293217+0000","last_fullsized":"2024-04-24T09:19:15.293217+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:41:40.949650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.837232+0000","last_change":"2024-04-24T09:19:09.848619+0000","last_active":"2024-04-24T09:19:10.837232+0000","last_peered":"2024-04-24T09:19:10.837232+0000","last_clean":"2024-04-24T09:19:10.837232+0000","last_became_active":"2024-04-24T09:19:09.848246+0000","last_became_peered":"2024-04-24T09:19:09.848246+0000","last_unstale":"2024-04-24T09:19:10.837232+0000","last_undegraded":"2024-04-24T09:19:10.837232+0000","last_fullsized":"2024-04-24T09:19:10.837232+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:30:22.925368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:11.577955+0000","last_change":"2024-04-24T09:19:08.977448+0000","last_active":"2024-04-24T09:19:11.577955+0000","last_peered":"2024-04-24T09:19:11.577955+0000","last_clean":"2024-04-24T09:19:11.577955+0000","last_became_active":"2024-04-24T09:19:08.976525+0000","last_became_peered":"2024-04-24T09:19:08.976525+0000","last_unstale":"2024-04-24T09:19:11.577955+0000","last_undegraded":"2024-04-24T09:19:11.577955+0000","last_fullsized":"2024-04-24T09:19:11.577955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:06:59.151486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:12.501924+0000","last_change":"2024-04-24T09:19:09.856260+0000","last_active":"2024-04-24T09:19:12.501924+0000","last_peered":"2024-04-24T09:19:12.501924+0000","last_clean":"2024-04-24T09:19:12.501924+0000","last_became_active":"2024-04-24T09:19:09.855895+0000","last_became_peered":"2024-04-24T09:19:09.855895+0000","last_unstale":"2024-04-24T09:19:12.501924+0000","last_undegraded":"2024-04-24T09:19:12.501924+0000","last_fullsized":"2024-04-24T09:19:12.501924+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:44:22.804210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:17.269583+0000","last_change":"2024-04-24T09:19:08.962268+0000","last_active":"2024-04-24T09:19:17.269583+0000","last_peered":"2024-04-24T09:19:17.269583+0000","last_clean":"2024-04-24T09:19:17.269583+0000","last_became_active":"2024-04-24T09:19:08.961930+0000","last_became_peered":"2024-04-24T09:19:08.961930+0000","last_unstale":"2024-04-24T09:19:17.269583+0000","last_undegraded":"2024-04-24T09:19:17.269583+0000","last_fullsized":"2024-04-24T09:19:17.269583+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:51:28.357674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:12.501826+0000","last_change":"2024-04-24T09:19:09.850782+0000","last_active":"2024-04-24T09:19:12.501826+0000","last_peered":"2024-04-24T09:19:12.501826+0000","last_clean":"2024-04-24T09:19:12.501826+0000","last_became_active":"2024-04-24T09:19:09.850458+0000","last_became_peered":"2024-04-24T09:19:09.850458+0000","last_unstale":"2024-04-24T09:19:12.501826+0000","last_undegraded":"2024-04-24T09:19:12.501826+0000","last_fullsized":"2024-04-24T09:19:12.501826+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:02:19.464011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:11.404791+0000","last_change":"2024-04-24T09:19:09.841551+0000","last_active":"2024-04-24T09:19:11.404791+0000","last_peered":"2024-04-24T09:19:11.404791+0000","last_clean":"2024-04-24T09:19:11.404791+0000","last_became_active":"2024-04-24T09:19:09.841219+0000","last_became_peered":"2024-04-24T09:19:09.841219+0000","last_unstale":"2024-04-24T09:19:11.404791+0000","last_undegraded":"2024-04-24T09:19:11.404791+0000","last_fullsized":"2024-04-24T09:19:11.404791+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:19:03.978876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:16.252226+0000","last_change":"2024-04-24T09:19:08.972070+0000","last_active":"2024-04-24T09:19:16.252226+0000","last_peered":"2024-04-24T09:19:16.252226+0000","last_clean":"2024-04-24T09:19:16.252226+0000","last_became_active":"2024-04-24T09:19:08.971699+0000","last_became_peered":"2024-04-24T09:19:08.971699+0000","last_unstale":"2024-04-24T09:19:16.252226+0000","last_undegraded":"2024-04-24T09:19:16.252226+0000","last_fullsized":"2024-04-24T09:19:16.252226+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:38:13.312940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.823472+0000","last_change":"2024-04-24T09:19:09.855902+0000","last_active":"2024-04-24T09:19:10.823472+0000","last_peered":"2024-04-24T09:19:10.823472+0000","last_clean":"2024-04-24T09:19:10.823472+0000","last_became_active":"2024-04-24T09:19:09.855503+0000","last_became_peered":"2024-04-24T09:19:09.855503+0000","last_unstale":"2024-04-24T09:19:10.823472+0000","last_undegraded":"2024-04-24T09:19:10.823472+0000","last_fullsized":"2024-04-24T09:19:10.823472+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:29:41.621269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:15.304833+0000","last_change":"2024-04-24T09:19:08.973774+0000","last_active":"2024-04-24T09:19:15.304833+0000","last_peered":"2024-04-24T09:19:15.304833+0000","last_clean":"2024-04-24T09:19:15.304833+0000","last_became_active":"2024-04-24T09:19:08.973372+0000","last_became_peered":"2024-04-24T09:19:08.973372+0000","last_unstale":"2024-04-24T09:19:15.304833+0000","last_undegraded":"2024-04-24T09:19:15.304833+0000","last_fullsized":"2024-04-24T09:19:15.304833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:25:12.910023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":338,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.821688+0000","last_change":"2024-04-24T09:17:48.282773+0000","last_active":"2024-04-24T09:19:10.821688+0000","last_peered":"2024-04-24T09:19:10.821688+0000","last_clean":"2024-04-24T09:19:10.821688+0000","last_became_active":"2024-04-24T09:17:48.282128+0000","last_became_peered":"2024-04-24T09:17:48.282128+0000","last_unstale":"2024-04-24T09:19:10.821688+0000","last_undegraded":"2024-04-24T09:19:10.821688+0000","last_fullsized":"2024-04-24T09:19:10.821688+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:13:45.562295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:13:45.562295+0000","last_clean_scrub_stamp":"2024-04-24T09:13:45.562295+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:54:34.557051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.828597+0000","last_change":"2024-04-24T09:19:09.847771+0000","last_active":"2024-04-24T09:19:10.828597+0000","last_peered":"2024-04-24T09:19:10.828597+0000","last_clean":"2024-04-24T09:19:10.828597+0000","last_became_active":"2024-04-24T09:19:09.845873+0000","last_became_peered":"2024-04-24T09:19:09.845873+0000","last_unstale":"2024-04-24T09:19:10.828597+0000","last_undegraded":"2024-04-24T09:19:10.828597+0000","last_fullsized":"2024-04-24T09:19:10.828597+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:31:46.124082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:14.297484+0000","last_change":"2024-04-24T09:19:08.950415+0000","last_active":"2024-04-24T09:19:14.297484+0000","last_peered":"2024-04-24T09:19:14.297484+0000","last_clean":"2024-04-24T09:19:14.297484+0000","last_became_active":"2024-04-24T09:19:08.949532+0000","last_became_peered":"2024-04-24T09:19:08.949532+0000","last_unstale":"2024-04-24T09:19:14.297484+0000","last_undegraded":"2024-04-24T09:19:14.297484+0000","last_fullsized":"2024-04-24T09:19:14.297484+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:35:48.093817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:16.248975+0000","last_change":"2024-04-24T09:19:08.971475+0000","last_active":"2024-04-24T09:19:16.248975+0000","last_peered":"2024-04-24T09:19:16.248975+0000","last_clean":"2024-04-24T09:19:16.248975+0000","last_became_active":"2024-04-24T09:19:08.971188+0000","last_became_peered":"2024-04-24T09:19:08.971188+0000","last_unstale":"2024-04-24T09:19:16.248975+0000","last_undegraded":"2024-04-24T09:19:16.248975+0000","last_fullsized":"2024-04-24T09:19:16.248975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:50:06.630182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.829200+0000","last_change":"2024-04-24T09:19:09.857102+0000","last_active":"2024-04-24T09:19:10.829200+0000","last_peered":"2024-04-24T09:19:10.829200+0000","last_clean":"2024-04-24T09:19:10.829200+0000","last_became_active":"2024-04-24T09:19:09.856632+0000","last_became_peered":"2024-04-24T09:19:09.856632+0000","last_unstale":"2024-04-24T09:19:10.829200+0000","last_undegraded":"2024-04-24T09:19:10.829200+0000","last_fullsized":"2024-04-24T09:19:10.829200+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:04:03.300741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'7","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:15.307441+0000","last_change":"2024-04-24T09:19:08.973387+0000","last_active":"2024-04-24T09:19:15.307441+0000","last_peered":"2024-04-24T09:19:15.307441+0000","last_clean":"2024-04-24T09:19:15.307441+0000","last_became_active":"2024-04-24T09:19:08.973081+0000","last_became_peered":"2024-04-24T09:19:08.973081+0000","last_unstale":"2024-04-24T09:19:15.307441+0000","last_undegraded":"2024-04-24T09:19:15.307441+0000","last_fullsized":"2024-04-24T09:19:15.307441+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:00:36.843053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.823406+0000","last_change":"2024-04-24T09:19:09.863386+0000","last_active":"2024-04-24T09:19:10.823406+0000","last_peered":"2024-04-24T09:19:10.823406+0000","last_clean":"2024-04-24T09:19:10.823406+0000","last_became_active":"2024-04-24T09:19:09.856300+0000","last_became_peered":"2024-04-24T09:19:09.856300+0000","last_unstale":"2024-04-24T09:19:10.823406+0000","last_undegraded":"2024-04-24T09:19:10.823406+0000","last_fullsized":"2024-04-24T09:19:10.823406+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:26:37.603307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:13.275842+0000","last_change":"2024-04-24T09:19:08.961751+0000","last_active":"2024-04-24T09:19:13.275842+0000","last_peered":"2024-04-24T09:19:13.275842+0000","last_clean":"2024-04-24T09:19:13.275842+0000","last_became_active":"2024-04-24T09:19:08.961389+0000","last_became_peered":"2024-04-24T09:19:08.961389+0000","last_unstale":"2024-04-24T09:19:13.275842+0000","last_undegraded":"2024-04-24T09:19:13.275842+0000","last_fullsized":"2024-04-24T09:19:13.275842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:25:17.913335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:12.506020+0000","last_change":"2024-04-24T09:19:09.862140+0000","last_active":"2024-04-24T09:19:12.506020+0000","last_peered":"2024-04-24T09:19:12.506020+0000","last_clean":"2024-04-24T09:19:12.506020+0000","last_became_active":"2024-04-24T09:19:09.860776+0000","last_became_peered":"2024-04-24T09:19:09.860776+0000","last_unstale":"2024-04-24T09:19:12.506020+0000","last_undegraded":"2024-04-24T09:19:12.506020+0000","last_fullsized":"2024-04-24T09:19:12.506020+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:08:29.437422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:13.278494+0000","last_change":"2024-04-24T09:19:08.973503+0000","last_active":"2024-04-24T09:19:13.278494+0000","last_peered":"2024-04-24T09:19:13.278494+0000","last_clean":"2024-04-24T09:19:13.278494+0000","last_became_active":"2024-04-24T09:19:08.973153+0000","last_became_peered":"2024-04-24T09:19:08.973153+0000","last_unstale":"2024-04-24T09:19:13.278494+0000","last_undegraded":"2024-04-24T09:19:13.278494+0000","last_fullsized":"2024-04-24T09:19:13.278494+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:57:21.489696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:11.950299+0000","last_change":"2024-04-24T09:19:09.838175+0000","last_active":"2024-04-24T09:19:11.950299+0000","last_peered":"2024-04-24T09:19:11.950299+0000","last_clean":"2024-04-24T09:19:11.950299+0000","last_became_active":"2024-04-24T09:19:09.837781+0000","last_became_peered":"2024-04-24T09:19:09.837781+0000","last_unstale":"2024-04-24T09:19:11.950299+0000","last_undegraded":"2024-04-24T09:19:11.950299+0000","last_fullsized":"2024-04-24T09:19:11.950299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:48:03.967587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:16.275013+0000","last_change":"2024-04-24T09:19:08.976790+0000","last_active":"2024-04-24T09:19:16.275013+0000","last_peered":"2024-04-24T09:19:16.275013+0000","last_clean":"2024-04-24T09:19:16.275013+0000","last_became_active":"2024-04-24T09:19:08.975196+0000","last_became_peered":"2024-04-24T09:19:08.975196+0000","last_unstale":"2024-04-24T09:19:16.275013+0000","last_undegraded":"2024-04-24T09:19:16.275013+0000","last_fullsized":"2024-04-24T09:19:16.275013+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:54:59.415169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.836983+0000","last_change":"2024-04-24T09:19:09.850958+0000","last_active":"2024-04-24T09:19:10.836983+0000","last_peered":"2024-04-24T09:19:10.836983+0000","last_clean":"2024-04-24T09:19:10.836983+0000","last_became_active":"2024-04-24T09:19:09.850660+0000","last_became_peered":"2024-04-24T09:19:09.850660+0000","last_unstale":"2024-04-24T09:19:10.836983+0000","last_undegraded":"2024-04-24T09:19:10.836983+0000","last_fullsized":"2024-04-24T09:19:10.836983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:41:27.496169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:11.234903+0000","last_change":"2024-04-24T09:19:08.963743+0000","last_active":"2024-04-24T09:19:11.234903+0000","last_peered":"2024-04-24T09:19:11.234903+0000","last_clean":"2024-04-24T09:19:11.234903+0000","last_became_active":"2024-04-24T09:19:08.963378+0000","last_became_peered":"2024-04-24T09:19:08.963378+0000","last_unstale":"2024-04-24T09:19:11.234903+0000","last_undegraded":"2024-04-24T09:19:11.234903+0000","last_fullsized":"2024-04-24T09:19:11.234903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:29:54.256411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.823151+0000","last_change":"2024-04-24T09:19:09.863144+0000","last_active":"2024-04-24T09:19:10.823151+0000","last_peered":"2024-04-24T09:19:10.823151+0000","last_clean":"2024-04-24T09:19:10.823151+0000","last_became_active":"2024-04-24T09:19:09.862772+0000","last_became_peered":"2024-04-24T09:19:09.862772+0000","last_unstale":"2024-04-24T09:19:10.823151+0000","last_undegraded":"2024-04-24T09:19:10.823151+0000","last_fullsized":"2024-04-24T09:19:10.823151+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:05:42.926470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:13.273329+0000","last_change":"2024-04-24T09:19:08.953467+0000","last_active":"2024-04-24T09:19:13.273329+0000","last_peered":"2024-04-24T09:19:13.273329+0000","last_clean":"2024-04-24T09:19:13.273329+0000","last_became_active":"2024-04-24T09:19:08.953116+0000","last_became_peered":"2024-04-24T09:19:08.953116+0000","last_unstale":"2024-04-24T09:19:13.273329+0000","last_undegraded":"2024-04-24T09:19:13.273329+0000","last_fullsized":"2024-04-24T09:19:13.273329+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:07:35.834466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.821925+0000","last_change":"2024-04-24T09:19:09.852329+0000","last_active":"2024-04-24T09:19:10.821925+0000","last_peered":"2024-04-24T09:19:10.821925+0000","last_clean":"2024-04-24T09:19:10.821925+0000","last_became_active":"2024-04-24T09:19:09.851762+0000","last_became_peered":"2024-04-24T09:19:09.851762+0000","last_unstale":"2024-04-24T09:19:10.821925+0000","last_undegraded":"2024-04-24T09:19:10.821925+0000","last_fullsized":"2024-04-24T09:19:10.821925+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:56:59.234680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:14.302634+0000","last_change":"2024-04-24T09:19:08.966699+0000","last_active":"2024-04-24T09:19:14.302634+0000","last_peered":"2024-04-24T09:19:14.302634+0000","last_clean":"2024-04-24T09:19:14.302634+0000","last_became_active":"2024-04-24T09:19:08.966315+0000","last_became_peered":"2024-04-24T09:19:08.966315+0000","last_unstale":"2024-04-24T09:19:14.302634+0000","last_undegraded":"2024-04-24T09:19:14.302634+0000","last_fullsized":"2024-04-24T09:19:14.302634+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:38:16.278751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.822325+0000","last_change":"2024-04-24T09:19:09.840340+0000","last_active":"2024-04-24T09:19:10.822325+0000","last_peered":"2024-04-24T09:19:10.822325+0000","last_clean":"2024-04-24T09:19:10.822325+0000","last_became_active":"2024-04-24T09:19:09.839689+0000","last_became_peered":"2024-04-24T09:19:09.839689+0000","last_unstale":"2024-04-24T09:19:10.822325+0000","last_undegraded":"2024-04-24T09:19:10.822325+0000","last_fullsized":"2024-04-24T09:19:10.822325+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:36:06.174519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:11.404709+0000","last_change":"2024-04-24T09:19:08.959025+0000","last_active":"2024-04-24T09:19:11.404709+0000","last_peered":"2024-04-24T09:19:11.404709+0000","last_clean":"2024-04-24T09:19:11.404709+0000","last_became_active":"2024-04-24T09:19:08.958749+0000","last_became_peered":"2024-04-24T09:19:08.958749+0000","last_unstale":"2024-04-24T09:19:11.404709+0000","last_undegraded":"2024-04-24T09:19:11.404709+0000","last_fullsized":"2024-04-24T09:19:11.404709+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:59:41.334448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:13.591078+0000","last_change":"2024-04-24T09:19:09.856258+0000","last_active":"2024-04-24T09:19:13.591078+0000","last_peered":"2024-04-24T09:19:13.591078+0000","last_clean":"2024-04-24T09:19:13.591078+0000","last_became_active":"2024-04-24T09:19:09.855919+0000","last_became_peered":"2024-04-24T09:19:09.855919+0000","last_unstale":"2024-04-24T09:19:13.591078+0000","last_undegraded":"2024-04-24T09:19:13.591078+0000","last_fullsized":"2024-04-24T09:19:13.591078+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:06:09.628268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:16.275466+0000","last_change":"2024-04-24T09:19:08.965880+0000","last_active":"2024-04-24T09:19:16.275466+0000","last_peered":"2024-04-24T09:19:16.275466+0000","last_clean":"2024-04-24T09:19:16.275466+0000","last_became_active":"2024-04-24T09:19:08.965378+0000","last_became_peered":"2024-04-24T09:19:08.965378+0000","last_unstale":"2024-04-24T09:19:16.275466+0000","last_undegraded":"2024-04-24T09:19:16.275466+0000","last_fullsized":"2024-04-24T09:19:16.275466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:08:22.628556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.822292+0000","last_change":"2024-04-24T09:19:09.845151+0000","last_active":"2024-04-24T09:19:10.822292+0000","last_peered":"2024-04-24T09:19:10.822292+0000","last_clean":"2024-04-24T09:19:10.822292+0000","last_became_active":"2024-04-24T09:19:09.844791+0000","last_became_peered":"2024-04-24T09:19:09.844791+0000","last_unstale":"2024-04-24T09:19:10.822292+0000","last_undegraded":"2024-04-24T09:19:10.822292+0000","last_fullsized":"2024-04-24T09:19:10.822292+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:38:00.782591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.837150+0000","last_change":"2024-04-24T09:19:09.844449+0000","last_active":"2024-04-24T09:19:10.837150+0000","last_peered":"2024-04-24T09:19:10.837150+0000","last_clean":"2024-04-24T09:19:10.837150+0000","last_became_active":"2024-04-24T09:19:09.844122+0000","last_became_peered":"2024-04-24T09:19:09.844122+0000","last_unstale":"2024-04-24T09:19:10.837150+0000","last_undegraded":"2024-04-24T09:19:10.837150+0000","last_fullsized":"2024-04-24T09:19:10.837150+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:36:01.857309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:11.235061+0000","last_change":"2024-04-24T09:19:09.843151+0000","last_active":"2024-04-24T09:19:11.235061+0000","last_peered":"2024-04-24T09:19:11.235061+0000","last_clean":"2024-04-24T09:19:11.235061+0000","last_became_active":"2024-04-24T09:19:09.842519+0000","last_became_peered":"2024-04-24T09:19:09.842519+0000","last_unstale":"2024-04-24T09:19:11.235061+0000","last_undegraded":"2024-04-24T09:19:11.235061+0000","last_fullsized":"2024-04-24T09:19:11.235061+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:26:00.791618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:11.234928+0000","last_change":"2024-04-24T09:19:09.856291+0000","last_active":"2024-04-24T09:19:11.234928+0000","last_peered":"2024-04-24T09:19:11.234928+0000","last_clean":"2024-04-24T09:19:11.234928+0000","last_became_active":"2024-04-24T09:19:09.855932+0000","last_became_peered":"2024-04-24T09:19:09.855932+0000","last_unstale":"2024-04-24T09:19:11.234928+0000","last_undegraded":"2024-04-24T09:19:11.234928+0000","last_fullsized":"2024-04-24T09:19:11.234928+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:11:14.113321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:12.505612+0000","last_change":"2024-04-24T09:19:09.859258+0000","last_active":"2024-04-24T09:19:12.505612+0000","last_peered":"2024-04-24T09:19:12.505612+0000","last_clean":"2024-04-24T09:19:12.505612+0000","last_became_active":"2024-04-24T09:19:09.858388+0000","last_became_peered":"2024-04-24T09:19:09.858388+0000","last_unstale":"2024-04-24T09:19:12.505612+0000","last_undegraded":"2024-04-24T09:19:12.505612+0000","last_fullsized":"2024-04-24T09:19:12.505612+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:33:00.125914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.828956+0000","last_change":"2024-04-24T09:19:09.852140+0000","last_active":"2024-04-24T09:19:10.828956+0000","last_peered":"2024-04-24T09:19:10.828956+0000","last_clean":"2024-04-24T09:19:10.828956+0000","last_became_active":"2024-04-24T09:19:09.851766+0000","last_became_peered":"2024-04-24T09:19:09.851766+0000","last_unstale":"2024-04-24T09:19:10.828956+0000","last_undegraded":"2024-04-24T09:19:10.828956+0000","last_fullsized":"2024-04-24T09:19:10.828956+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:14:51.794470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.837237+0000","last_change":"2024-04-24T09:19:09.839389+0000","last_active":"2024-04-24T09:19:10.837237+0000","last_peered":"2024-04-24T09:19:10.837237+0000","last_clean":"2024-04-24T09:19:10.837237+0000","last_became_active":"2024-04-24T09:19:09.838989+0000","last_became_peered":"2024-04-24T09:19:09.838989+0000","last_unstale":"2024-04-24T09:19:10.837237+0000","last_undegraded":"2024-04-24T09:19:10.837237+0000","last_fullsized":"2024-04-24T09:19:10.837237+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:00:28.081837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:11.235469+0000","last_change":"2024-04-24T09:19:09.845304+0000","last_active":"2024-04-24T09:19:11.235469+0000","last_peered":"2024-04-24T09:19:11.235469+0000","last_clean":"2024-04-24T09:19:11.235469+0000","last_became_active":"2024-04-24T09:19:09.844979+0000","last_became_peered":"2024-04-24T09:19:09.844979+0000","last_unstale":"2024-04-24T09:19:11.235469+0000","last_undegraded":"2024-04-24T09:19:11.235469+0000","last_fullsized":"2024-04-24T09:19:11.235469+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:36:58.294217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.823248+0000","last_change":"2024-04-24T09:19:09.865120+0000","last_active":"2024-04-24T09:19:10.823248+0000","last_peered":"2024-04-24T09:19:10.823248+0000","last_clean":"2024-04-24T09:19:10.823248+0000","last_became_active":"2024-04-24T09:19:09.856837+0000","last_became_peered":"2024-04-24T09:19:09.856837+0000","last_unstale":"2024-04-24T09:19:10.823248+0000","last_undegraded":"2024-04-24T09:19:10.823248+0000","last_fullsized":"2024-04-24T09:19:10.823248+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:30:51.888927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:11.404773+0000","last_change":"2024-04-24T09:19:09.853625+0000","last_active":"2024-04-24T09:19:11.404773+0000","last_peered":"2024-04-24T09:19:11.404773+0000","last_clean":"2024-04-24T09:19:11.404773+0000","last_became_active":"2024-04-24T09:19:09.853390+0000","last_became_peered":"2024-04-24T09:19:09.853390+0000","last_unstale":"2024-04-24T09:19:11.404773+0000","last_undegraded":"2024-04-24T09:19:11.404773+0000","last_fullsized":"2024-04-24T09:19:11.404773+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:14:48.571283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.822719+0000","last_change":"2024-04-24T09:19:09.851380+0000","last_active":"2024-04-24T09:19:10.822719+0000","last_peered":"2024-04-24T09:19:10.822719+0000","last_clean":"2024-04-24T09:19:10.822719+0000","last_became_active":"2024-04-24T09:19:09.843723+0000","last_became_peered":"2024-04-24T09:19:09.843723+0000","last_unstale":"2024-04-24T09:19:10.822719+0000","last_undegraded":"2024-04-24T09:19:10.822719+0000","last_fullsized":"2024-04-24T09:19:10.822719+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:55:50.103891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:12.500712+0000","last_change":"2024-04-24T09:19:09.856620+0000","last_active":"2024-04-24T09:19:12.500712+0000","last_peered":"2024-04-24T09:19:12.500712+0000","last_clean":"2024-04-24T09:19:12.500712+0000","last_became_active":"2024-04-24T09:19:09.856101+0000","last_became_peered":"2024-04-24T09:19:09.856101+0000","last_unstale":"2024-04-24T09:19:12.500712+0000","last_undegraded":"2024-04-24T09:19:12.500712+0000","last_fullsized":"2024-04-24T09:19:12.500712+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:32:26.035336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.829230+0000","last_change":"2024-04-24T09:19:09.853959+0000","last_active":"2024-04-24T09:19:10.829230+0000","last_peered":"2024-04-24T09:19:10.829230+0000","last_clean":"2024-04-24T09:19:10.829230+0000","last_became_active":"2024-04-24T09:19:09.853588+0000","last_became_peered":"2024-04-24T09:19:09.853588+0000","last_unstale":"2024-04-24T09:19:10.829230+0000","last_undegraded":"2024-04-24T09:19:10.829230+0000","last_fullsized":"2024-04-24T09:19:10.829230+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:43:09.128758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.837058+0000","last_change":"2024-04-24T09:19:09.845317+0000","last_active":"2024-04-24T09:19:10.837058+0000","last_peered":"2024-04-24T09:19:10.837058+0000","last_clean":"2024-04-24T09:19:10.837058+0000","last_became_active":"2024-04-24T09:19:09.844975+0000","last_became_peered":"2024-04-24T09:19:09.844975+0000","last_unstale":"2024-04-24T09:19:10.837058+0000","last_undegraded":"2024-04-24T09:19:10.837058+0000","last_fullsized":"2024-04-24T09:19:10.837058+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:36:28.053045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":246913,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":49143,"num_objects":90,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":29,"num_read_kb":59,"num_write":205,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":61,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":64129,"data_compressed":1870,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},"log_size":177,"ondisk_log_size":177,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350992,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":933888,"data_stored":442764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612628,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1069056,"data_stored":1077561,"data_compressed":37613,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776155,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":946176,"data_stored":442480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037794,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":443072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299432,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":1035473,"data_compressed":35743,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462957,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":441684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":933888,"data_stored":443184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":950272,"data_stored":443050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411396,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":443840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149758,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":442002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888120,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":442216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724595,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1011712,"data_stored":1034178,"data_compressed":35743,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34099,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34099,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35743,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35743,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35743,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35743,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35743,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":45504,"data_compressed":1870,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T09:19:19.609 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T09:19:19.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:19 smithi149 ceph-mon[32592]: from='client.24671 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:19:19.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:19 smithi149 ceph-mon[32592]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-24T09:19:19.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:19 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/636796812' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:19:19.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:19 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/401846159' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:19:19.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:19 smithi114 ceph-mon[32501]: from='client.24671 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:19:19.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:19 smithi114 ceph-mon[32501]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-24T09:19:19.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:19 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/636796812' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:19:19.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:19 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/401846159' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:19:19.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.851+0000 7faf33000640 1 -- 172.21.15.90:0/1732098491 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faf2c109670 msgr2=0x7faf2c109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:19.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.851+0000 7faf33000640 1 --2- 172.21.15.90:0/1732098491 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faf2c109670 0x7faf2c109af0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7faf18009900 tx=0x7faf1802a410 comp rx=0 tx=0).stop 2024-04-24T09:19:19.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.851+0000 7faf33000640 1 -- 172.21.15.90:0/1732098491 shutdown_connections 2024-04-24T09:19:19.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.851+0000 7faf33000640 1 --2- 172.21.15.90:0/1732098491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf2c10a030 0x7faf2c1108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:19.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.851+0000 7faf33000640 1 --2- 172.21.15.90:0/1732098491 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faf2c109670 0x7faf2c109af0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:19.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.851+0000 7faf33000640 1 --2- 172.21.15.90:0/1732098491 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf2c108480 0x7faf2c1088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:19.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.851+0000 7faf33000640 1 -- 172.21.15.90:0/1732098491 >> 172.21.15.90:0/1732098491 conn(0x7faf2c077440 msgr2=0x7faf2c079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:19.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.851+0000 7faf33000640 1 -- 172.21.15.90:0/1732098491 shutdown_connections 2024-04-24T09:19:19.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.851+0000 7faf33000640 1 -- 172.21.15.90:0/1732098491 wait complete. 2024-04-24T09:19:19.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.852+0000 7faf33000640 1 Processor -- start 2024-04-24T09:19:19.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.852+0000 7faf33000640 1 -- start start 2024-04-24T09:19:19.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.853+0000 7faf33000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf2c108480 0x7faf2c19e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:19.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.853+0000 7faf33000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf2c109670 0x7faf2c19ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:19.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.853+0000 7faf33000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faf2c10a030 0x7faf2c1a5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:19.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.853+0000 7faf33000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7faf2c114160 con 0x7faf2c109670 2024-04-24T09:19:19.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.853+0000 7faf33000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7faf2c113fe0 con 0x7faf2c10a030 2024-04-24T09:19:19.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.853+0000 7faf33000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7faf2c1142e0 con 0x7faf2c108480 2024-04-24T09:19:19.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.853+0000 7faf31200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf2c109670 0x7faf2c19ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:19.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.853+0000 7faf31c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf2c108480 0x7faf2c19e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:19.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.853+0000 7faf31c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf2c108480 0x7faf2c19e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:43098/0 (socket says 172.21.15.90:43098) 2024-04-24T09:19:19.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.853+0000 7faf31c00640 1 -- 172.21.15.90:0/2311441034 learned_addr learned my addr 172.21.15.90:0/2311441034 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:19:19.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.853+0000 7faf32600640 1 --2- 172.21.15.90:0/2311441034 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faf2c10a030 0x7faf2c1a5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:19.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.853+0000 7faf31c00640 1 -- 172.21.15.90:0/2311441034 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faf2c10a030 msgr2=0x7faf2c1a5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:19.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.853+0000 7faf31c00640 1 --2- 172.21.15.90:0/2311441034 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faf2c10a030 0x7faf2c1a5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:19.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.853+0000 7faf31c00640 1 -- 172.21.15.90:0/2311441034 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf2c109670 msgr2=0x7faf2c19ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:19.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.853+0000 7faf31c00640 1 --2- 172.21.15.90:0/2311441034 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf2c109670 0x7faf2c19ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:19.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.853+0000 7faf31c00640 1 -- 172.21.15.90:0/2311441034 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf2c1a63a0 con 0x7faf2c108480 2024-04-24T09:19:19.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.853+0000 7faf32600640 1 --2- 172.21.15.90:0/2311441034 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faf2c10a030 0x7faf2c1a5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:19:19.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.854+0000 7faf31c00640 1 --2- 172.21.15.90:0/2311441034 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf2c108480 0x7faf2c19e780 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7faf2000e9c0 tx=0x7faf2000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:19.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.854+0000 7faf2a000640 1 -- 172.21.15.90:0/2311441034 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2000cd80 con 0x7faf2c108480 2024-04-24T09:19:19.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.854+0000 7faf2a000640 1 -- 172.21.15.90:0/2311441034 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf200035f0 con 0x7faf2c108480 2024-04-24T09:19:19.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.854+0000 7faf33000640 1 -- 172.21.15.90:0/2311441034 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf2c1a6690 con 0x7faf2c108480 2024-04-24T09:19:19.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.854+0000 7faf2a000640 1 -- 172.21.15.90:0/2311441034 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf20010690 con 0x7faf2c108480 2024-04-24T09:19:19.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.854+0000 7faf33000640 1 -- 172.21.15.90:0/2311441034 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf2c1a6b60 con 0x7faf2c108480 2024-04-24T09:19:19.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.855+0000 7faf33000640 1 -- 172.21.15.90:0/2311441034 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf2c10f7a0 con 0x7faf2c108480 2024-04-24T09:19:19.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.857+0000 7faf2a000640 1 -- 172.21.15.90:0/2311441034 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7faf200037a0 con 0x7faf2c108480 2024-04-24T09:19:19.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.858+0000 7faf2a000640 1 --2- 172.21.15.90:0/2311441034 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7faf140a3c40 0x7faf140a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:19.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.859+0000 7faf2a000640 1 -- 172.21.15.90:0/2311441034 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7faf200d0840 con 0x7faf2c108480 2024-04-24T09:19:19.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.859+0000 7faf31200640 1 --2- 172.21.15.90:0/2311441034 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7faf140a3c40 0x7faf140a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:19.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.860+0000 7faf31200640 1 --2- 172.21.15.90:0/2311441034 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7faf140a3c40 0x7faf140a6100 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7faf18031440 tx=0x7faf180315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:19.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:19.863+0000 7faf2a000640 1 -- 172.21.15.90:0/2311441034 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faf2009d170 con 0x7faf2c108480 2024-04-24T09:19:20.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:20.063+0000 7faf33000640 1 -- 172.21.15.90:0/2311441034 --> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faf2c062440 con 0x7faf140a3c40 2024-04-24T09:19:20.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:20.076+0000 7faf2a000640 1 -- 172.21.15.90:0/2311441034 <== mgr.14152 v2:172.21.15.90:6800/4261662956 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258947 (secure 0 0 0) 0x7faf2c062440 con 0x7faf140a3c40 2024-04-24T09:19:20.079 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:19:20.080 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-24T09:19:20.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:20.081+0000 7faf33000640 1 -- 172.21.15.90:0/2311441034 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7faf140a3c40 msgr2=0x7faf140a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:20.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:20.081+0000 7faf33000640 1 --2- 172.21.15.90:0/2311441034 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7faf140a3c40 0x7faf140a6100 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7faf18031440 tx=0x7faf180315b0 comp rx=0 tx=0).stop 2024-04-24T09:19:20.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:20.081+0000 7faf33000640 1 -- 172.21.15.90:0/2311441034 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf2c108480 msgr2=0x7faf2c19e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:20.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:20.081+0000 7faf33000640 1 --2- 172.21.15.90:0/2311441034 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf2c108480 0x7faf2c19e780 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7faf2000e9c0 tx=0x7faf2000ee80 comp rx=0 tx=0).stop 2024-04-24T09:19:20.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:20.081+0000 7faf33000640 1 -- 172.21.15.90:0/2311441034 shutdown_connections 2024-04-24T09:19:20.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:20.081+0000 7faf33000640 1 --2- 172.21.15.90:0/2311441034 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7faf140a3c40 0x7faf140a6100 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:20.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:20.081+0000 7faf33000640 1 --2- 172.21.15.90:0/2311441034 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faf2c10a030 0x7faf2c1a5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:20.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:20.081+0000 7faf33000640 1 --2- 172.21.15.90:0/2311441034 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf2c109670 0x7faf2c19ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:20.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:20.081+0000 7faf33000640 1 --2- 172.21.15.90:0/2311441034 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf2c108480 0x7faf2c19e780 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:20.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:20.081+0000 7faf33000640 1 -- 172.21.15.90:0/2311441034 >> 172.21.15.90:0/2311441034 conn(0x7faf2c077440 msgr2=0x7faf2c10c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:20.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:20.081+0000 7faf33000640 1 -- 172.21.15.90:0/2311441034 shutdown_connections 2024-04-24T09:19:20.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:20.081+0000 7faf33000640 1 -- 172.21.15.90:0/2311441034 wait complete. 2024-04-24T09:19:20.100 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2024-04-24T09:19:19.519971+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":639745,"num_objects":94,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":155,"num_read_kb":168,"num_write":388,"num_write_kb":3043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":61,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":316,"ondisk_log_size":316,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368772,"kb_used_data":11292,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124573052,"statfs":{"total":1151940427776,"available":1151562805248,"internally_reserved":0,"allocated":11563008,"data_stored":7131504,"data_compressed":109099,"data_compressed_allocated":139264,"data_compressed_original":1777664,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":49377,"num_objects":92,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":0,"num_read":29,"num_read_kb":59,"num_write":207,"num_write_kb":105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":61,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.701890"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.837342+0000","last_change":"2024-04-24T09:19:09.860043+0000","last_active":"2024-04-24T09:19:10.837342+0000","last_peered":"2024-04-24T09:19:10.837342+0000","last_clean":"2024-04-24T09:19:10.837342+0000","last_became_active":"2024-04-24T09:19:09.859766+0000","last_became_peered":"2024-04-24T09:19:09.859766+0000","last_unstale":"2024-04-24T09:19:10.837342+0000","last_undegraded":"2024-04-24T09:19:10.837342+0000","last_fullsized":"2024-04-24T09:19:10.837342+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:40:46.241239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:12.505761+0000","last_change":"2024-04-24T09:19:09.847293+0000","last_active":"2024-04-24T09:19:12.505761+0000","last_peered":"2024-04-24T09:19:12.505761+0000","last_clean":"2024-04-24T09:19:12.505761+0000","last_became_active":"2024-04-24T09:19:09.846955+0000","last_became_peered":"2024-04-24T09:19:09.846955+0000","last_unstale":"2024-04-24T09:19:12.505761+0000","last_undegraded":"2024-04-24T09:19:12.505761+0000","last_fullsized":"2024-04-24T09:19:12.505761+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:38:10.062654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:12.505624+0000","last_change":"2024-04-24T09:19:09.858954+0000","last_active":"2024-04-24T09:19:12.505624+0000","last_peered":"2024-04-24T09:19:12.505624+0000","last_clean":"2024-04-24T09:19:12.505624+0000","last_became_active":"2024-04-24T09:19:09.855915+0000","last_became_peered":"2024-04-24T09:19:09.855915+0000","last_unstale":"2024-04-24T09:19:12.505624+0000","last_undegraded":"2024-04-24T09:19:12.505624+0000","last_fullsized":"2024-04-24T09:19:12.505624+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:12:44.214700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:11.347658+0000","last_change":"2024-04-24T09:19:09.848582+0000","last_active":"2024-04-24T09:19:11.347658+0000","last_peered":"2024-04-24T09:19:11.347658+0000","last_clean":"2024-04-24T09:19:11.347658+0000","last_became_active":"2024-04-24T09:19:09.848163+0000","last_became_peered":"2024-04-24T09:19:09.848163+0000","last_unstale":"2024-04-24T09:19:11.347658+0000","last_undegraded":"2024-04-24T09:19:11.347658+0000","last_fullsized":"2024-04-24T09:19:11.347658+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:33:46.106081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:11.950548+0000","last_change":"2024-04-24T09:19:09.864608+0000","last_active":"2024-04-24T09:19:11.950548+0000","last_peered":"2024-04-24T09:19:11.950548+0000","last_clean":"2024-04-24T09:19:11.950548+0000","last_became_active":"2024-04-24T09:19:09.864271+0000","last_became_peered":"2024-04-24T09:19:09.864271+0000","last_unstale":"2024-04-24T09:19:11.950548+0000","last_undegraded":"2024-04-24T09:19:11.950548+0000","last_fullsized":"2024-04-24T09:19:11.950548+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:08:57.535882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:12.501605+0000","last_change":"2024-04-24T09:19:09.838589+0000","last_active":"2024-04-24T09:19:12.501605+0000","last_peered":"2024-04-24T09:19:12.501605+0000","last_clean":"2024-04-24T09:19:12.501605+0000","last_became_active":"2024-04-24T09:19:09.837628+0000","last_became_peered":"2024-04-24T09:19:09.837628+0000","last_unstale":"2024-04-24T09:19:12.501605+0000","last_undegraded":"2024-04-24T09:19:12.501605+0000","last_fullsized":"2024-04-24T09:19:12.501605+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:29:56.967777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.821798+0000","last_change":"2024-04-24T09:19:09.846296+0000","last_active":"2024-04-24T09:19:10.821798+0000","last_peered":"2024-04-24T09:19:10.821798+0000","last_clean":"2024-04-24T09:19:10.821798+0000","last_became_active":"2024-04-24T09:19:09.846010+0000","last_became_peered":"2024-04-24T09:19:09.846010+0000","last_unstale":"2024-04-24T09:19:10.821798+0000","last_undegraded":"2024-04-24T09:19:10.821798+0000","last_fullsized":"2024-04-24T09:19:10.821798+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:29:38.507747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.822766+0000","last_change":"2024-04-24T09:19:09.866821+0000","last_active":"2024-04-24T09:19:10.822766+0000","last_peered":"2024-04-24T09:19:10.822766+0000","last_clean":"2024-04-24T09:19:10.822766+0000","last_became_active":"2024-04-24T09:19:09.861793+0000","last_became_peered":"2024-04-24T09:19:09.861793+0000","last_unstale":"2024-04-24T09:19:10.822766+0000","last_undegraded":"2024-04-24T09:19:10.822766+0000","last_fullsized":"2024-04-24T09:19:10.822766+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:25:25.095642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:12.500884+0000","last_change":"2024-04-24T09:19:09.846433+0000","last_active":"2024-04-24T09:19:12.500884+0000","last_peered":"2024-04-24T09:19:12.500884+0000","last_clean":"2024-04-24T09:19:12.500884+0000","last_became_active":"2024-04-24T09:19:09.846044+0000","last_became_peered":"2024-04-24T09:19:09.846044+0000","last_unstale":"2024-04-24T09:19:12.500884+0000","last_undegraded":"2024-04-24T09:19:12.500884+0000","last_fullsized":"2024-04-24T09:19:12.500884+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:02:41.566626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:12.502075+0000","last_change":"2024-04-24T09:19:09.836662+0000","last_active":"2024-04-24T09:19:12.502075+0000","last_peered":"2024-04-24T09:19:12.502075+0000","last_clean":"2024-04-24T09:19:12.502075+0000","last_became_active":"2024-04-24T09:19:09.836283+0000","last_became_peered":"2024-04-24T09:19:09.836283+0000","last_unstale":"2024-04-24T09:19:12.502075+0000","last_undegraded":"2024-04-24T09:19:12.502075+0000","last_fullsized":"2024-04-24T09:19:12.502075+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:16:16.503469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.822274+0000","last_change":"2024-04-24T09:19:09.844369+0000","last_active":"2024-04-24T09:19:10.822274+0000","last_peered":"2024-04-24T09:19:10.822274+0000","last_clean":"2024-04-24T09:19:10.822274+0000","last_became_active":"2024-04-24T09:19:09.844044+0000","last_became_peered":"2024-04-24T09:19:09.844044+0000","last_unstale":"2024-04-24T09:19:10.822274+0000","last_undegraded":"2024-04-24T09:19:10.822274+0000","last_fullsized":"2024-04-24T09:19:10.822274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:51:45.861678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:11.347581+0000","last_change":"2024-04-24T09:19:09.846589+0000","last_active":"2024-04-24T09:19:11.347581+0000","last_peered":"2024-04-24T09:19:11.347581+0000","last_clean":"2024-04-24T09:19:11.347581+0000","last_became_active":"2024-04-24T09:19:09.844074+0000","last_became_peered":"2024-04-24T09:19:09.844074+0000","last_unstale":"2024-04-24T09:19:11.347581+0000","last_undegraded":"2024-04-24T09:19:11.347581+0000","last_fullsized":"2024-04-24T09:19:11.347581+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:47:12.002280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.828968+0000","last_change":"2024-04-24T09:19:09.847860+0000","last_active":"2024-04-24T09:19:10.828968+0000","last_peered":"2024-04-24T09:19:10.828968+0000","last_clean":"2024-04-24T09:19:10.828968+0000","last_became_active":"2024-04-24T09:19:09.847477+0000","last_became_peered":"2024-04-24T09:19:09.847477+0000","last_unstale":"2024-04-24T09:19:10.828968+0000","last_undegraded":"2024-04-24T09:19:10.828968+0000","last_fullsized":"2024-04-24T09:19:10.828968+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:39:44.655033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:12.505404+0000","last_change":"2024-04-24T09:19:09.854561+0000","last_active":"2024-04-24T09:19:12.505404+0000","last_peered":"2024-04-24T09:19:12.505404+0000","last_clean":"2024-04-24T09:19:12.505404+0000","last_became_active":"2024-04-24T09:19:09.849681+0000","last_became_peered":"2024-04-24T09:19:09.849681+0000","last_unstale":"2024-04-24T09:19:12.505404+0000","last_undegraded":"2024-04-24T09:19:12.505404+0000","last_fullsized":"2024-04-24T09:19:12.505404+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:46:36.572491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:12.505341+0000","last_change":"2024-04-24T09:19:09.859518+0000","last_active":"2024-04-24T09:19:12.505341+0000","last_peered":"2024-04-24T09:19:12.505341+0000","last_clean":"2024-04-24T09:19:12.505341+0000","last_became_active":"2024-04-24T09:19:09.859211+0000","last_became_peered":"2024-04-24T09:19:09.859211+0000","last_unstale":"2024-04-24T09:19:12.505341+0000","last_undegraded":"2024-04-24T09:19:12.505341+0000","last_fullsized":"2024-04-24T09:19:12.505341+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:20:04.273210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:12.500374+0000","last_change":"2024-04-24T09:19:09.837995+0000","last_active":"2024-04-24T09:19:12.500374+0000","last_peered":"2024-04-24T09:19:12.500374+0000","last_clean":"2024-04-24T09:19:12.500374+0000","last_became_active":"2024-04-24T09:19:09.837625+0000","last_became_peered":"2024-04-24T09:19:09.837625+0000","last_unstale":"2024-04-24T09:19:12.500374+0000","last_undegraded":"2024-04-24T09:19:12.500374+0000","last_fullsized":"2024-04-24T09:19:12.500374+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:14:30.602137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.837461+0000","last_change":"2024-04-24T09:19:09.855596+0000","last_active":"2024-04-24T09:19:10.837461+0000","last_peered":"2024-04-24T09:19:10.837461+0000","last_clean":"2024-04-24T09:19:10.837461+0000","last_became_active":"2024-04-24T09:19:09.855250+0000","last_became_peered":"2024-04-24T09:19:09.855250+0000","last_unstale":"2024-04-24T09:19:10.837461+0000","last_undegraded":"2024-04-24T09:19:10.837461+0000","last_fullsized":"2024-04-24T09:19:10.837461+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:48:58.862920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:11.116162+0000","last_change":"2024-04-24T09:19:09.844898+0000","last_active":"2024-04-24T09:19:11.116162+0000","last_peered":"2024-04-24T09:19:11.116162+0000","last_clean":"2024-04-24T09:19:11.116162+0000","last_became_active":"2024-04-24T09:19:09.844566+0000","last_became_peered":"2024-04-24T09:19:09.844566+0000","last_unstale":"2024-04-24T09:19:11.116162+0000","last_undegraded":"2024-04-24T09:19:11.116162+0000","last_fullsized":"2024-04-24T09:19:11.116162+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:15:34.145221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:11.235234+0000","last_change":"2024-04-24T09:19:09.856299+0000","last_active":"2024-04-24T09:19:11.235234+0000","last_peered":"2024-04-24T09:19:11.235234+0000","last_clean":"2024-04-24T09:19:11.235234+0000","last_became_active":"2024-04-24T09:19:09.855963+0000","last_became_peered":"2024-04-24T09:19:09.855963+0000","last_unstale":"2024-04-24T09:19:11.235234+0000","last_undegraded":"2024-04-24T09:19:11.235234+0000","last_fullsized":"2024-04-24T09:19:11.235234+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:10:27.372894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:16.267602+0000","last_change":"2024-04-24T09:19:08.973018+0000","last_active":"2024-04-24T09:19:16.267602+0000","last_peered":"2024-04-24T09:19:16.267602+0000","last_clean":"2024-04-24T09:19:16.267602+0000","last_became_active":"2024-04-24T09:19:08.972637+0000","last_became_peered":"2024-04-24T09:19:08.972637+0000","last_unstale":"2024-04-24T09:19:16.267602+0000","last_undegraded":"2024-04-24T09:19:16.267602+0000","last_fullsized":"2024-04-24T09:19:16.267602+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:11:10.890872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.823426+0000","last_change":"2024-04-24T09:19:09.862806+0000","last_active":"2024-04-24T09:19:10.823426+0000","last_peered":"2024-04-24T09:19:10.823426+0000","last_clean":"2024-04-24T09:19:10.823426+0000","last_became_active":"2024-04-24T09:19:09.862499+0000","last_became_peered":"2024-04-24T09:19:09.862499+0000","last_unstale":"2024-04-24T09:19:10.823426+0000","last_undegraded":"2024-04-24T09:19:10.823426+0000","last_fullsized":"2024-04-24T09:19:10.823426+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:13:19.494260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:12.103116+0000","last_change":"2024-04-24T09:19:08.973130+0000","last_active":"2024-04-24T09:19:12.103116+0000","last_peered":"2024-04-24T09:19:12.103116+0000","last_clean":"2024-04-24T09:19:12.103116+0000","last_became_active":"2024-04-24T09:19:08.972797+0000","last_became_peered":"2024-04-24T09:19:08.972797+0000","last_unstale":"2024-04-24T09:19:12.103116+0000","last_undegraded":"2024-04-24T09:19:12.103116+0000","last_fullsized":"2024-04-24T09:19:12.103116+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:21:49.236525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:12.505889+0000","last_change":"2024-04-24T09:19:09.860809+0000","last_active":"2024-04-24T09:19:12.505889+0000","last_peered":"2024-04-24T09:19:12.505889+0000","last_clean":"2024-04-24T09:19:12.505889+0000","last_became_active":"2024-04-24T09:19:09.860480+0000","last_became_peered":"2024-04-24T09:19:09.860480+0000","last_unstale":"2024-04-24T09:19:12.505889+0000","last_undegraded":"2024-04-24T09:19:12.505889+0000","last_fullsized":"2024-04-24T09:19:12.505889+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:39:21.114116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:16.270214+0000","last_change":"2024-04-24T09:19:08.950145+0000","last_active":"2024-04-24T09:19:16.270214+0000","last_peered":"2024-04-24T09:19:16.270214+0000","last_clean":"2024-04-24T09:19:16.270214+0000","last_became_active":"2024-04-24T09:19:08.949808+0000","last_became_peered":"2024-04-24T09:19:08.949808+0000","last_unstale":"2024-04-24T09:19:16.270214+0000","last_undegraded":"2024-04-24T09:19:16.270214+0000","last_fullsized":"2024-04-24T09:19:16.270214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:15:36.962348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:12.500706+0000","last_change":"2024-04-24T09:19:09.850123+0000","last_active":"2024-04-24T09:19:12.500706+0000","last_peered":"2024-04-24T09:19:12.500706+0000","last_clean":"2024-04-24T09:19:12.500706+0000","last_became_active":"2024-04-24T09:19:09.849770+0000","last_became_peered":"2024-04-24T09:19:09.849770+0000","last_unstale":"2024-04-24T09:19:12.500706+0000","last_undegraded":"2024-04-24T09:19:12.500706+0000","last_fullsized":"2024-04-24T09:19:12.500706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:10:00.374950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:16.269407+0000","last_change":"2024-04-24T09:19:08.952190+0000","last_active":"2024-04-24T09:19:16.269407+0000","last_peered":"2024-04-24T09:19:16.269407+0000","last_clean":"2024-04-24T09:19:16.269407+0000","last_became_active":"2024-04-24T09:19:08.951848+0000","last_became_peered":"2024-04-24T09:19:08.951848+0000","last_unstale":"2024-04-24T09:19:16.269407+0000","last_undegraded":"2024-04-24T09:19:16.269407+0000","last_fullsized":"2024-04-24T09:19:16.269407+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:02:54.892658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:11.404960+0000","last_change":"2024-04-24T09:19:09.845083+0000","last_active":"2024-04-24T09:19:11.404960+0000","last_peered":"2024-04-24T09:19:11.404960+0000","last_clean":"2024-04-24T09:19:11.404960+0000","last_became_active":"2024-04-24T09:19:09.844516+0000","last_became_peered":"2024-04-24T09:19:09.844516+0000","last_unstale":"2024-04-24T09:19:11.404960+0000","last_undegraded":"2024-04-24T09:19:11.404960+0000","last_fullsized":"2024-04-24T09:19:11.404960+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:10:41.995167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:13.273666+0000","last_change":"2024-04-24T09:19:08.950498+0000","last_active":"2024-04-24T09:19:13.273666+0000","last_peered":"2024-04-24T09:19:13.273666+0000","last_clean":"2024-04-24T09:19:13.273666+0000","last_became_active":"2024-04-24T09:19:08.950189+0000","last_became_peered":"2024-04-24T09:19:08.950189+0000","last_unstale":"2024-04-24T09:19:13.273666+0000","last_undegraded":"2024-04-24T09:19:13.273666+0000","last_fullsized":"2024-04-24T09:19:13.273666+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:40:41.342349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:11.950403+0000","last_change":"2024-04-24T09:19:09.847388+0000","last_active":"2024-04-24T09:19:11.950403+0000","last_peered":"2024-04-24T09:19:11.950403+0000","last_clean":"2024-04-24T09:19:11.950403+0000","last_became_active":"2024-04-24T09:19:09.846999+0000","last_became_peered":"2024-04-24T09:19:09.846999+0000","last_unstale":"2024-04-24T09:19:11.950403+0000","last_undegraded":"2024-04-24T09:19:11.950403+0000","last_fullsized":"2024-04-24T09:19:11.950403+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:49:09.814212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:15.305362+0000","last_change":"2024-04-24T09:19:08.965244+0000","last_active":"2024-04-24T09:19:15.305362+0000","last_peered":"2024-04-24T09:19:15.305362+0000","last_clean":"2024-04-24T09:19:15.305362+0000","last_became_active":"2024-04-24T09:19:08.964899+0000","last_became_peered":"2024-04-24T09:19:08.964899+0000","last_unstale":"2024-04-24T09:19:15.305362+0000","last_undegraded":"2024-04-24T09:19:15.305362+0000","last_fullsized":"2024-04-24T09:19:15.305362+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:53:28.035567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:11.404721+0000","last_change":"2024-04-24T09:19:09.864902+0000","last_active":"2024-04-24T09:19:11.404721+0000","last_peered":"2024-04-24T09:19:11.404721+0000","last_clean":"2024-04-24T09:19:11.404721+0000","last_became_active":"2024-04-24T09:19:09.864500+0000","last_became_peered":"2024-04-24T09:19:09.864500+0000","last_unstale":"2024-04-24T09:19:11.404721+0000","last_undegraded":"2024-04-24T09:19:11.404721+0000","last_fullsized":"2024-04-24T09:19:11.404721+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:37:31.810050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:14.303865+0000","last_change":"2024-04-24T09:19:08.977987+0000","last_active":"2024-04-24T09:19:14.303865+0000","last_peered":"2024-04-24T09:19:14.303865+0000","last_clean":"2024-04-24T09:19:14.303865+0000","last_became_active":"2024-04-24T09:19:08.977666+0000","last_became_peered":"2024-04-24T09:19:08.977666+0000","last_unstale":"2024-04-24T09:19:14.303865+0000","last_undegraded":"2024-04-24T09:19:14.303865+0000","last_fullsized":"2024-04-24T09:19:14.303865+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:21:21.328463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.822595+0000","last_change":"2024-04-24T09:19:09.863990+0000","last_active":"2024-04-24T09:19:10.822595+0000","last_peered":"2024-04-24T09:19:10.822595+0000","last_clean":"2024-04-24T09:19:10.822595+0000","last_became_active":"2024-04-24T09:19:09.854938+0000","last_became_peered":"2024-04-24T09:19:09.854938+0000","last_unstale":"2024-04-24T09:19:10.822595+0000","last_undegraded":"2024-04-24T09:19:10.822595+0000","last_fullsized":"2024-04-24T09:19:10.822595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:41:55.054373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:13.273611+0000","last_change":"2024-04-24T09:19:08.949938+0000","last_active":"2024-04-24T09:19:13.273611+0000","last_peered":"2024-04-24T09:19:13.273611+0000","last_clean":"2024-04-24T09:19:13.273611+0000","last_became_active":"2024-04-24T09:19:08.949561+0000","last_became_peered":"2024-04-24T09:19:08.949561+0000","last_unstale":"2024-04-24T09:19:13.273611+0000","last_undegraded":"2024-04-24T09:19:13.273611+0000","last_fullsized":"2024-04-24T09:19:13.273611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:55:50.488129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:11.115600+0000","last_change":"2024-04-24T09:19:09.841782+0000","last_active":"2024-04-24T09:19:11.115600+0000","last_peered":"2024-04-24T09:19:11.115600+0000","last_clean":"2024-04-24T09:19:11.115600+0000","last_became_active":"2024-04-24T09:19:09.841452+0000","last_became_peered":"2024-04-24T09:19:09.841452+0000","last_unstale":"2024-04-24T09:19:11.115600+0000","last_undegraded":"2024-04-24T09:19:11.115600+0000","last_fullsized":"2024-04-24T09:19:11.115600+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:58:19.618999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:18.254031+0000","last_change":"2024-04-24T09:19:08.973169+0000","last_active":"2024-04-24T09:19:18.254031+0000","last_peered":"2024-04-24T09:19:18.254031+0000","last_clean":"2024-04-24T09:19:18.254031+0000","last_became_active":"2024-04-24T09:19:08.972835+0000","last_became_peered":"2024-04-24T09:19:08.972835+0000","last_unstale":"2024-04-24T09:19:18.254031+0000","last_undegraded":"2024-04-24T09:19:18.254031+0000","last_fullsized":"2024-04-24T09:19:18.254031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:34:04.156280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:12.500895+0000","last_change":"2024-04-24T09:19:09.854957+0000","last_active":"2024-04-24T09:19:12.500895+0000","last_peered":"2024-04-24T09:19:12.500895+0000","last_clean":"2024-04-24T09:19:12.500895+0000","last_became_active":"2024-04-24T09:19:09.854574+0000","last_became_peered":"2024-04-24T09:19:09.854574+0000","last_unstale":"2024-04-24T09:19:12.500895+0000","last_undegraded":"2024-04-24T09:19:12.500895+0000","last_fullsized":"2024-04-24T09:19:12.500895+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:07:28.296335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:16.267920+0000","last_change":"2024-04-24T09:19:08.953179+0000","last_active":"2024-04-24T09:19:16.267920+0000","last_peered":"2024-04-24T09:19:16.267920+0000","last_clean":"2024-04-24T09:19:16.267920+0000","last_became_active":"2024-04-24T09:19:08.952864+0000","last_became_peered":"2024-04-24T09:19:08.952864+0000","last_unstale":"2024-04-24T09:19:16.267920+0000","last_undegraded":"2024-04-24T09:19:16.267920+0000","last_fullsized":"2024-04-24T09:19:16.267920+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:22:31.470593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:12.505883+0000","last_change":"2024-04-24T09:19:09.850855+0000","last_active":"2024-04-24T09:19:12.505883+0000","last_peered":"2024-04-24T09:19:12.505883+0000","last_clean":"2024-04-24T09:19:12.505883+0000","last_became_active":"2024-04-24T09:19:09.850535+0000","last_became_peered":"2024-04-24T09:19:09.850535+0000","last_unstale":"2024-04-24T09:19:12.505883+0000","last_undegraded":"2024-04-24T09:19:12.505883+0000","last_fullsized":"2024-04-24T09:19:12.505883+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:27:23.294935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:13.278571+0000","last_change":"2024-04-24T09:19:08.967882+0000","last_active":"2024-04-24T09:19:13.278571+0000","last_peered":"2024-04-24T09:19:13.278571+0000","last_clean":"2024-04-24T09:19:13.278571+0000","last_became_active":"2024-04-24T09:19:08.967625+0000","last_became_peered":"2024-04-24T09:19:08.967625+0000","last_unstale":"2024-04-24T09:19:13.278571+0000","last_undegraded":"2024-04-24T09:19:13.278571+0000","last_fullsized":"2024-04-24T09:19:13.278571+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:39:57.802717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:12.500765+0000","last_change":"2024-04-24T09:19:09.844721+0000","last_active":"2024-04-24T09:19:12.500765+0000","last_peered":"2024-04-24T09:19:12.500765+0000","last_clean":"2024-04-24T09:19:12.500765+0000","last_became_active":"2024-04-24T09:19:09.844365+0000","last_became_peered":"2024-04-24T09:19:09.844365+0000","last_unstale":"2024-04-24T09:19:12.500765+0000","last_undegraded":"2024-04-24T09:19:12.500765+0000","last_fullsized":"2024-04-24T09:19:12.500765+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:11:11.215015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'22","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:18.261921+0000","last_change":"2024-04-24T09:19:08.955409+0000","last_active":"2024-04-24T09:19:18.261921+0000","last_peered":"2024-04-24T09:19:18.261921+0000","last_clean":"2024-04-24T09:19:18.261921+0000","last_became_active":"2024-04-24T09:19:08.955065+0000","last_became_peered":"2024-04-24T09:19:08.955065+0000","last_unstale":"2024-04-24T09:19:18.261921+0000","last_undegraded":"2024-04-24T09:19:18.261921+0000","last_fullsized":"2024-04-24T09:19:18.261921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:28:34.411386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43377,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":10,"num_read_kb":42,"num_write":24,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:11.950698+0000","last_change":"2024-04-24T09:19:09.846521+0000","last_active":"2024-04-24T09:19:11.950698+0000","last_peered":"2024-04-24T09:19:11.950698+0000","last_clean":"2024-04-24T09:19:11.950698+0000","last_became_active":"2024-04-24T09:19:09.846162+0000","last_became_peered":"2024-04-24T09:19:09.846162+0000","last_unstale":"2024-04-24T09:19:11.950698+0000","last_undegraded":"2024-04-24T09:19:11.950698+0000","last_fullsized":"2024-04-24T09:19:11.950698+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:05:50.383432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:14.303266+0000","last_change":"2024-04-24T09:19:08.963754+0000","last_active":"2024-04-24T09:19:14.303266+0000","last_peered":"2024-04-24T09:19:14.303266+0000","last_clean":"2024-04-24T09:19:14.303266+0000","last_became_active":"2024-04-24T09:19:08.963379+0000","last_became_peered":"2024-04-24T09:19:08.963379+0000","last_unstale":"2024-04-24T09:19:14.303266+0000","last_undegraded":"2024-04-24T09:19:14.303266+0000","last_fullsized":"2024-04-24T09:19:14.303266+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:25:01.538860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.828763+0000","last_change":"2024-04-24T09:19:09.863649+0000","last_active":"2024-04-24T09:19:10.828763+0000","last_peered":"2024-04-24T09:19:10.828763+0000","last_clean":"2024-04-24T09:19:10.828763+0000","last_became_active":"2024-04-24T09:19:09.863295+0000","last_became_peered":"2024-04-24T09:19:09.863295+0000","last_unstale":"2024-04-24T09:19:10.828763+0000","last_undegraded":"2024-04-24T09:19:10.828763+0000","last_fullsized":"2024-04-24T09:19:10.828763+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:06:56.404909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:13.276677+0000","last_change":"2024-04-24T09:19:08.961409+0000","last_active":"2024-04-24T09:19:13.276677+0000","last_peered":"2024-04-24T09:19:13.276677+0000","last_clean":"2024-04-24T09:19:13.276677+0000","last_became_active":"2024-04-24T09:19:08.961085+0000","last_became_peered":"2024-04-24T09:19:08.961085+0000","last_unstale":"2024-04-24T09:19:13.276677+0000","last_undegraded":"2024-04-24T09:19:13.276677+0000","last_fullsized":"2024-04-24T09:19:13.276677+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:10:32.816204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.822226+0000","last_change":"2024-04-24T09:19:09.845992+0000","last_active":"2024-04-24T09:19:10.822226+0000","last_peered":"2024-04-24T09:19:10.822226+0000","last_clean":"2024-04-24T09:19:10.822226+0000","last_became_active":"2024-04-24T09:19:09.845673+0000","last_became_peered":"2024-04-24T09:19:09.845673+0000","last_unstale":"2024-04-24T09:19:10.822226+0000","last_undegraded":"2024-04-24T09:19:10.822226+0000","last_fullsized":"2024-04-24T09:19:10.822226+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:42:53.136476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:15.303603+0000","last_change":"2024-04-24T09:19:08.954872+0000","last_active":"2024-04-24T09:19:15.303603+0000","last_peered":"2024-04-24T09:19:15.303603+0000","last_clean":"2024-04-24T09:19:15.303603+0000","last_became_active":"2024-04-24T09:19:08.954540+0000","last_became_peered":"2024-04-24T09:19:08.954540+0000","last_unstale":"2024-04-24T09:19:15.303603+0000","last_undegraded":"2024-04-24T09:19:15.303603+0000","last_fullsized":"2024-04-24T09:19:15.303603+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:32:15.637337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.821813+0000","last_change":"2024-04-24T09:19:09.841735+0000","last_active":"2024-04-24T09:19:10.821813+0000","last_peered":"2024-04-24T09:19:10.821813+0000","last_clean":"2024-04-24T09:19:10.821813+0000","last_became_active":"2024-04-24T09:19:09.841422+0000","last_became_peered":"2024-04-24T09:19:09.841422+0000","last_unstale":"2024-04-24T09:19:10.821813+0000","last_undegraded":"2024-04-24T09:19:10.821813+0000","last_fullsized":"2024-04-24T09:19:10.821813+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:37:46.128344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:16.248975+0000","last_change":"2024-04-24T09:19:08.971108+0000","last_active":"2024-04-24T09:19:16.248975+0000","last_peered":"2024-04-24T09:19:16.248975+0000","last_clean":"2024-04-24T09:19:16.248975+0000","last_became_active":"2024-04-24T09:19:08.970808+0000","last_became_peered":"2024-04-24T09:19:08.970808+0000","last_unstale":"2024-04-24T09:19:16.248975+0000","last_undegraded":"2024-04-24T09:19:16.248975+0000","last_fullsized":"2024-04-24T09:19:16.248975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:37:26.311351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:11.404455+0000","last_change":"2024-04-24T09:19:09.848041+0000","last_active":"2024-04-24T09:19:11.404455+0000","last_peered":"2024-04-24T09:19:11.404455+0000","last_clean":"2024-04-24T09:19:11.404455+0000","last_became_active":"2024-04-24T09:19:09.847090+0000","last_became_peered":"2024-04-24T09:19:09.847090+0000","last_unstale":"2024-04-24T09:19:11.404455+0000","last_undegraded":"2024-04-24T09:19:11.404455+0000","last_fullsized":"2024-04-24T09:19:11.404455+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:45:13.652402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:15.293217+0000","last_change":"2024-04-24T09:19:08.952061+0000","last_active":"2024-04-24T09:19:15.293217+0000","last_peered":"2024-04-24T09:19:15.293217+0000","last_clean":"2024-04-24T09:19:15.293217+0000","last_became_active":"2024-04-24T09:19:08.950549+0000","last_became_peered":"2024-04-24T09:19:08.950549+0000","last_unstale":"2024-04-24T09:19:15.293217+0000","last_undegraded":"2024-04-24T09:19:15.293217+0000","last_fullsized":"2024-04-24T09:19:15.293217+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:41:40.949650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.837232+0000","last_change":"2024-04-24T09:19:09.848619+0000","last_active":"2024-04-24T09:19:10.837232+0000","last_peered":"2024-04-24T09:19:10.837232+0000","last_clean":"2024-04-24T09:19:10.837232+0000","last_became_active":"2024-04-24T09:19:09.848246+0000","last_became_peered":"2024-04-24T09:19:09.848246+0000","last_unstale":"2024-04-24T09:19:10.837232+0000","last_undegraded":"2024-04-24T09:19:10.837232+0000","last_fullsized":"2024-04-24T09:19:10.837232+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:30:22.925368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:11.577955+0000","last_change":"2024-04-24T09:19:08.977448+0000","last_active":"2024-04-24T09:19:11.577955+0000","last_peered":"2024-04-24T09:19:11.577955+0000","last_clean":"2024-04-24T09:19:11.577955+0000","last_became_active":"2024-04-24T09:19:08.976525+0000","last_became_peered":"2024-04-24T09:19:08.976525+0000","last_unstale":"2024-04-24T09:19:11.577955+0000","last_undegraded":"2024-04-24T09:19:11.577955+0000","last_fullsized":"2024-04-24T09:19:11.577955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:06:59.151486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:12.501924+0000","last_change":"2024-04-24T09:19:09.856260+0000","last_active":"2024-04-24T09:19:12.501924+0000","last_peered":"2024-04-24T09:19:12.501924+0000","last_clean":"2024-04-24T09:19:12.501924+0000","last_became_active":"2024-04-24T09:19:09.855895+0000","last_became_peered":"2024-04-24T09:19:09.855895+0000","last_unstale":"2024-04-24T09:19:12.501924+0000","last_undegraded":"2024-04-24T09:19:12.501924+0000","last_fullsized":"2024-04-24T09:19:12.501924+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:44:22.804210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:17.269583+0000","last_change":"2024-04-24T09:19:08.962268+0000","last_active":"2024-04-24T09:19:17.269583+0000","last_peered":"2024-04-24T09:19:17.269583+0000","last_clean":"2024-04-24T09:19:17.269583+0000","last_became_active":"2024-04-24T09:19:08.961930+0000","last_became_peered":"2024-04-24T09:19:08.961930+0000","last_unstale":"2024-04-24T09:19:17.269583+0000","last_undegraded":"2024-04-24T09:19:17.269583+0000","last_fullsized":"2024-04-24T09:19:17.269583+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:51:28.357674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:12.501826+0000","last_change":"2024-04-24T09:19:09.850782+0000","last_active":"2024-04-24T09:19:12.501826+0000","last_peered":"2024-04-24T09:19:12.501826+0000","last_clean":"2024-04-24T09:19:12.501826+0000","last_became_active":"2024-04-24T09:19:09.850458+0000","last_became_peered":"2024-04-24T09:19:09.850458+0000","last_unstale":"2024-04-24T09:19:12.501826+0000","last_undegraded":"2024-04-24T09:19:12.501826+0000","last_fullsized":"2024-04-24T09:19:12.501826+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:02:19.464011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:11.404791+0000","last_change":"2024-04-24T09:19:09.841551+0000","last_active":"2024-04-24T09:19:11.404791+0000","last_peered":"2024-04-24T09:19:11.404791+0000","last_clean":"2024-04-24T09:19:11.404791+0000","last_became_active":"2024-04-24T09:19:09.841219+0000","last_became_peered":"2024-04-24T09:19:09.841219+0000","last_unstale":"2024-04-24T09:19:11.404791+0000","last_undegraded":"2024-04-24T09:19:11.404791+0000","last_fullsized":"2024-04-24T09:19:11.404791+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:19:03.978876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:16.252226+0000","last_change":"2024-04-24T09:19:08.972070+0000","last_active":"2024-04-24T09:19:16.252226+0000","last_peered":"2024-04-24T09:19:16.252226+0000","last_clean":"2024-04-24T09:19:16.252226+0000","last_became_active":"2024-04-24T09:19:08.971699+0000","last_became_peered":"2024-04-24T09:19:08.971699+0000","last_unstale":"2024-04-24T09:19:16.252226+0000","last_undegraded":"2024-04-24T09:19:16.252226+0000","last_fullsized":"2024-04-24T09:19:16.252226+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:38:13.312940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.823472+0000","last_change":"2024-04-24T09:19:09.855902+0000","last_active":"2024-04-24T09:19:10.823472+0000","last_peered":"2024-04-24T09:19:10.823472+0000","last_clean":"2024-04-24T09:19:10.823472+0000","last_became_active":"2024-04-24T09:19:09.855503+0000","last_became_peered":"2024-04-24T09:19:09.855503+0000","last_unstale":"2024-04-24T09:19:10.823472+0000","last_undegraded":"2024-04-24T09:19:10.823472+0000","last_fullsized":"2024-04-24T09:19:10.823472+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:29:41.621269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:15.304833+0000","last_change":"2024-04-24T09:19:08.973774+0000","last_active":"2024-04-24T09:19:15.304833+0000","last_peered":"2024-04-24T09:19:15.304833+0000","last_clean":"2024-04-24T09:19:15.304833+0000","last_became_active":"2024-04-24T09:19:08.973372+0000","last_became_peered":"2024-04-24T09:19:08.973372+0000","last_unstale":"2024-04-24T09:19:15.304833+0000","last_undegraded":"2024-04-24T09:19:15.304833+0000","last_fullsized":"2024-04-24T09:19:15.304833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:25:12.910023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":338,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.821688+0000","last_change":"2024-04-24T09:17:48.282773+0000","last_active":"2024-04-24T09:19:10.821688+0000","last_peered":"2024-04-24T09:19:10.821688+0000","last_clean":"2024-04-24T09:19:10.821688+0000","last_became_active":"2024-04-24T09:17:48.282128+0000","last_became_peered":"2024-04-24T09:17:48.282128+0000","last_unstale":"2024-04-24T09:19:10.821688+0000","last_undegraded":"2024-04-24T09:19:10.821688+0000","last_fullsized":"2024-04-24T09:19:10.821688+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:13:45.562295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:13:45.562295+0000","last_clean_scrub_stamp":"2024-04-24T09:13:45.562295+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:54:34.557051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.828597+0000","last_change":"2024-04-24T09:19:09.847771+0000","last_active":"2024-04-24T09:19:10.828597+0000","last_peered":"2024-04-24T09:19:10.828597+0000","last_clean":"2024-04-24T09:19:10.828597+0000","last_became_active":"2024-04-24T09:19:09.845873+0000","last_became_peered":"2024-04-24T09:19:09.845873+0000","last_unstale":"2024-04-24T09:19:10.828597+0000","last_undegraded":"2024-04-24T09:19:10.828597+0000","last_fullsized":"2024-04-24T09:19:10.828597+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:31:46.124082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:14.297484+0000","last_change":"2024-04-24T09:19:08.950415+0000","last_active":"2024-04-24T09:19:14.297484+0000","last_peered":"2024-04-24T09:19:14.297484+0000","last_clean":"2024-04-24T09:19:14.297484+0000","last_became_active":"2024-04-24T09:19:08.949532+0000","last_became_peered":"2024-04-24T09:19:08.949532+0000","last_unstale":"2024-04-24T09:19:14.297484+0000","last_undegraded":"2024-04-24T09:19:14.297484+0000","last_fullsized":"2024-04-24T09:19:14.297484+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:35:48.093817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:16.248975+0000","last_change":"2024-04-24T09:19:08.971475+0000","last_active":"2024-04-24T09:19:16.248975+0000","last_peered":"2024-04-24T09:19:16.248975+0000","last_clean":"2024-04-24T09:19:16.248975+0000","last_became_active":"2024-04-24T09:19:08.971188+0000","last_became_peered":"2024-04-24T09:19:08.971188+0000","last_unstale":"2024-04-24T09:19:16.248975+0000","last_undegraded":"2024-04-24T09:19:16.248975+0000","last_fullsized":"2024-04-24T09:19:16.248975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:50:06.630182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.829200+0000","last_change":"2024-04-24T09:19:09.857102+0000","last_active":"2024-04-24T09:19:10.829200+0000","last_peered":"2024-04-24T09:19:10.829200+0000","last_clean":"2024-04-24T09:19:10.829200+0000","last_became_active":"2024-04-24T09:19:09.856632+0000","last_became_peered":"2024-04-24T09:19:09.856632+0000","last_unstale":"2024-04-24T09:19:10.829200+0000","last_undegraded":"2024-04-24T09:19:10.829200+0000","last_fullsized":"2024-04-24T09:19:10.829200+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:04:03.300741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'7","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:15.307441+0000","last_change":"2024-04-24T09:19:08.973387+0000","last_active":"2024-04-24T09:19:15.307441+0000","last_peered":"2024-04-24T09:19:15.307441+0000","last_clean":"2024-04-24T09:19:15.307441+0000","last_became_active":"2024-04-24T09:19:08.973081+0000","last_became_peered":"2024-04-24T09:19:08.973081+0000","last_unstale":"2024-04-24T09:19:15.307441+0000","last_undegraded":"2024-04-24T09:19:15.307441+0000","last_fullsized":"2024-04-24T09:19:15.307441+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:00:36.843053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.823406+0000","last_change":"2024-04-24T09:19:09.863386+0000","last_active":"2024-04-24T09:19:10.823406+0000","last_peered":"2024-04-24T09:19:10.823406+0000","last_clean":"2024-04-24T09:19:10.823406+0000","last_became_active":"2024-04-24T09:19:09.856300+0000","last_became_peered":"2024-04-24T09:19:09.856300+0000","last_unstale":"2024-04-24T09:19:10.823406+0000","last_undegraded":"2024-04-24T09:19:10.823406+0000","last_fullsized":"2024-04-24T09:19:10.823406+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:26:37.603307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:13.275842+0000","last_change":"2024-04-24T09:19:08.961751+0000","last_active":"2024-04-24T09:19:13.275842+0000","last_peered":"2024-04-24T09:19:13.275842+0000","last_clean":"2024-04-24T09:19:13.275842+0000","last_became_active":"2024-04-24T09:19:08.961389+0000","last_became_peered":"2024-04-24T09:19:08.961389+0000","last_unstale":"2024-04-24T09:19:13.275842+0000","last_undegraded":"2024-04-24T09:19:13.275842+0000","last_fullsized":"2024-04-24T09:19:13.275842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:25:17.913335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:12.506020+0000","last_change":"2024-04-24T09:19:09.862140+0000","last_active":"2024-04-24T09:19:12.506020+0000","last_peered":"2024-04-24T09:19:12.506020+0000","last_clean":"2024-04-24T09:19:12.506020+0000","last_became_active":"2024-04-24T09:19:09.860776+0000","last_became_peered":"2024-04-24T09:19:09.860776+0000","last_unstale":"2024-04-24T09:19:12.506020+0000","last_undegraded":"2024-04-24T09:19:12.506020+0000","last_fullsized":"2024-04-24T09:19:12.506020+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:08:29.437422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:13.278494+0000","last_change":"2024-04-24T09:19:08.973503+0000","last_active":"2024-04-24T09:19:13.278494+0000","last_peered":"2024-04-24T09:19:13.278494+0000","last_clean":"2024-04-24T09:19:13.278494+0000","last_became_active":"2024-04-24T09:19:08.973153+0000","last_became_peered":"2024-04-24T09:19:08.973153+0000","last_unstale":"2024-04-24T09:19:13.278494+0000","last_undegraded":"2024-04-24T09:19:13.278494+0000","last_fullsized":"2024-04-24T09:19:13.278494+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:57:21.489696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:11.950299+0000","last_change":"2024-04-24T09:19:09.838175+0000","last_active":"2024-04-24T09:19:11.950299+0000","last_peered":"2024-04-24T09:19:11.950299+0000","last_clean":"2024-04-24T09:19:11.950299+0000","last_became_active":"2024-04-24T09:19:09.837781+0000","last_became_peered":"2024-04-24T09:19:09.837781+0000","last_unstale":"2024-04-24T09:19:11.950299+0000","last_undegraded":"2024-04-24T09:19:11.950299+0000","last_fullsized":"2024-04-24T09:19:11.950299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:48:03.967587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:16.275013+0000","last_change":"2024-04-24T09:19:08.976790+0000","last_active":"2024-04-24T09:19:16.275013+0000","last_peered":"2024-04-24T09:19:16.275013+0000","last_clean":"2024-04-24T09:19:16.275013+0000","last_became_active":"2024-04-24T09:19:08.975196+0000","last_became_peered":"2024-04-24T09:19:08.975196+0000","last_unstale":"2024-04-24T09:19:16.275013+0000","last_undegraded":"2024-04-24T09:19:16.275013+0000","last_fullsized":"2024-04-24T09:19:16.275013+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:54:59.415169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.836983+0000","last_change":"2024-04-24T09:19:09.850958+0000","last_active":"2024-04-24T09:19:10.836983+0000","last_peered":"2024-04-24T09:19:10.836983+0000","last_clean":"2024-04-24T09:19:10.836983+0000","last_became_active":"2024-04-24T09:19:09.850660+0000","last_became_peered":"2024-04-24T09:19:09.850660+0000","last_unstale":"2024-04-24T09:19:10.836983+0000","last_undegraded":"2024-04-24T09:19:10.836983+0000","last_fullsized":"2024-04-24T09:19:10.836983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:41:27.496169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:11.234903+0000","last_change":"2024-04-24T09:19:08.963743+0000","last_active":"2024-04-24T09:19:11.234903+0000","last_peered":"2024-04-24T09:19:11.234903+0000","last_clean":"2024-04-24T09:19:11.234903+0000","last_became_active":"2024-04-24T09:19:08.963378+0000","last_became_peered":"2024-04-24T09:19:08.963378+0000","last_unstale":"2024-04-24T09:19:11.234903+0000","last_undegraded":"2024-04-24T09:19:11.234903+0000","last_fullsized":"2024-04-24T09:19:11.234903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:29:54.256411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.823151+0000","last_change":"2024-04-24T09:19:09.863144+0000","last_active":"2024-04-24T09:19:10.823151+0000","last_peered":"2024-04-24T09:19:10.823151+0000","last_clean":"2024-04-24T09:19:10.823151+0000","last_became_active":"2024-04-24T09:19:09.862772+0000","last_became_peered":"2024-04-24T09:19:09.862772+0000","last_unstale":"2024-04-24T09:19:10.823151+0000","last_undegraded":"2024-04-24T09:19:10.823151+0000","last_fullsized":"2024-04-24T09:19:10.823151+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:05:42.926470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:13.273329+0000","last_change":"2024-04-24T09:19:08.953467+0000","last_active":"2024-04-24T09:19:13.273329+0000","last_peered":"2024-04-24T09:19:13.273329+0000","last_clean":"2024-04-24T09:19:13.273329+0000","last_became_active":"2024-04-24T09:19:08.953116+0000","last_became_peered":"2024-04-24T09:19:08.953116+0000","last_unstale":"2024-04-24T09:19:13.273329+0000","last_undegraded":"2024-04-24T09:19:13.273329+0000","last_fullsized":"2024-04-24T09:19:13.273329+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:07:35.834466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.821925+0000","last_change":"2024-04-24T09:19:09.852329+0000","last_active":"2024-04-24T09:19:10.821925+0000","last_peered":"2024-04-24T09:19:10.821925+0000","last_clean":"2024-04-24T09:19:10.821925+0000","last_became_active":"2024-04-24T09:19:09.851762+0000","last_became_peered":"2024-04-24T09:19:09.851762+0000","last_unstale":"2024-04-24T09:19:10.821925+0000","last_undegraded":"2024-04-24T09:19:10.821925+0000","last_fullsized":"2024-04-24T09:19:10.821925+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:56:59.234680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:14.302634+0000","last_change":"2024-04-24T09:19:08.966699+0000","last_active":"2024-04-24T09:19:14.302634+0000","last_peered":"2024-04-24T09:19:14.302634+0000","last_clean":"2024-04-24T09:19:14.302634+0000","last_became_active":"2024-04-24T09:19:08.966315+0000","last_became_peered":"2024-04-24T09:19:08.966315+0000","last_unstale":"2024-04-24T09:19:14.302634+0000","last_undegraded":"2024-04-24T09:19:14.302634+0000","last_fullsized":"2024-04-24T09:19:14.302634+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:38:16.278751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.822325+0000","last_change":"2024-04-24T09:19:09.840340+0000","last_active":"2024-04-24T09:19:10.822325+0000","last_peered":"2024-04-24T09:19:10.822325+0000","last_clean":"2024-04-24T09:19:10.822325+0000","last_became_active":"2024-04-24T09:19:09.839689+0000","last_became_peered":"2024-04-24T09:19:09.839689+0000","last_unstale":"2024-04-24T09:19:10.822325+0000","last_undegraded":"2024-04-24T09:19:10.822325+0000","last_fullsized":"2024-04-24T09:19:10.822325+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:36:06.174519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:11.404709+0000","last_change":"2024-04-24T09:19:08.959025+0000","last_active":"2024-04-24T09:19:11.404709+0000","last_peered":"2024-04-24T09:19:11.404709+0000","last_clean":"2024-04-24T09:19:11.404709+0000","last_became_active":"2024-04-24T09:19:08.958749+0000","last_became_peered":"2024-04-24T09:19:08.958749+0000","last_unstale":"2024-04-24T09:19:11.404709+0000","last_undegraded":"2024-04-24T09:19:11.404709+0000","last_fullsized":"2024-04-24T09:19:11.404709+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:59:41.334448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:13.591078+0000","last_change":"2024-04-24T09:19:09.856258+0000","last_active":"2024-04-24T09:19:13.591078+0000","last_peered":"2024-04-24T09:19:13.591078+0000","last_clean":"2024-04-24T09:19:13.591078+0000","last_became_active":"2024-04-24T09:19:09.855919+0000","last_became_peered":"2024-04-24T09:19:09.855919+0000","last_unstale":"2024-04-24T09:19:13.591078+0000","last_undegraded":"2024-04-24T09:19:13.591078+0000","last_fullsized":"2024-04-24T09:19:13.591078+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:06:09.628268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:16.275466+0000","last_change":"2024-04-24T09:19:08.965880+0000","last_active":"2024-04-24T09:19:16.275466+0000","last_peered":"2024-04-24T09:19:16.275466+0000","last_clean":"2024-04-24T09:19:16.275466+0000","last_became_active":"2024-04-24T09:19:08.965378+0000","last_became_peered":"2024-04-24T09:19:08.965378+0000","last_unstale":"2024-04-24T09:19:16.275466+0000","last_undegraded":"2024-04-24T09:19:16.275466+0000","last_fullsized":"2024-04-24T09:19:16.275466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:08:22.628556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.822292+0000","last_change":"2024-04-24T09:19:09.845151+0000","last_active":"2024-04-24T09:19:10.822292+0000","last_peered":"2024-04-24T09:19:10.822292+0000","last_clean":"2024-04-24T09:19:10.822292+0000","last_became_active":"2024-04-24T09:19:09.844791+0000","last_became_peered":"2024-04-24T09:19:09.844791+0000","last_unstale":"2024-04-24T09:19:10.822292+0000","last_undegraded":"2024-04-24T09:19:10.822292+0000","last_fullsized":"2024-04-24T09:19:10.822292+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:38:00.782591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.837150+0000","last_change":"2024-04-24T09:19:09.844449+0000","last_active":"2024-04-24T09:19:10.837150+0000","last_peered":"2024-04-24T09:19:10.837150+0000","last_clean":"2024-04-24T09:19:10.837150+0000","last_became_active":"2024-04-24T09:19:09.844122+0000","last_became_peered":"2024-04-24T09:19:09.844122+0000","last_unstale":"2024-04-24T09:19:10.837150+0000","last_undegraded":"2024-04-24T09:19:10.837150+0000","last_fullsized":"2024-04-24T09:19:10.837150+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:36:01.857309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:11.235061+0000","last_change":"2024-04-24T09:19:09.843151+0000","last_active":"2024-04-24T09:19:11.235061+0000","last_peered":"2024-04-24T09:19:11.235061+0000","last_clean":"2024-04-24T09:19:11.235061+0000","last_became_active":"2024-04-24T09:19:09.842519+0000","last_became_peered":"2024-04-24T09:19:09.842519+0000","last_unstale":"2024-04-24T09:19:11.235061+0000","last_undegraded":"2024-04-24T09:19:11.235061+0000","last_fullsized":"2024-04-24T09:19:11.235061+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:26:00.791618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:11.234928+0000","last_change":"2024-04-24T09:19:09.856291+0000","last_active":"2024-04-24T09:19:11.234928+0000","last_peered":"2024-04-24T09:19:11.234928+0000","last_clean":"2024-04-24T09:19:11.234928+0000","last_became_active":"2024-04-24T09:19:09.855932+0000","last_became_peered":"2024-04-24T09:19:09.855932+0000","last_unstale":"2024-04-24T09:19:11.234928+0000","last_undegraded":"2024-04-24T09:19:11.234928+0000","last_fullsized":"2024-04-24T09:19:11.234928+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:11:14.113321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:12.505612+0000","last_change":"2024-04-24T09:19:09.859258+0000","last_active":"2024-04-24T09:19:12.505612+0000","last_peered":"2024-04-24T09:19:12.505612+0000","last_clean":"2024-04-24T09:19:12.505612+0000","last_became_active":"2024-04-24T09:19:09.858388+0000","last_became_peered":"2024-04-24T09:19:09.858388+0000","last_unstale":"2024-04-24T09:19:12.505612+0000","last_undegraded":"2024-04-24T09:19:12.505612+0000","last_fullsized":"2024-04-24T09:19:12.505612+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:33:00.125914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.828956+0000","last_change":"2024-04-24T09:19:09.852140+0000","last_active":"2024-04-24T09:19:10.828956+0000","last_peered":"2024-04-24T09:19:10.828956+0000","last_clean":"2024-04-24T09:19:10.828956+0000","last_became_active":"2024-04-24T09:19:09.851766+0000","last_became_peered":"2024-04-24T09:19:09.851766+0000","last_unstale":"2024-04-24T09:19:10.828956+0000","last_undegraded":"2024-04-24T09:19:10.828956+0000","last_fullsized":"2024-04-24T09:19:10.828956+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:14:51.794470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.837237+0000","last_change":"2024-04-24T09:19:09.839389+0000","last_active":"2024-04-24T09:19:10.837237+0000","last_peered":"2024-04-24T09:19:10.837237+0000","last_clean":"2024-04-24T09:19:10.837237+0000","last_became_active":"2024-04-24T09:19:09.838989+0000","last_became_peered":"2024-04-24T09:19:09.838989+0000","last_unstale":"2024-04-24T09:19:10.837237+0000","last_undegraded":"2024-04-24T09:19:10.837237+0000","last_fullsized":"2024-04-24T09:19:10.837237+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:00:28.081837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:11.235469+0000","last_change":"2024-04-24T09:19:09.845304+0000","last_active":"2024-04-24T09:19:11.235469+0000","last_peered":"2024-04-24T09:19:11.235469+0000","last_clean":"2024-04-24T09:19:11.235469+0000","last_became_active":"2024-04-24T09:19:09.844979+0000","last_became_peered":"2024-04-24T09:19:09.844979+0000","last_unstale":"2024-04-24T09:19:11.235469+0000","last_undegraded":"2024-04-24T09:19:11.235469+0000","last_fullsized":"2024-04-24T09:19:11.235469+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:36:58.294217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.823248+0000","last_change":"2024-04-24T09:19:09.865120+0000","last_active":"2024-04-24T09:19:10.823248+0000","last_peered":"2024-04-24T09:19:10.823248+0000","last_clean":"2024-04-24T09:19:10.823248+0000","last_became_active":"2024-04-24T09:19:09.856837+0000","last_became_peered":"2024-04-24T09:19:09.856837+0000","last_unstale":"2024-04-24T09:19:10.823248+0000","last_undegraded":"2024-04-24T09:19:10.823248+0000","last_fullsized":"2024-04-24T09:19:10.823248+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:30:51.888927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:11.404773+0000","last_change":"2024-04-24T09:19:09.853625+0000","last_active":"2024-04-24T09:19:11.404773+0000","last_peered":"2024-04-24T09:19:11.404773+0000","last_clean":"2024-04-24T09:19:11.404773+0000","last_became_active":"2024-04-24T09:19:09.853390+0000","last_became_peered":"2024-04-24T09:19:09.853390+0000","last_unstale":"2024-04-24T09:19:11.404773+0000","last_undegraded":"2024-04-24T09:19:11.404773+0000","last_fullsized":"2024-04-24T09:19:11.404773+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:14:48.571283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.822719+0000","last_change":"2024-04-24T09:19:09.851380+0000","last_active":"2024-04-24T09:19:10.822719+0000","last_peered":"2024-04-24T09:19:10.822719+0000","last_clean":"2024-04-24T09:19:10.822719+0000","last_became_active":"2024-04-24T09:19:09.843723+0000","last_became_peered":"2024-04-24T09:19:09.843723+0000","last_unstale":"2024-04-24T09:19:10.822719+0000","last_undegraded":"2024-04-24T09:19:10.822719+0000","last_fullsized":"2024-04-24T09:19:10.822719+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:55:50.103891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:12.500712+0000","last_change":"2024-04-24T09:19:09.856620+0000","last_active":"2024-04-24T09:19:12.500712+0000","last_peered":"2024-04-24T09:19:12.500712+0000","last_clean":"2024-04-24T09:19:12.500712+0000","last_became_active":"2024-04-24T09:19:09.856101+0000","last_became_peered":"2024-04-24T09:19:09.856101+0000","last_unstale":"2024-04-24T09:19:12.500712+0000","last_undegraded":"2024-04-24T09:19:12.500712+0000","last_fullsized":"2024-04-24T09:19:12.500712+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:32:26.035336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.829230+0000","last_change":"2024-04-24T09:19:09.853959+0000","last_active":"2024-04-24T09:19:10.829230+0000","last_peered":"2024-04-24T09:19:10.829230+0000","last_clean":"2024-04-24T09:19:10.829230+0000","last_became_active":"2024-04-24T09:19:09.853588+0000","last_became_peered":"2024-04-24T09:19:09.853588+0000","last_unstale":"2024-04-24T09:19:10.829230+0000","last_undegraded":"2024-04-24T09:19:10.829230+0000","last_fullsized":"2024-04-24T09:19:10.829230+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:43:09.128758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-24T09:19:10.837058+0000","last_change":"2024-04-24T09:19:09.845317+0000","last_active":"2024-04-24T09:19:10.837058+0000","last_peered":"2024-04-24T09:19:10.837058+0000","last_clean":"2024-04-24T09:19:10.837058+0000","last_became_active":"2024-04-24T09:19:09.844975+0000","last_became_peered":"2024-04-24T09:19:09.844975+0000","last_unstale":"2024-04-24T09:19:10.837058+0000","last_undegraded":"2024-04-24T09:19:10.837058+0000","last_fullsized":"2024-04-24T09:19:10.837058+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:36:28.053045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":246913,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":49143,"num_objects":90,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":29,"num_read_kb":59,"num_write":205,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":61,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":64129,"data_compressed":1870,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},"log_size":177,"ondisk_log_size":177,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350992,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":933888,"data_stored":442764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612628,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1069056,"data_stored":1077561,"data_compressed":37613,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776155,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":946176,"data_stored":442480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037794,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":443072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299432,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":1035473,"data_compressed":35743,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462957,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":441684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":933888,"data_stored":443184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":950272,"data_stored":443050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411396,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":443840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149758,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":442002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888120,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":442216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724595,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1011712,"data_stored":1034178,"data_compressed":35743,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34099,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34099,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35743,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35743,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35743,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35743,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35743,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":45504,"data_compressed":1870,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T09:19:20.104 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-24T09:19:20.104 INFO:tasks.cephadm:Setting up client nodes... 2024-04-24T09:19:20.105 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-24T09:19:20.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:20 smithi090 ceph-mon[27825]: pgmap v327: 97 pgs: 97 active+clean; 625 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 KiB/s wr, 24 op/s 2024-04-24T09:19:20.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:20 smithi090 ceph-mon[27825]: from='client.24721 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T09:19:20.581 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:19:20.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:20 smithi149 ceph-mon[32592]: pgmap v327: 97 pgs: 97 active+clean; 625 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 KiB/s wr, 24 op/s 2024-04-24T09:19:20.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:20 smithi149 ceph-mon[32592]: from='client.24721 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T09:19:20.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:20 smithi114 ceph-mon[32501]: pgmap v327: 97 pgs: 97 active+clean; 625 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 KiB/s wr, 24 op/s 2024-04-24T09:19:20.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:20 smithi114 ceph-mon[32501]: from='client.24721 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T09:19:20.676 INFO:journalctl@ceph.mds.b.smithi114.stdout:Apr 24 09:19:20 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-b[63608]: 2024-04-24T09:19:20.216+0000 7fda95279700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-04-24T09:19:20.676 INFO:journalctl@ceph.mds.b.smithi114.stdout:Apr 24 09:19:20 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-b[63608]: 2024-04-24T09:19:20.216+0000 7fda95279700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-04-24T09:19:21.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:21 smithi090 ceph-mon[27825]: from='client.24727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T09:19:21.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:21 smithi149 ceph-mon[32592]: from='client.24727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T09:19:21.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:21 smithi114 ceph-mon[32501]: from='client.24727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T09:19:22.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:22 smithi090 ceph-mon[27825]: pgmap v328: 97 pgs: 97 active+clean; 626 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 10 KiB/s wr, 26 op/s 2024-04-24T09:19:22.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:22 smithi149 ceph-mon[32592]: pgmap v328: 97 pgs: 97 active+clean; 626 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 10 KiB/s wr, 26 op/s 2024-04-24T09:19:22.675 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:22 smithi114 ceph-mon[32501]: pgmap v328: 97 pgs: 97 active+clean; 626 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 10 KiB/s wr, 26 op/s 2024-04-24T09:19:23.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.271+0000 7fa053600700 1 -- 172.21.15.90:0/3194221871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa04c105f90 msgr2=0x7fa04c108380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:23.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.271+0000 7fa053600700 1 --2- 172.21.15.90:0/3194221871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa04c105f90 0x7fa04c108380 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa038009990 tx=0x7fa038009ca0 comp rx=0 tx=0).stop 2024-04-24T09:19:23.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.271+0000 7fa053600700 1 -- 172.21.15.90:0/3194221871 shutdown_connections 2024-04-24T09:19:23.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.271+0000 7fa053600700 1 --2- 172.21.15.90:0/3194221871 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa04c1088c0 0x7fa04c10acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:23.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.271+0000 7fa053600700 1 --2- 172.21.15.90:0/3194221871 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa04c103660 0x7fa04c105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:23.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.271+0000 7fa053600700 1 --2- 172.21.15.90:0/3194221871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa04c105f90 0x7fa04c108380 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:23.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.271+0000 7fa053600700 1 -- 172.21.15.90:0/3194221871 >> 172.21.15.90:0/3194221871 conn(0x7fa04c0fd010 msgr2=0x7fa04c0ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:23.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.271+0000 7fa053600700 1 -- 172.21.15.90:0/3194221871 shutdown_connections 2024-04-24T09:19:23.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.271+0000 7fa053600700 1 -- 172.21.15.90:0/3194221871 wait complete. 2024-04-24T09:19:23.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.272+0000 7fa053600700 1 Processor -- start 2024-04-24T09:19:23.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.273+0000 7fa053600700 1 -- start start 2024-04-24T09:19:23.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.273+0000 7fa053600700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa04c103660 0x7fa04c198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:23.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.273+0000 7fa053600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa04c105f90 0x7fa04c1991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:23.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.273+0000 7fa053600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa04c1088c0 0x7fa04c1a0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:23.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.273+0000 7fa053600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa04c10e770 con 0x7fa04c105f90 2024-04-24T09:19:23.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.273+0000 7fa053600700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa04c10e5f0 con 0x7fa04c103660 2024-04-24T09:19:23.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.273+0000 7fa053600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa04c10e8f0 con 0x7fa04c1088c0 2024-04-24T09:19:23.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.273+0000 7fa051600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa04c1088c0 0x7fa04c1a0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:23.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.273+0000 7fa043e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa04c105f90 0x7fa04c1991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:23.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.273+0000 7fa051600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa04c1088c0 0x7fa04c1a0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:43110/0 (socket says 172.21.15.90:43110) 2024-04-24T09:19:23.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.273+0000 7fa050c00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa04c103660 0x7fa04c198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:23.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.273+0000 7fa051600700 1 -- 172.21.15.90:0/72500678 learned_addr learned my addr 172.21.15.90:0/72500678 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:19:23.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.273+0000 7fa043e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa04c105f90 0x7fa04c1991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47822/0 (socket says 172.21.15.90:47822) 2024-04-24T09:19:23.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.274+0000 7fa051600700 1 -- 172.21.15.90:0/72500678 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa04c103660 msgr2=0x7fa04c198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:23.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.274+0000 7fa051600700 1 --2- 172.21.15.90:0/72500678 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa04c103660 0x7fa04c198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:23.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.274+0000 7fa051600700 1 -- 172.21.15.90:0/72500678 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa04c105f90 msgr2=0x7fa04c1991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:23.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.274+0000 7fa051600700 1 --2- 172.21.15.90:0/72500678 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa04c105f90 0x7fa04c1991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:23.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.274+0000 7fa051600700 1 -- 172.21.15.90:0/72500678 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa04c1a0920 con 0x7fa04c1088c0 2024-04-24T09:19:23.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.274+0000 7fa050c00700 1 --2- 172.21.15.90:0/72500678 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa04c103660 0x7fa04c198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:19:23.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.274+0000 7fa051600700 1 --2- 172.21.15.90:0/72500678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa04c1088c0 0x7fa04c1a0250 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa04400c110 tx=0x7fa04400c420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:23.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.274+0000 7fa04a000700 1 -- 172.21.15.90:0/72500678 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0440092e0 con 0x7fa04c1088c0 2024-04-24T09:19:23.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.274+0000 7fa04a000700 1 -- 172.21.15.90:0/72500678 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa044003a00 con 0x7fa04c1088c0 2024-04-24T09:19:23.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.274+0000 7fa053600700 1 -- 172.21.15.90:0/72500678 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa04c101450 con 0x7fa04c1088c0 2024-04-24T09:19:23.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.274+0000 7fa04a000700 1 -- 172.21.15.90:0/72500678 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa044004680 con 0x7fa04c1088c0 2024-04-24T09:19:23.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.275+0000 7fa053600700 1 -- 172.21.15.90:0/72500678 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa04c101910 con 0x7fa04c1088c0 2024-04-24T09:19:23.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.276+0000 7fa053600700 1 -- 172.21.15.90:0/72500678 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa04c051030 con 0x7fa04c1088c0 2024-04-24T09:19:23.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.278+0000 7fa04a000700 1 -- 172.21.15.90:0/72500678 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa04401a020 con 0x7fa04c1088c0 2024-04-24T09:19:23.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.279+0000 7fa04a000700 1 --2- 172.21.15.90:0/72500678 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa0300a3d80 0x7fa0300a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:23.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.280+0000 7fa04a000700 1 -- 172.21.15.90:0/72500678 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fa0440cff80 con 0x7fa04c1088c0 2024-04-24T09:19:23.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.280+0000 7fa050c00700 1 --2- 172.21.15.90:0/72500678 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa0300a3d80 0x7fa0300a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:23.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.280+0000 7fa050c00700 1 --2- 172.21.15.90:0/72500678 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa0300a3d80 0x7fa0300a6240 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa04c199b80 tx=0x7fa03c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:23.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.281+0000 7fa04a000700 1 -- 172.21.15.90:0/72500678 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa044010040 con 0x7fa04c1088c0 2024-04-24T09:19:23.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.600+0000 7fa053600700 1 -- 172.21.15.90:0/72500678 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fa04c102cf0 con 0x7fa04c1088c0 2024-04-24T09:19:23.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.610+0000 7fa04a000700 1 -- 172.21.15.90:0/72500678 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fa04409c8b0 con 0x7fa04c1088c0 2024-04-24T09:19:23.610 INFO:teuthology.orchestra.run.smithi090.stdout:[client.0] 2024-04-24T09:19:23.610 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQCbzihmccf5IxAA4D+optt0Ny8SZ4nU9B8u6w== 2024-04-24T09:19:23.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.613+0000 7fa053600700 1 -- 172.21.15.90:0/72500678 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa0300a3d80 msgr2=0x7fa0300a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:23.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.613+0000 7fa053600700 1 --2- 172.21.15.90:0/72500678 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa0300a3d80 0x7fa0300a6240 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa04c199b80 tx=0x7fa03c009260 comp rx=0 tx=0).stop 2024-04-24T09:19:23.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.613+0000 7fa053600700 1 -- 172.21.15.90:0/72500678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa04c1088c0 msgr2=0x7fa04c1a0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:23.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.613+0000 7fa053600700 1 --2- 172.21.15.90:0/72500678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa04c1088c0 0x7fa04c1a0250 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa04400c110 tx=0x7fa04400c420 comp rx=0 tx=0).stop 2024-04-24T09:19:23.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.613+0000 7fa053600700 1 -- 172.21.15.90:0/72500678 shutdown_connections 2024-04-24T09:19:23.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.613+0000 7fa053600700 1 --2- 172.21.15.90:0/72500678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa04c1088c0 0x7fa04c1a0250 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:23.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.613+0000 7fa053600700 1 --2- 172.21.15.90:0/72500678 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa04c103660 0x7fa04c198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:23.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.613+0000 7fa053600700 1 --2- 172.21.15.90:0/72500678 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa0300a3d80 0x7fa0300a6240 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:23.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.613+0000 7fa053600700 1 --2- 172.21.15.90:0/72500678 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa04c105f90 0x7fa04c1991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:23.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.613+0000 7fa053600700 1 -- 172.21.15.90:0/72500678 >> 172.21.15.90:0/72500678 conn(0x7fa04c0fd010 msgr2=0x7fa04c103fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:23.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.613+0000 7fa053600700 1 -- 172.21.15.90:0/72500678 shutdown_connections 2024-04-24T09:19:23.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:23.613+0000 7fa053600700 1 -- 172.21.15.90:0/72500678 wait complete. 2024-04-24T09:19:24.592 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-24T09:19:24.592 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-24T09:19:24.592 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-24T09:19:24.663 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:24 smithi090 ceph-mon[27825]: pgmap v329: 97 pgs: 97 active+clean; 626 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 9.2 KiB/s wr, 24 op/s 2024-04-24T09:19:24.663 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:24 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/72500678' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T09:19:24.663 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:24 smithi090 ceph-mon[27825]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T09:19:24.663 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:24 smithi090 ceph-mon[27825]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-24T09:19:24.721 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-24T09:19:24.721 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-24T09:19:24.722 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-24T09:19:24.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:24 smithi149 ceph-mon[32592]: pgmap v329: 97 pgs: 97 active+clean; 626 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 9.2 KiB/s wr, 24 op/s 2024-04-24T09:19:24.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:24 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/72500678' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T09:19:24.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:24 smithi149 ceph-mon[32592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T09:19:24.862 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:24 smithi149 ceph-mon[32592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-24T09:19:24.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:24 smithi114 ceph-mon[32501]: pgmap v329: 97 pgs: 97 active+clean; 626 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 9.2 KiB/s wr, 24 op/s 2024-04-24T09:19:24.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:24 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/72500678' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T09:19:24.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:24 smithi114 ceph-mon[32501]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T09:19:24.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:24 smithi114 ceph-mon[32501]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-24T09:19:25.212 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:19:26.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:26 smithi090 ceph-mon[27825]: pgmap v330: 97 pgs: 97 active+clean; 638 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 10 KiB/s wr, 26 op/s 2024-04-24T09:19:26.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:26 smithi149 ceph-mon[32592]: pgmap v330: 97 pgs: 97 active+clean; 638 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 10 KiB/s wr, 26 op/s 2024-04-24T09:19:26.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:26 smithi114 ceph-mon[32501]: pgmap v330: 97 pgs: 97 active+clean; 638 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 10 KiB/s wr, 26 op/s 2024-04-24T09:19:27.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.562+0000 7f450be00700 1 -- 172.21.15.90:0/841044971 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4504104f70 msgr2=0x7f4504105350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:27.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.562+0000 7f450be00700 1 --2- 172.21.15.90:0/841044971 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4504104f70 0x7f4504105350 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f44f8009920 tx=0x7f44f8009c30 comp rx=0 tx=0).stop 2024-04-24T09:19:27.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.562+0000 7f450be00700 1 -- 172.21.15.90:0/841044971 shutdown_connections 2024-04-24T09:19:27.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.562+0000 7f450be00700 1 --2- 172.21.15.90:0/841044971 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f450410a050 0x7f4504111c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:27.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.562+0000 7f450be00700 1 --2- 172.21.15.90:0/841044971 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4504104f70 0x7f4504105350 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:27.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.562+0000 7f450be00700 1 --2- 172.21.15.90:0/841044971 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4504105920 0x7f4504109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:27.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.562+0000 7f450be00700 1 -- 172.21.15.90:0/841044971 >> 172.21.15.90:0/841044971 conn(0x7f4504100800 msgr2=0x7f4504102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:27.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.562+0000 7f450be00700 1 -- 172.21.15.90:0/841044971 shutdown_connections 2024-04-24T09:19:27.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.562+0000 7f450be00700 1 -- 172.21.15.90:0/841044971 wait complete. 2024-04-24T09:19:27.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.563+0000 7f450be00700 1 Processor -- start 2024-04-24T09:19:27.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.564+0000 7f450be00700 1 -- start start 2024-04-24T09:19:27.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.564+0000 7f450be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4504104f70 0x7f450419b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:27.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.564+0000 7f450be00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4504105920 0x7f450419bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:27.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.564+0000 7f450be00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f450410a050 0x7f450419fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:27.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.564+0000 7f450be00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f45041150a0 con 0x7f4504104f70 2024-04-24T09:19:27.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.564+0000 7f450be00700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4504114f20 con 0x7f450410a050 2024-04-24T09:19:27.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.564+0000 7f450be00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4504115220 con 0x7f4504105920 2024-04-24T09:19:27.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.565+0000 7f4509400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4504104f70 0x7f450419b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:27.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.565+0000 7f4509400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4504104f70 0x7f450419b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49002/0 (socket says 172.21.15.90:49002) 2024-04-24T09:19:27.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.565+0000 7f4509400700 1 -- 172.21.15.90:0/3729994747 learned_addr learned my addr 172.21.15.90:0/3729994747 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:19:27.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.565+0000 7f4508a00700 1 --2- 172.21.15.90:0/3729994747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4504105920 0x7f450419bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:27.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.565+0000 7f4509e00700 1 --2- 172.21.15.90:0/3729994747 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f450410a050 0x7f450419fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:27.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.565+0000 7f4509400700 1 -- 172.21.15.90:0/3729994747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4504105920 msgr2=0x7f450419bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:27.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.565+0000 7f4509400700 1 --2- 172.21.15.90:0/3729994747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4504105920 0x7f450419bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:27.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.565+0000 7f4509400700 1 -- 172.21.15.90:0/3729994747 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f450410a050 msgr2=0x7f450419fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:27.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.565+0000 7f4509400700 1 --2- 172.21.15.90:0/3729994747 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f450410a050 0x7f450419fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:27.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.565+0000 7f4509400700 1 -- 172.21.15.90:0/3729994747 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45041a05b0 con 0x7f4504104f70 2024-04-24T09:19:27.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.565+0000 7f4509400700 1 --2- 172.21.15.90:0/3729994747 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4504104f70 0x7f450419b5f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f44f8009230 tx=0x7f44f800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:27.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.566+0000 7f4502000700 1 -- 172.21.15.90:0/3729994747 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f8027070 con 0x7f4504104f70 2024-04-24T09:19:27.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.566+0000 7f4502000700 1 -- 172.21.15.90:0/3729994747 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44f80055a0 con 0x7f4504104f70 2024-04-24T09:19:27.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.566+0000 7f450be00700 1 -- 172.21.15.90:0/3729994747 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45041a0840 con 0x7f4504104f70 2024-04-24T09:19:27.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.566+0000 7f4502000700 1 -- 172.21.15.90:0/3729994747 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f8004910 con 0x7f4504104f70 2024-04-24T09:19:27.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.566+0000 7f450be00700 1 -- 172.21.15.90:0/3729994747 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45041a8120 con 0x7f4504104f70 2024-04-24T09:19:27.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.567+0000 7f450be00700 1 -- 172.21.15.90:0/3729994747 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4504106450 con 0x7f4504104f70 2024-04-24T09:19:27.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.569+0000 7f4502000700 1 -- 172.21.15.90:0/3729994747 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f44f802a070 con 0x7f4504104f70 2024-04-24T09:19:27.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.570+0000 7f4502000700 1 --2- 172.21.15.90:0/3729994747 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f44ec0a3cd0 0x7f44ec0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:27.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.570+0000 7f4502000700 1 -- 172.21.15.90:0/3729994747 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f44f80d0e60 con 0x7f4504104f70 2024-04-24T09:19:27.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.570+0000 7f4508a00700 1 --2- 172.21.15.90:0/3729994747 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f44ec0a3cd0 0x7f44ec0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:27.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.571+0000 7f4508a00700 1 --2- 172.21.15.90:0/3729994747 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f44ec0a3cd0 0x7f44ec0a6190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f450410b670 tx=0x7f44f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:27.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.575+0000 7f4502000700 1 -- 172.21.15.90:0/3729994747 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f44f805eff0 con 0x7f4504104f70 2024-04-24T09:19:27.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.872+0000 7f450be00700 1 -- 172.21.15.90:0/3729994747 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f450410d2c0 con 0x7f4504104f70 2024-04-24T09:19:27.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.889+0000 7f4502000700 1 -- 172.21.15.90:0/3729994747 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264998 (secure 0 0 0) 0x7f44f8013030 con 0x7f4504104f70 2024-04-24T09:19:27.892 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:19:27.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.896+0000 7f450be00700 1 -- 172.21.15.90:0/3729994747 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f44ec0a3cd0 msgr2=0x7f44ec0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:27.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.896+0000 7f450be00700 1 --2- 172.21.15.90:0/3729994747 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f44ec0a3cd0 0x7f44ec0a6190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f450410b670 tx=0x7f44f0009260 comp rx=0 tx=0).stop 2024-04-24T09:19:27.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.896+0000 7f450be00700 1 -- 172.21.15.90:0/3729994747 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4504104f70 msgr2=0x7f450419b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:27.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.896+0000 7f450be00700 1 --2- 172.21.15.90:0/3729994747 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4504104f70 0x7f450419b5f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f44f8009230 tx=0x7f44f800fa00 comp rx=0 tx=0).stop 2024-04-24T09:19:27.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.897+0000 7f450be00700 1 -- 172.21.15.90:0/3729994747 shutdown_connections 2024-04-24T09:19:27.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.897+0000 7f450be00700 1 --2- 172.21.15.90:0/3729994747 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f450410a050 0x7f450419fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:27.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.897+0000 7f450be00700 1 --2- 172.21.15.90:0/3729994747 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f44ec0a3cd0 0x7f44ec0a6190 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:27.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.897+0000 7f450be00700 1 --2- 172.21.15.90:0/3729994747 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4504104f70 0x7f450419b5f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:27.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.897+0000 7f450be00700 1 --2- 172.21.15.90:0/3729994747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4504105920 0x7f450419bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:27.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.897+0000 7f450be00700 1 -- 172.21.15.90:0/3729994747 >> 172.21.15.90:0/3729994747 conn(0x7f4504100800 msgr2=0x7f4504101b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:27.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.897+0000 7f450be00700 1 -- 172.21.15.90:0/3729994747 shutdown_connections 2024-04-24T09:19:27.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:27.897+0000 7f450be00700 1 -- 172.21.15.90:0/3729994747 wait complete. 2024-04-24T09:19:28.766 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":4261662956},{"type":"v1","addr":"172.21.15.90:6801","nonce":4261662956}]},"active_addr":"172.21.15.90:6801/4261662956","active_change":"2024-04-24T09:10:57.377773+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.90:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.90:0","nonce":2457607786}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.90:0","nonce":3397803323}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.90:0","nonce":1007262564}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.90:0","nonce":2135173549}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.90:0","nonce":1280564570}]}]} 2024-04-24T09:19:28.772 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-24T09:19:28.772 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-24T09:19:28.772 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-24T09:19:28.851 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:28 smithi090 ceph-mon[27825]: pgmap v331: 97 pgs: 97 active+clean; 638 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.2 KiB/s wr, 23 op/s 2024-04-24T09:19:28.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:28 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/3729994747' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-24T09:19:28.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:28 smithi149 ceph-mon[32592]: pgmap v331: 97 pgs: 97 active+clean; 638 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.2 KiB/s wr, 23 op/s 2024-04-24T09:19:28.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:28 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/3729994747' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-24T09:19:28.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:28 smithi114 ceph-mon[32501]: pgmap v331: 97 pgs: 97 active+clean; 638 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.2 KiB/s wr, 23 op/s 2024-04-24T09:19:28.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:28 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/3729994747' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-24T09:19:29.259 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:19:30.768 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:30 smithi090 ceph-mon[27825]: pgmap v332: 97 pgs: 97 active+clean; 638 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.7 KiB/s wr, 19 op/s 2024-04-24T09:19:30.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:30 smithi149 ceph-mon[32592]: pgmap v332: 97 pgs: 97 active+clean; 638 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.7 KiB/s wr, 19 op/s 2024-04-24T09:19:30.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:30 smithi114 ceph-mon[32501]: pgmap v332: 97 pgs: 97 active+clean; 638 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.7 KiB/s wr, 19 op/s 2024-04-24T09:19:31.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.565+0000 7f0c18200700 1 -- 172.21.15.90:0/753452266 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c1006a800 msgr2=0x7f0c1006ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:31.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.565+0000 7f0c18200700 1 --2- 172.21.15.90:0/753452266 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c1006a800 0x7f0c1006ac60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc009990 tx=0x7f0bfc009ca0 comp rx=0 tx=0).stop 2024-04-24T09:19:31.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.565+0000 7f0c18200700 1 -- 172.21.15.90:0/753452266 shutdown_connections 2024-04-24T09:19:31.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.565+0000 7f0c18200700 1 --2- 172.21.15.90:0/753452266 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c1006b1a0 0x7f0c1010f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:31.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.565+0000 7f0c18200700 1 --2- 172.21.15.90:0/753452266 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c10108b70 0x7f0c10108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:31.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.565+0000 7f0c18200700 1 --2- 172.21.15.90:0/753452266 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c1006a800 0x7f0c1006ac60 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:31.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.565+0000 7f0c18200700 1 -- 172.21.15.90:0/753452266 >> 172.21.15.90:0/753452266 conn(0x7f0c100789a0 msgr2=0x7f0c10078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:31.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.566+0000 7f0c18200700 1 -- 172.21.15.90:0/753452266 shutdown_connections 2024-04-24T09:19:31.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.566+0000 7f0c18200700 1 -- 172.21.15.90:0/753452266 wait complete. 2024-04-24T09:19:31.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.567+0000 7f0c18200700 1 Processor -- start 2024-04-24T09:19:31.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.567+0000 7f0c18200700 1 -- start start 2024-04-24T09:19:31.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.567+0000 7f0c18200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c1006a800 0x7f0c101992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:31.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.567+0000 7f0c18200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c1006b1a0 0x7f0c10199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:31.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.567+0000 7f0c18200700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c10108b70 0x7f0c1019dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:31.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.567+0000 7f0c18200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0c10112e60 con 0x7f0c1006b1a0 2024-04-24T09:19:31.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.567+0000 7f0c18200700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0c10112ce0 con 0x7f0c10108b70 2024-04-24T09:19:31.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.567+0000 7f0c18200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0c10112fe0 con 0x7f0c1006a800 2024-04-24T09:19:31.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.568+0000 7f0c14e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c1006b1a0 0x7f0c10199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:31.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.568+0000 7f0c16200700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c10108b70 0x7f0c1019dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:31.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.568+0000 7f0c15800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c1006a800 0x7f0c101992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:31.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.568+0000 7f0c16200700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c10108b70 0x7f0c1019dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.90:56420/0 (socket says 172.21.15.90:56420) 2024-04-24T09:19:31.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.568+0000 7f0c16200700 1 -- 172.21.15.90:0/1403283197 learned_addr learned my addr 172.21.15.90:0/1403283197 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:19:31.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.568+0000 7f0c16200700 1 -- 172.21.15.90:0/1403283197 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c1006a800 msgr2=0x7f0c101992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:31.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.568+0000 7f0c16200700 1 --2- 172.21.15.90:0/1403283197 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c1006a800 0x7f0c101992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:31.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.568+0000 7f0c16200700 1 -- 172.21.15.90:0/1403283197 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c1006b1a0 msgr2=0x7f0c10199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:31.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.568+0000 7f0c16200700 1 --2- 172.21.15.90:0/1403283197 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c1006b1a0 0x7f0c10199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:31.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.568+0000 7f0c16200700 1 -- 172.21.15.90:0/1403283197 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c1019e300 con 0x7f0c10108b70 2024-04-24T09:19:31.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.568+0000 7f0c16200700 1 --2- 172.21.15.90:0/1403283197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c10108b70 0x7f0c1019dbc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0c0800d980 tx=0x7f0c0800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:31.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.569+0000 7f0c0e000700 1 -- 172.21.15.90:0/1403283197 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c08017070 con 0x7f0c10108b70 2024-04-24T09:19:31.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.569+0000 7f0c0e000700 1 -- 172.21.15.90:0/1403283197 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c08005020 con 0x7f0c10108b70 2024-04-24T09:19:31.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.569+0000 7f0c0e000700 1 -- 172.21.15.90:0/1403283197 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0800fbf0 con 0x7f0c10108b70 2024-04-24T09:19:31.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.569+0000 7f0c18200700 1 -- 172.21.15.90:0/1403283197 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c1019e5f0 con 0x7f0c10108b70 2024-04-24T09:19:31.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.569+0000 7f0c18200700 1 -- 172.21.15.90:0/1403283197 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c101a5e70 con 0x7f0c10108b70 2024-04-24T09:19:31.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.570+0000 7f0c18200700 1 -- 172.21.15.90:0/1403283197 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c100697e0 con 0x7f0c10108b70 2024-04-24T09:19:31.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.572+0000 7f0c0e000700 1 -- 172.21.15.90:0/1403283197 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0c080051d0 con 0x7f0c10108b70 2024-04-24T09:19:31.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.573+0000 7f0c0e000700 1 --2- 172.21.15.90:0/1403283197 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f0bf80a3d50 0x7f0bf80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:31.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.574+0000 7f0c0e000700 1 -- 172.21.15.90:0/1403283197 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f0c080d0240 con 0x7f0c10108b70 2024-04-24T09:19:31.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.574+0000 7f0c15800700 1 --2- 172.21.15.90:0/1403283197 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f0bf80a3d50 0x7f0bf80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:31.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.574+0000 7f0c15800700 1 --2- 172.21.15.90:0/1403283197 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f0bf80a3d50 0x7f0bf80a6210 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400f540 tx=0x7f0c0400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:31.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.578+0000 7f0c0e000700 1 -- 172.21.15.90:0/1403283197 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0c08020020 con 0x7f0c10108b70 2024-04-24T09:19:31.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.804+0000 7f0c18200700 1 -- 172.21.15.90:0/1403283197 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0c1002e1a0 con 0x7f0c10108b70 2024-04-24T09:19:31.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.806+0000 7f0c0e000700 1 -- 172.21.15.90:0/1403283197 <== mon.2 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22231 (secure 0 0 0) 0x7f0c0809cb70 con 0x7f0c10108b70 2024-04-24T09:19:31.807 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:19:31.807 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":86,"fsid":"30ee9be2-021a-11ef-bc93-c7b262605968","created":"2024-04-24T09:09:49.496019+0000","modified":"2024-04-24T09:19:10.816601+0000","last_up_change":"2024-04-24T09:18:10.455632+0000","last_in_change":"2024-04-24T09:17:50.600366+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T09:13:45.548774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T09:19:07.185580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T09:19:08.374818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13a59b9f-b46e-4ec4-90af-bb7a3d5c753b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6803","nonce":2864288066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6805","nonce":2864288066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6809","nonce":2864288066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6807","nonce":2864288066}]},"public_addr":"172.21.15.90:6803/2864288066","cluster_addr":"172.21.15.90:6805/2864288066","heartbeat_back_addr":"172.21.15.90:6809/2864288066","heartbeat_front_addr":"172.21.15.90:6807/2864288066","state":["exists","up"]},{"osd":1,"uuid":"1567ce8f-24e6-42cf-bf64-f6f0c534a9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6801","nonce":344222414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6803","nonce":344222414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6807","nonce":344222414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6805","nonce":344222414}]},"public_addr":"172.21.15.114:6801/344222414","cluster_addr":"172.21.15.114:6803/344222414","heartbeat_back_addr":"172.21.15.114:6807/344222414","heartbeat_front_addr":"172.21.15.114:6805/344222414","state":["exists","up"]},{"osd":2,"uuid":"f861b304-9c7d-4629-8522-37f33a747945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6801","nonce":3257067419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6803","nonce":3257067419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6807","nonce":3257067419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6805","nonce":3257067419}]},"public_addr":"172.21.15.149:6801/3257067419","cluster_addr":"172.21.15.149:6803/3257067419","heartbeat_back_addr":"172.21.15.149:6807/3257067419","heartbeat_front_addr":"172.21.15.149:6805/3257067419","state":["exists","up"]},{"osd":3,"uuid":"9377ab6e-5996-45fb-93a9-475a2e5f17a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6811","nonce":3068524970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6813","nonce":3068524970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6817","nonce":3068524970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6815","nonce":3068524970}]},"public_addr":"172.21.15.90:6811/3068524970","cluster_addr":"172.21.15.90:6813/3068524970","heartbeat_back_addr":"172.21.15.90:6817/3068524970","heartbeat_front_addr":"172.21.15.90:6815/3068524970","state":["exists","up"]},{"osd":4,"uuid":"e2ddd19e-e641-4d61-b342-02b5f12b1c4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6809","nonce":2782015522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6811","nonce":2782015522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6815","nonce":2782015522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6813","nonce":2782015522}]},"public_addr":"172.21.15.114:6809/2782015522","cluster_addr":"172.21.15.114:6811/2782015522","heartbeat_back_addr":"172.21.15.114:6815/2782015522","heartbeat_front_addr":"172.21.15.114:6813/2782015522","state":["exists","up"]},{"osd":5,"uuid":"b6ff9d10-5210-4b46-a192-6dcbf9d1f156","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6809","nonce":4266460479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6811","nonce":4266460479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6815","nonce":4266460479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6813","nonce":4266460479}]},"public_addr":"172.21.15.149:6809/4266460479","cluster_addr":"172.21.15.149:6811/4266460479","heartbeat_back_addr":"172.21.15.149:6815/4266460479","heartbeat_front_addr":"172.21.15.149:6813/4266460479","state":["exists","up"]},{"osd":6,"uuid":"4287b5a1-f953-421c-bf62-5e390bee834d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6819","nonce":1121461276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6821","nonce":1121461276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6825","nonce":1121461276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6823","nonce":1121461276}]},"public_addr":"172.21.15.90:6819/1121461276","cluster_addr":"172.21.15.90:6821/1121461276","heartbeat_back_addr":"172.21.15.90:6825/1121461276","heartbeat_front_addr":"172.21.15.90:6823/1121461276","state":["exists","up"]},{"osd":7,"uuid":"a54e507d-d6ba-4bf2-9a7f-a7b95c3859bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6817","nonce":3222546280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6819","nonce":3222546280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6823","nonce":3222546280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6821","nonce":3222546280}]},"public_addr":"172.21.15.114:6817/3222546280","cluster_addr":"172.21.15.114:6819/3222546280","heartbeat_back_addr":"172.21.15.114:6823/3222546280","heartbeat_front_addr":"172.21.15.114:6821/3222546280","state":["exists","up"]},{"osd":8,"uuid":"9950b010-5865-4109-ae37-582b1dbd2f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6817","nonce":157615149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6819","nonce":157615149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6823","nonce":157615149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6821","nonce":157615149}]},"public_addr":"172.21.15.149:6817/157615149","cluster_addr":"172.21.15.149:6819/157615149","heartbeat_back_addr":"172.21.15.149:6823/157615149","heartbeat_front_addr":"172.21.15.149:6821/157615149","state":["exists","up"]},{"osd":9,"uuid":"31e2635c-e234-421f-9757-592fd673c936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6827","nonce":1307318076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6829","nonce":1307318076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6833","nonce":1307318076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6831","nonce":1307318076}]},"public_addr":"172.21.15.90:6827/1307318076","cluster_addr":"172.21.15.90:6829/1307318076","heartbeat_back_addr":"172.21.15.90:6833/1307318076","heartbeat_front_addr":"172.21.15.90:6831/1307318076","state":["exists","up"]},{"osd":10,"uuid":"46a3b2a6-e6e5-4578-b4bb-f3e998b9bc95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6825","nonce":1719802140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6827","nonce":1719802140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6831","nonce":1719802140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6829","nonce":1719802140}]},"public_addr":"172.21.15.114:6825/1719802140","cluster_addr":"172.21.15.114:6827/1719802140","heartbeat_back_addr":"172.21.15.114:6831/1719802140","heartbeat_front_addr":"172.21.15.114:6829/1719802140","state":["exists","up"]},{"osd":11,"uuid":"ccd4456b-e947-46cf-a779-148ffa289b58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6825","nonce":1737275166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6827","nonce":1737275166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6831","nonce":1737275166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6829","nonce":1737275166}]},"public_addr":"172.21.15.149:6825/1737275166","cluster_addr":"172.21.15.149:6827/1737275166","heartbeat_back_addr":"172.21.15.149:6831/1737275166","heartbeat_front_addr":"172.21.15.149:6829/1737275166","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:12:53.015078+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:13:18.047053+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:13:41.935891+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:14:15.095388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:14:42.512355+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:15:08.113775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:15:43.828550+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:16:10.901909+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:16:36.422962+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:17:13.924446+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:17:42.814401+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:18:08.672793+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/3090852580":"2024-04-25T09:10:57.377655+0000","172.21.15.90:0/687197394":"2024-04-25T09:10:57.377655+0000","172.21.15.90:6801/1809038230":"2024-04-25T09:10:57.377655+0000","172.21.15.90:6800/1809038230":"2024-04-25T09:10:57.377655+0000","172.21.15.90:0/2964149071":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/3617388113":"2024-04-25T09:10:22.599674+0000","172.21.15.90:6801/3089140842":"2024-04-25T09:10:22.599674+0000","172.21.15.90:6800/3089140842":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/2840416149":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/305432710":"2024-04-25T09:10:57.377655+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T09:19:31.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.809+0000 7f0c18200700 1 -- 172.21.15.90:0/1403283197 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f0bf80a3d50 msgr2=0x7f0bf80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:31.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.809+0000 7f0c18200700 1 --2- 172.21.15.90:0/1403283197 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f0bf80a3d50 0x7f0bf80a6210 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400f540 tx=0x7f0c0400b040 comp rx=0 tx=0).stop 2024-04-24T09:19:31.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.810+0000 7f0c18200700 1 -- 172.21.15.90:0/1403283197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c10108b70 msgr2=0x7f0c1019dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:31.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.810+0000 7f0c18200700 1 --2- 172.21.15.90:0/1403283197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c10108b70 0x7f0c1019dbc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0c0800d980 tx=0x7f0c0800dc90 comp rx=0 tx=0).stop 2024-04-24T09:19:31.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.810+0000 7f0c18200700 1 -- 172.21.15.90:0/1403283197 shutdown_connections 2024-04-24T09:19:31.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.810+0000 7f0c18200700 1 --2- 172.21.15.90:0/1403283197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0c10108b70 0x7f0c1019dbc0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:31.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.810+0000 7f0c18200700 1 --2- 172.21.15.90:0/1403283197 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c1006a800 0x7f0c101992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:31.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.810+0000 7f0c18200700 1 --2- 172.21.15.90:0/1403283197 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f0bf80a3d50 0x7f0bf80a6210 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:31.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.810+0000 7f0c18200700 1 --2- 172.21.15.90:0/1403283197 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c1006b1a0 0x7f0c10199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:31.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.810+0000 7f0c18200700 1 -- 172.21.15.90:0/1403283197 >> 172.21.15.90:0/1403283197 conn(0x7f0c100789a0 msgr2=0x7f0c100ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:31.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.810+0000 7f0c18200700 1 -- 172.21.15.90:0/1403283197 shutdown_connections 2024-04-24T09:19:31.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:31.810+0000 7f0c18200700 1 -- 172.21.15.90:0/1403283197 wait complete. 2024-04-24T09:19:32.697 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-24T09:19:32.697 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-24T09:19:32.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:32 smithi090 ceph-mon[27825]: pgmap v333: 97 pgs: 97 active+clean; 638 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-04-24T09:19:32.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:32 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/1403283197' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:19:32.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:32 smithi149 ceph-mon[32592]: pgmap v333: 97 pgs: 97 active+clean; 638 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-04-24T09:19:32.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:32 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/1403283197' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:19:32.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:32 smithi114 ceph-mon[32501]: pgmap v333: 97 pgs: 97 active+clean; 638 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-04-24T09:19:32.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:32 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/1403283197' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:19:33.195 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:19:34.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:34 smithi149 ceph-mon[32592]: pgmap v334: 97 pgs: 97 active+clean; 638 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-04-24T09:19:34.925 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:34 smithi114 ceph-mon[32501]: pgmap v334: 97 pgs: 97 active+clean; 638 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-04-24T09:19:34.950 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:34 smithi090 ceph-mon[27825]: pgmap v334: 97 pgs: 97 active+clean; 638 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-04-24T09:19:35.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.475+0000 7f8ae3600700 1 -- 172.21.15.90:0/2393268189 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8adc108b70 msgr2=0x7f8adc108f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:35.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.475+0000 7f8ae3600700 1 --2- 172.21.15.90:0/2393268189 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8adc108b70 0x7f8adc108f50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009990 tx=0x7f8ac4009ca0 comp rx=0 tx=0).stop 2024-04-24T09:19:35.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.476+0000 7f8ae3600700 1 -- 172.21.15.90:0/2393268189 shutdown_connections 2024-04-24T09:19:35.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.476+0000 7f8ae3600700 1 --2- 172.21.15.90:0/2393268189 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8adc06b1a0 0x7f8adc10f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:35.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.476+0000 7f8ae3600700 1 --2- 172.21.15.90:0/2393268189 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8adc108b70 0x7f8adc108f50 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:35.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.476+0000 7f8ae3600700 1 --2- 172.21.15.90:0/2393268189 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8adc06a800 0x7f8adc06ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:35.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.476+0000 7f8ae3600700 1 -- 172.21.15.90:0/2393268189 >> 172.21.15.90:0/2393268189 conn(0x7f8adc0789a0 msgr2=0x7f8adc078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:35.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.476+0000 7f8ae3600700 1 -- 172.21.15.90:0/2393268189 shutdown_connections 2024-04-24T09:19:35.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.476+0000 7f8ae3600700 1 -- 172.21.15.90:0/2393268189 wait complete. 2024-04-24T09:19:35.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.477+0000 7f8ae3600700 1 Processor -- start 2024-04-24T09:19:35.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.477+0000 7f8ae3600700 1 -- start start 2024-04-24T09:19:35.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.477+0000 7f8ae3600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8adc06a800 0x7f8adc19b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:35.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.477+0000 7f8ae3600700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8adc06b1a0 0x7f8adc19ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:35.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.477+0000 7f8ae3600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8adc108b70 0x7f8adc19fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:35.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.477+0000 7f8ae3600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8adc112e00 con 0x7f8adc06a800 2024-04-24T09:19:35.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.477+0000 7f8ae3600700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8adc112c80 con 0x7f8adc06b1a0 2024-04-24T09:19:35.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.477+0000 7f8ae3600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8adc112f80 con 0x7f8adc108b70 2024-04-24T09:19:35.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.478+0000 7f8ae0c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8adc06a800 0x7f8adc19b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:35.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.478+0000 7f8ad3e00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8adc06b1a0 0x7f8adc19ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:35.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.478+0000 7f8ae0c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8adc06a800 0x7f8adc19b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49032/0 (socket says 172.21.15.90:49032) 2024-04-24T09:19:35.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.478+0000 7f8ad3e00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8adc06b1a0 0x7f8adc19ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.90:56440/0 (socket says 172.21.15.90:56440) 2024-04-24T09:19:35.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.478+0000 7f8ae0c00700 1 -- 172.21.15.90:0/3777920715 learned_addr learned my addr 172.21.15.90:0/3777920715 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:19:35.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.478+0000 7f8ae1600700 1 --2- 172.21.15.90:0/3777920715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8adc108b70 0x7f8adc19fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:35.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.478+0000 7f8ad3e00700 1 -- 172.21.15.90:0/3777920715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8adc108b70 msgr2=0x7f8adc19fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:35.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.478+0000 7f8ad3e00700 1 --2- 172.21.15.90:0/3777920715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8adc108b70 0x7f8adc19fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:35.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.478+0000 7f8ad3e00700 1 -- 172.21.15.90:0/3777920715 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8adc06a800 msgr2=0x7f8adc19b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:35.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.478+0000 7f8ad3e00700 1 --2- 172.21.15.90:0/3777920715 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8adc06a800 0x7f8adc19b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:35.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.478+0000 7f8ad3e00700 1 -- 172.21.15.90:0/3777920715 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8adc1a0520 con 0x7f8adc06b1a0 2024-04-24T09:19:35.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.478+0000 7f8ae1600700 1 --2- 172.21.15.90:0/3777920715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8adc108b70 0x7f8adc19fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:19:35.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.478+0000 7f8ae0c00700 1 --2- 172.21.15.90:0/3777920715 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8adc06a800 0x7f8adc19b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:19:35.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.479+0000 7f8ad3e00700 1 --2- 172.21.15.90:0/3777920715 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8adc06b1a0 0x7f8adc19ba50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8acc00e9f0 tx=0x7f8acc00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:35.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.479+0000 7f8ada000700 1 -- 172.21.15.90:0/3777920715 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8acc00cb70 con 0x7f8adc06b1a0 2024-04-24T09:19:35.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.479+0000 7f8ae3600700 1 -- 172.21.15.90:0/3777920715 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8adc1a0810 con 0x7f8adc06b1a0 2024-04-24T09:19:35.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.479+0000 7f8ada000700 1 -- 172.21.15.90:0/3777920715 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8acc004500 con 0x7f8adc06b1a0 2024-04-24T09:19:35.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.479+0000 7f8ada000700 1 -- 172.21.15.90:0/3777920715 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8acc010430 con 0x7f8adc06b1a0 2024-04-24T09:19:35.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.479+0000 7f8ae3600700 1 -- 172.21.15.90:0/3777920715 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8adc1a8090 con 0x7f8adc06b1a0 2024-04-24T09:19:35.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.480+0000 7f8ae3600700 1 -- 172.21.15.90:0/3777920715 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8adc0697e0 con 0x7f8adc06b1a0 2024-04-24T09:19:35.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.482+0000 7f8ada000700 1 -- 172.21.15.90:0/3777920715 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8acc003eb0 con 0x7f8adc06b1a0 2024-04-24T09:19:35.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.483+0000 7f8ada000700 1 --2- 172.21.15.90:0/3777920715 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f8abc0a3cd0 0x7f8abc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:35.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.484+0000 7f8ada000700 1 -- 172.21.15.90:0/3777920715 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f8acc0d0690 con 0x7f8adc06b1a0 2024-04-24T09:19:35.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.484+0000 7f8ae0c00700 1 --2- 172.21.15.90:0/3777920715 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f8abc0a3cd0 0x7f8abc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:35.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.484+0000 7f8ae0c00700 1 --2- 172.21.15.90:0/3777920715 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f8abc0a3cd0 0x7f8abc0a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009750 tx=0x7f8ac400b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:35.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.488+0000 7f8ada000700 1 -- 172.21.15.90:0/3777920715 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8acc09cfc0 con 0x7f8adc06b1a0 2024-04-24T09:19:35.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.713+0000 7f8ae3600700 1 -- 172.21.15.90:0/3777920715 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8adc19c8a0 con 0x7f8adc06b1a0 2024-04-24T09:19:35.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.715+0000 7f8ada000700 1 -- 172.21.15.90:0/3777920715 <== mon.2 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22231 (secure 0 0 0) 0x7f8adc19c8a0 con 0x7f8adc06b1a0 2024-04-24T09:19:35.716 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:19:35.716 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":86,"fsid":"30ee9be2-021a-11ef-bc93-c7b262605968","created":"2024-04-24T09:09:49.496019+0000","modified":"2024-04-24T09:19:10.816601+0000","last_up_change":"2024-04-24T09:18:10.455632+0000","last_in_change":"2024-04-24T09:17:50.600366+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T09:13:45.548774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T09:19:07.185580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T09:19:08.374818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13a59b9f-b46e-4ec4-90af-bb7a3d5c753b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6803","nonce":2864288066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6805","nonce":2864288066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6809","nonce":2864288066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6807","nonce":2864288066}]},"public_addr":"172.21.15.90:6803/2864288066","cluster_addr":"172.21.15.90:6805/2864288066","heartbeat_back_addr":"172.21.15.90:6809/2864288066","heartbeat_front_addr":"172.21.15.90:6807/2864288066","state":["exists","up"]},{"osd":1,"uuid":"1567ce8f-24e6-42cf-bf64-f6f0c534a9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6801","nonce":344222414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6803","nonce":344222414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6807","nonce":344222414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6805","nonce":344222414}]},"public_addr":"172.21.15.114:6801/344222414","cluster_addr":"172.21.15.114:6803/344222414","heartbeat_back_addr":"172.21.15.114:6807/344222414","heartbeat_front_addr":"172.21.15.114:6805/344222414","state":["exists","up"]},{"osd":2,"uuid":"f861b304-9c7d-4629-8522-37f33a747945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6801","nonce":3257067419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6803","nonce":3257067419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6807","nonce":3257067419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6805","nonce":3257067419}]},"public_addr":"172.21.15.149:6801/3257067419","cluster_addr":"172.21.15.149:6803/3257067419","heartbeat_back_addr":"172.21.15.149:6807/3257067419","heartbeat_front_addr":"172.21.15.149:6805/3257067419","state":["exists","up"]},{"osd":3,"uuid":"9377ab6e-5996-45fb-93a9-475a2e5f17a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6811","nonce":3068524970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6813","nonce":3068524970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6817","nonce":3068524970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6815","nonce":3068524970}]},"public_addr":"172.21.15.90:6811/3068524970","cluster_addr":"172.21.15.90:6813/3068524970","heartbeat_back_addr":"172.21.15.90:6817/3068524970","heartbeat_front_addr":"172.21.15.90:6815/3068524970","state":["exists","up"]},{"osd":4,"uuid":"e2ddd19e-e641-4d61-b342-02b5f12b1c4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6809","nonce":2782015522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6811","nonce":2782015522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6815","nonce":2782015522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6813","nonce":2782015522}]},"public_addr":"172.21.15.114:6809/2782015522","cluster_addr":"172.21.15.114:6811/2782015522","heartbeat_back_addr":"172.21.15.114:6815/2782015522","heartbeat_front_addr":"172.21.15.114:6813/2782015522","state":["exists","up"]},{"osd":5,"uuid":"b6ff9d10-5210-4b46-a192-6dcbf9d1f156","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6809","nonce":4266460479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6811","nonce":4266460479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6815","nonce":4266460479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6813","nonce":4266460479}]},"public_addr":"172.21.15.149:6809/4266460479","cluster_addr":"172.21.15.149:6811/4266460479","heartbeat_back_addr":"172.21.15.149:6815/4266460479","heartbeat_front_addr":"172.21.15.149:6813/4266460479","state":["exists","up"]},{"osd":6,"uuid":"4287b5a1-f953-421c-bf62-5e390bee834d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6819","nonce":1121461276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6821","nonce":1121461276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6825","nonce":1121461276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6823","nonce":1121461276}]},"public_addr":"172.21.15.90:6819/1121461276","cluster_addr":"172.21.15.90:6821/1121461276","heartbeat_back_addr":"172.21.15.90:6825/1121461276","heartbeat_front_addr":"172.21.15.90:6823/1121461276","state":["exists","up"]},{"osd":7,"uuid":"a54e507d-d6ba-4bf2-9a7f-a7b95c3859bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6817","nonce":3222546280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6819","nonce":3222546280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6823","nonce":3222546280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6821","nonce":3222546280}]},"public_addr":"172.21.15.114:6817/3222546280","cluster_addr":"172.21.15.114:6819/3222546280","heartbeat_back_addr":"172.21.15.114:6823/3222546280","heartbeat_front_addr":"172.21.15.114:6821/3222546280","state":["exists","up"]},{"osd":8,"uuid":"9950b010-5865-4109-ae37-582b1dbd2f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6817","nonce":157615149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6819","nonce":157615149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6823","nonce":157615149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6821","nonce":157615149}]},"public_addr":"172.21.15.149:6817/157615149","cluster_addr":"172.21.15.149:6819/157615149","heartbeat_back_addr":"172.21.15.149:6823/157615149","heartbeat_front_addr":"172.21.15.149:6821/157615149","state":["exists","up"]},{"osd":9,"uuid":"31e2635c-e234-421f-9757-592fd673c936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6827","nonce":1307318076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6829","nonce":1307318076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6833","nonce":1307318076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6831","nonce":1307318076}]},"public_addr":"172.21.15.90:6827/1307318076","cluster_addr":"172.21.15.90:6829/1307318076","heartbeat_back_addr":"172.21.15.90:6833/1307318076","heartbeat_front_addr":"172.21.15.90:6831/1307318076","state":["exists","up"]},{"osd":10,"uuid":"46a3b2a6-e6e5-4578-b4bb-f3e998b9bc95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6825","nonce":1719802140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6827","nonce":1719802140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6831","nonce":1719802140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6829","nonce":1719802140}]},"public_addr":"172.21.15.114:6825/1719802140","cluster_addr":"172.21.15.114:6827/1719802140","heartbeat_back_addr":"172.21.15.114:6831/1719802140","heartbeat_front_addr":"172.21.15.114:6829/1719802140","state":["exists","up"]},{"osd":11,"uuid":"ccd4456b-e947-46cf-a779-148ffa289b58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6825","nonce":1737275166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6827","nonce":1737275166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6831","nonce":1737275166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6829","nonce":1737275166}]},"public_addr":"172.21.15.149:6825/1737275166","cluster_addr":"172.21.15.149:6827/1737275166","heartbeat_back_addr":"172.21.15.149:6831/1737275166","heartbeat_front_addr":"172.21.15.149:6829/1737275166","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:12:53.015078+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:13:18.047053+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:13:41.935891+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:14:15.095388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:14:42.512355+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:15:08.113775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:15:43.828550+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:16:10.901909+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:16:36.422962+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:17:13.924446+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:17:42.814401+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:18:08.672793+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/3090852580":"2024-04-25T09:10:57.377655+0000","172.21.15.90:0/687197394":"2024-04-25T09:10:57.377655+0000","172.21.15.90:6801/1809038230":"2024-04-25T09:10:57.377655+0000","172.21.15.90:6800/1809038230":"2024-04-25T09:10:57.377655+0000","172.21.15.90:0/2964149071":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/3617388113":"2024-04-25T09:10:22.599674+0000","172.21.15.90:6801/3089140842":"2024-04-25T09:10:22.599674+0000","172.21.15.90:6800/3089140842":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/2840416149":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/305432710":"2024-04-25T09:10:57.377655+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T09:19:35.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.719+0000 7f8ae3600700 1 -- 172.21.15.90:0/3777920715 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f8abc0a3cd0 msgr2=0x7f8abc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:35.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.719+0000 7f8ae3600700 1 --2- 172.21.15.90:0/3777920715 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f8abc0a3cd0 0x7f8abc0a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009750 tx=0x7f8ac400b280 comp rx=0 tx=0).stop 2024-04-24T09:19:35.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.719+0000 7f8ae3600700 1 -- 172.21.15.90:0/3777920715 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8adc06b1a0 msgr2=0x7f8adc19ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:35.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.719+0000 7f8ae3600700 1 --2- 172.21.15.90:0/3777920715 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8adc06b1a0 0x7f8adc19ba50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8acc00e9f0 tx=0x7f8acc00ed00 comp rx=0 tx=0).stop 2024-04-24T09:19:35.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.719+0000 7f8ae3600700 1 -- 172.21.15.90:0/3777920715 shutdown_connections 2024-04-24T09:19:35.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.719+0000 7f8ae3600700 1 --2- 172.21.15.90:0/3777920715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8adc108b70 0x7f8adc19fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:35.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.719+0000 7f8ae3600700 1 --2- 172.21.15.90:0/3777920715 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f8abc0a3cd0 0x7f8abc0a6190 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:35.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.719+0000 7f8ae3600700 1 --2- 172.21.15.90:0/3777920715 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8adc06a800 0x7f8adc19b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:35.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.719+0000 7f8ae3600700 1 --2- 172.21.15.90:0/3777920715 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8adc06b1a0 0x7f8adc19ba50 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:35.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.719+0000 7f8ae3600700 1 -- 172.21.15.90:0/3777920715 >> 172.21.15.90:0/3777920715 conn(0x7f8adc0789a0 msgr2=0x7f8adc10dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:35.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.719+0000 7f8ae3600700 1 -- 172.21.15.90:0/3777920715 shutdown_connections 2024-04-24T09:19:35.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:35.719+0000 7f8ae3600700 1 -- 172.21.15.90:0/3777920715 wait complete. 2024-04-24T09:19:36.686 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-24T09:19:36.687 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-24T09:19:36.687 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-24T09:19:36.688 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-24T09:19:36.688 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-24T09:19:36.688 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-24T09:19:36.689 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-24T09:19:36.689 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-24T09:19:36.690 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-24T09:19:36.690 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-24T09:19:36.690 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-24T09:19:36.691 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-24T09:19:36.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:36 smithi149 ceph-mon[32592]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-24T09:19:36.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:36 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/3777920715' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:19:36.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:36 smithi090 ceph-mon[27825]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-24T09:19:36.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:36 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/3777920715' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:19:36.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:36 smithi114 ceph-mon[32501]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-24T09:19:36.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:36 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/3777920715' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:19:37.691 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:19:37.719 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:19:37.728 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:19:38.735 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:38 smithi090 ceph-mon[27825]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-24T09:19:38.925 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:38 smithi114 ceph-mon[32501]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-24T09:19:39.049 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:19:39.050 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:19:39.051 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:19:39.054 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:19:39.057 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:19:39.059 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:19:39.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:38 smithi149 ceph-mon[32592]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-24T09:19:39.190 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:19:39.542 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:19:40.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:40 smithi090 ceph-mon[27825]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-24T09:19:40.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:40 smithi114 ceph-mon[32501]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-24T09:19:41.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:40 smithi149 ceph-mon[32592]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-24T09:19:42.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:42 smithi090 ceph-mon[27825]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-24T09:19:42.914 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:42 smithi149 ceph-mon[32592]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-24T09:19:42.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:42 smithi114 ceph-mon[32501]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-24T09:19:44.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:44 smithi114 ceph-mon[32501]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-24T09:19:45.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:44 smithi090 ceph-mon[27825]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-24T09:19:45.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:44 smithi149 ceph-mon[32592]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-24T09:19:45.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.612+0000 7f0b21800700 1 -- 172.21.15.90:0/4042075204 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b1c10a070 msgr2=0x7f0b1c111c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:45.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.612+0000 7f0b21800700 1 --2- 172.21.15.90:0/4042075204 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b1c10a070 0x7f0b1c111c40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0b1400b0a0 tx=0x7f0b1400b3b0 comp rx=0 tx=0).stop 2024-04-24T09:19:45.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.613+0000 7f0b21800700 1 -- 172.21.15.90:0/4042075204 shutdown_connections 2024-04-24T09:19:45.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.613+0000 7f0b21800700 1 --2- 172.21.15.90:0/4042075204 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b1c10a070 0x7f0b1c111c40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:45.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.613+0000 7f0b21800700 1 --2- 172.21.15.90:0/4042075204 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b1c104f90 0x7f0b1c105370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:45.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.613+0000 7f0b21800700 1 --2- 172.21.15.90:0/4042075204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0b1c105940 0x7f0b1c109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:45.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.613+0000 7f0b21800700 1 -- 172.21.15.90:0/4042075204 >> 172.21.15.90:0/4042075204 conn(0x7f0b1c100820 msgr2=0x7f0b1c102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:45.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.613+0000 7f0b21800700 1 -- 172.21.15.90:0/4042075204 shutdown_connections 2024-04-24T09:19:45.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.613+0000 7f0b21800700 1 -- 172.21.15.90:0/4042075204 wait complete. 2024-04-24T09:19:45.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.614+0000 7f0b21800700 1 Processor -- start 2024-04-24T09:19:45.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.614+0000 7f0b21800700 1 -- start start 2024-04-24T09:19:45.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.615+0000 7f0b21800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b1c104f90 0x7f0b1c19b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:45.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.615+0000 7f0b21800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0b1c105940 0x7f0b1c19b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:45.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.615+0000 7f0b21800700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b1c10a070 0x7f0b1c19fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:45.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.615+0000 7f0b21800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1c1150b0 con 0x7f0b1c104f90 2024-04-24T09:19:45.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.615+0000 7f0b21800700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1c114f30 con 0x7f0b1c10a070 2024-04-24T09:19:45.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.615+0000 7f0b21800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1c115230 con 0x7f0b1c105940 2024-04-24T09:19:45.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.615+0000 7f0b1aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b1c104f90 0x7f0b1c19b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:45.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.615+0000 7f0b1aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b1c104f90 0x7f0b1c19b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38474/0 (socket says 172.21.15.90:38474) 2024-04-24T09:19:45.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.615+0000 7f0b1aa00700 1 -- 172.21.15.90:0/3266096779 learned_addr learned my addr 172.21.15.90:0/3266096779 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:19:45.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.615+0000 7f0b1a000700 1 --2- 172.21.15.90:0/3266096779 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0b1c105940 0x7f0b1c19b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:45.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.615+0000 7f0b1b400700 1 --2- 172.21.15.90:0/3266096779 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b1c10a070 0x7f0b1c19fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:45.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.616+0000 7f0b1aa00700 1 -- 172.21.15.90:0/3266096779 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0b1c105940 msgr2=0x7f0b1c19b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:45.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.616+0000 7f0b1aa00700 1 --2- 172.21.15.90:0/3266096779 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0b1c105940 0x7f0b1c19b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:45.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.616+0000 7f0b1aa00700 1 -- 172.21.15.90:0/3266096779 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b1c10a070 msgr2=0x7f0b1c19fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:45.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.616+0000 7f0b1aa00700 1 --2- 172.21.15.90:0/3266096779 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b1c10a070 0x7f0b1c19fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:45.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.616+0000 7f0b1aa00700 1 -- 172.21.15.90:0/3266096779 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b1c1a0490 con 0x7f0b1c104f90 2024-04-24T09:19:45.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.616+0000 7f0b1aa00700 1 --2- 172.21.15.90:0/3266096779 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b1c104f90 0x7f0b1c19b480 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c00d950 tx=0x7f0b0c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:45.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.616+0000 7f0b13400700 1 -- 172.21.15.90:0/3266096779 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c016070 con 0x7f0b1c104f90 2024-04-24T09:19:45.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.616+0000 7f0b13400700 1 -- 172.21.15.90:0/3266096779 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b0c00fd70 con 0x7f0b1c104f90 2024-04-24T09:19:45.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.616+0000 7f0b21800700 1 -- 172.21.15.90:0/3266096779 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b1c1a0780 con 0x7f0b1c104f90 2024-04-24T09:19:45.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.616+0000 7f0b13400700 1 -- 172.21.15.90:0/3266096779 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c00e610 con 0x7f0b1c104f90 2024-04-24T09:19:45.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.617+0000 7f0b21800700 1 -- 172.21.15.90:0/3266096779 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b1c1a8000 con 0x7f0b1c104f90 2024-04-24T09:19:45.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.617+0000 7f0b21800700 1 -- 172.21.15.90:0/3266096779 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0b1c063280 con 0x7f0b1c104f90 2024-04-24T09:19:45.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.620+0000 7f0b13400700 1 -- 172.21.15.90:0/3266096779 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0b0c019070 con 0x7f0b1c104f90 2024-04-24T09:19:45.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.621+0000 7f0b13400700 1 --2- 172.21.15.90:0/3266096779 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f0b000a3cd0 0x7f0b000a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:45.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.621+0000 7f0b13400700 1 -- 172.21.15.90:0/3266096779 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f0b0c0d0490 con 0x7f0b1c104f90 2024-04-24T09:19:45.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.621+0000 7f0b1a000700 1 --2- 172.21.15.90:0/3266096779 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f0b000a3cd0 0x7f0b000a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:45.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.622+0000 7f0b13400700 1 --2- 172.21.15.90:0/3266096779 >> [v2:172.21.15.149:6808/4266460479,v1:172.21.15.149:6809/4266460479] conn(0x7f0b000ad4b0 0x7f0b000af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:45.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.622+0000 7f0b13400700 1 -- 172.21.15.90:0/3266096779 --> [v2:172.21.15.149:6808/4266460479,v1:172.21.15.149:6809/4266460479] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0b0c024ea0 con 0x7f0b000ad4b0 2024-04-24T09:19:45.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.622+0000 7f0b13400700 1 -- 172.21.15.90:0/3266096779 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f0b0c0d0840 con 0x7f0b1c104f90 2024-04-24T09:19:45.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.622+0000 7f0b1b400700 1 --2- 172.21.15.90:0/3266096779 >> [v2:172.21.15.149:6808/4266460479,v1:172.21.15.149:6809/4266460479] conn(0x7f0b000ad4b0 0x7f0b000af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:45.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.622+0000 7f0b1a000700 1 --2- 172.21.15.90:0/3266096779 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f0b000a3cd0 0x7f0b000a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c10b5a0 tx=0x7f0b08009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:45.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.623+0000 7f0b1b400700 1 --2- 172.21.15.90:0/3266096779 >> [v2:172.21.15.149:6808/4266460479,v1:172.21.15.149:6809/4266460479] conn(0x7f0b000ad4b0 0x7f0b000af8d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:45.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.624+0000 7f0b13400700 1 -- 172.21.15.90:0/3266096779 <== osd.5 v2:172.21.15.149:6808/4266460479 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0b0c024ea0 con 0x7f0b000ad4b0 2024-04-24T09:19:45.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.643+0000 7f0b21800700 1 -- 172.21.15.90:0/3266096779 --> [v2:172.21.15.149:6808/4266460479,v1:172.21.15.149:6809/4266460479] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0b1c063680 con 0x7f0b000ad4b0 2024-04-24T09:19:45.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.643+0000 7f0b13400700 1 -- 172.21.15.90:0/3266096779 <== osd.5 v2:172.21.15.149:6808/4266460479 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0b1c063680 con 0x7f0b000ad4b0 2024-04-24T09:19:45.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.644+0000 7f0b21800700 1 -- 172.21.15.90:0/3266096779 >> [v2:172.21.15.149:6808/4266460479,v1:172.21.15.149:6809/4266460479] conn(0x7f0b000ad4b0 msgr2=0x7f0b000af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:45.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.644+0000 7f0b21800700 1 --2- 172.21.15.90:0/3266096779 >> [v2:172.21.15.149:6808/4266460479,v1:172.21.15.149:6809/4266460479] conn(0x7f0b000ad4b0 0x7f0b000af8d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:45.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.644+0000 7f0b21800700 1 -- 172.21.15.90:0/3266096779 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f0b000a3cd0 msgr2=0x7f0b000a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:45.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.644+0000 7f0b21800700 1 --2- 172.21.15.90:0/3266096779 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f0b000a3cd0 0x7f0b000a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c10b5a0 tx=0x7f0b08009260 comp rx=0 tx=0).stop 2024-04-24T09:19:45.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.644+0000 7f0b21800700 1 -- 172.21.15.90:0/3266096779 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b1c104f90 msgr2=0x7f0b1c19b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:45.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.644+0000 7f0b21800700 1 --2- 172.21.15.90:0/3266096779 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b1c104f90 0x7f0b1c19b480 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c00d950 tx=0x7f0b0c00dc60 comp rx=0 tx=0).stop 2024-04-24T09:19:45.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.644+0000 7f0b1b400700 1 -- 172.21.15.90:0/3266096779 reap_dead start 2024-04-24T09:19:45.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.644+0000 7f0b21800700 1 -- 172.21.15.90:0/3266096779 shutdown_connections 2024-04-24T09:19:45.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.644+0000 7f0b21800700 1 -- 172.21.15.90:0/3266096779 >> 172.21.15.90:0/3266096779 conn(0x7f0b1c100820 msgr2=0x7f0b1c1012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:45.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.644+0000 7f0b21800700 1 -- 172.21.15.90:0/3266096779 shutdown_connections 2024-04-24T09:19:45.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:45.645+0000 7f0b21800700 1 -- 172.21.15.90:0/3266096779 wait complete. 2024-04-24T09:19:46.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:46 smithi090 ceph-mon[27825]: pgmap v340: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-24T09:19:46.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:46 smithi114 ceph-mon[32501]: pgmap v340: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-24T09:19:47.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:46 smithi149 ceph-mon[32592]: pgmap v340: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-24T09:19:48.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:48 smithi090 ceph-mon[27825]: pgmap v341: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-24T09:19:48.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:48 smithi114 ceph-mon[32501]: pgmap v341: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-24T09:19:49.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:48 smithi149 ceph-mon[32592]: pgmap v341: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-24T09:19:50.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:50 smithi090 ceph-mon[27825]: pgmap v342: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-24T09:19:50.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:50 smithi114 ceph-mon[32501]: pgmap v342: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-24T09:19:51.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:50 smithi149 ceph-mon[32592]: pgmap v342: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-24T09:19:52.577 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:19:52.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:52 smithi090 ceph-mon[27825]: pgmap v343: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-24T09:19:52.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:52 smithi149 ceph-mon[32592]: pgmap v343: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-24T09:19:52.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:52 smithi114 ceph-mon[32501]: pgmap v343: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-24T09:19:53.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.588+0000 7f3ba3e00700 1 -- 172.21.15.90:0/172504143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ba410d010 msgr2=0x7f3ba410f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:53.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.588+0000 7f3ba3e00700 1 --2- 172.21.15.90:0/172504143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ba410d010 0x7f3ba410f4d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c00b0a0 tx=0x7f3b9c00b3b0 comp rx=0 tx=0).stop 2024-04-24T09:19:53.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.589+0000 7f3ba3e00700 1 -- 172.21.15.90:0/172504143 shutdown_connections 2024-04-24T09:19:53.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.589+0000 7f3ba3e00700 1 --2- 172.21.15.90:0/172504143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ba410d010 0x7f3ba410f4d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:53.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.589+0000 7f3ba3e00700 1 --2- 172.21.15.90:0/172504143 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ba4103be0 0x7f3ba4103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:53.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.589+0000 7f3ba3e00700 1 --2- 172.21.15.90:0/172504143 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ba4104500 0x7f3ba410c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:53.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.589+0000 7f3ba3e00700 1 -- 172.21.15.90:0/172504143 >> 172.21.15.90:0/172504143 conn(0x7f3ba40fd410 msgr2=0x7f3ba40ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:53.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.589+0000 7f3ba3e00700 1 -- 172.21.15.90:0/172504143 shutdown_connections 2024-04-24T09:19:53.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.589+0000 7f3ba3e00700 1 -- 172.21.15.90:0/172504143 wait complete. 2024-04-24T09:19:53.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.590+0000 7f3ba3e00700 1 Processor -- start 2024-04-24T09:19:53.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.590+0000 7f3ba3e00700 1 -- start start 2024-04-24T09:19:53.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.591+0000 7f3ba3e00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ba4103be0 0x7f3ba419b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:53.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.591+0000 7f3ba3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ba4104500 0x7f3ba419b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:53.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.591+0000 7f3ba3e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ba410d010 0x7f3ba419fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:53.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.591+0000 7f3ba3e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba4112d70 con 0x7f3ba4104500 2024-04-24T09:19:53.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.591+0000 7f3ba3e00700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba4112bf0 con 0x7f3ba4103be0 2024-04-24T09:19:53.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.591+0000 7f3ba3e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba4112ef0 con 0x7f3ba410d010 2024-04-24T09:19:53.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.591+0000 7f3ba2a00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ba4103be0 0x7f3ba419b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:53.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.591+0000 7f3ba2000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ba4104500 0x7f3ba419b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:53.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.591+0000 7f3ba3400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ba410d010 0x7f3ba419fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:53.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.591+0000 7f3ba2a00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ba4103be0 0x7f3ba419b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.90:34910/0 (socket says 172.21.15.90:34910) 2024-04-24T09:19:53.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.591+0000 7f3ba2a00700 1 -- 172.21.15.90:0/2185840186 learned_addr learned my addr 172.21.15.90:0/2185840186 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:19:53.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.592+0000 7f3ba3400700 1 -- 172.21.15.90:0/2185840186 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ba4103be0 msgr2=0x7f3ba419b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:53.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.592+0000 7f3ba3400700 1 --2- 172.21.15.90:0/2185840186 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ba4103be0 0x7f3ba419b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:53.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.592+0000 7f3ba3400700 1 -- 172.21.15.90:0/2185840186 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ba4104500 msgr2=0x7f3ba419b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:53.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.592+0000 7f3ba3400700 1 --2- 172.21.15.90:0/2185840186 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ba4104500 0x7f3ba419b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:53.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.592+0000 7f3ba3400700 1 -- 172.21.15.90:0/2185840186 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ba41a0360 con 0x7f3ba410d010 2024-04-24T09:19:53.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.592+0000 7f3ba2a00700 1 --2- 172.21.15.90:0/2185840186 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ba4103be0 0x7f3ba419b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:19:53.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.592+0000 7f3ba2000700 1 --2- 172.21.15.90:0/2185840186 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ba4104500 0x7f3ba419b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:19:53.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.592+0000 7f3ba3400700 1 --2- 172.21.15.90:0/2185840186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ba410d010 0x7f3ba419fc20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c00b0a0 tx=0x7f3b9c004d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:53.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.592+0000 7f3ba8a00700 1 -- 172.21.15.90:0/2185840186 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c021070 con 0x7f3ba410d010 2024-04-24T09:19:53.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.592+0000 7f3ba8a00700 1 -- 172.21.15.90:0/2185840186 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b9c007b80 con 0x7f3ba410d010 2024-04-24T09:19:53.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.592+0000 7f3ba8a00700 1 -- 172.21.15.90:0/2185840186 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c005040 con 0x7f3ba410d010 2024-04-24T09:19:53.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.592+0000 7f3ba3e00700 1 -- 172.21.15.90:0/2185840186 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ba41a05f0 con 0x7f3ba410d010 2024-04-24T09:19:53.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.593+0000 7f3ba3e00700 1 -- 172.21.15.90:0/2185840186 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ba41a7ed0 con 0x7f3ba410d010 2024-04-24T09:19:53.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.593+0000 7f3ba3e00700 1 -- 172.21.15.90:0/2185840186 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3ba4103290 con 0x7f3ba410d010 2024-04-24T09:19:53.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.595+0000 7f3ba8a00700 1 -- 172.21.15.90:0/2185840186 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3b9c024070 con 0x7f3ba410d010 2024-04-24T09:19:53.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.597+0000 7f3ba8a00700 1 --2- 172.21.15.90:0/2185840186 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f3b880a3d50 0x7f3b880a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:53.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.597+0000 7f3ba8a00700 1 -- 172.21.15.90:0/2185840186 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f3b9c00f020 con 0x7f3ba410d010 2024-04-24T09:19:53.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.597+0000 7f3ba2a00700 1 --2- 172.21.15.90:0/2185840186 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f3b880a3d50 0x7f3b880a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:53.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.597+0000 7f3ba8a00700 1 --2- 172.21.15.90:0/2185840186 >> [v2:172.21.15.149:6800/3257067419,v1:172.21.15.149:6801/3257067419] conn(0x7f3b880ad530 0x7f3b880af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:53.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.597+0000 7f3ba8a00700 1 -- 172.21.15.90:0/2185840186 --> [v2:172.21.15.149:6800/3257067419,v1:172.21.15.149:6801/3257067419] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3b880b0000 con 0x7f3b880ad530 2024-04-24T09:19:53.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.597+0000 7f3ba8a00700 1 -- 172.21.15.90:0/2185840186 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f3b9c0d8050 con 0x7f3ba410d010 2024-04-24T09:19:53.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.598+0000 7f3ba2000700 1 --2- 172.21.15.90:0/2185840186 >> [v2:172.21.15.149:6800/3257067419,v1:172.21.15.149:6801/3257067419] conn(0x7f3b880ad530 0x7f3b880af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:53.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.598+0000 7f3ba2a00700 1 --2- 172.21.15.90:0/2185840186 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f3b880a3d50 0x7f3b880a6210 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3b94011590 tx=0x7f3b94009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:53.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.598+0000 7f3ba2000700 1 --2- 172.21.15.90:0/2185840186 >> [v2:172.21.15.149:6800/3257067419,v1:172.21.15.149:6801/3257067419] conn(0x7f3b880ad530 0x7f3b880af950 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:53.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.599+0000 7f3ba8a00700 1 -- 172.21.15.90:0/2185840186 <== osd.2 v2:172.21.15.149:6800/3257067419 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3b880b0000 con 0x7f3b880ad530 2024-04-24T09:19:53.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.618+0000 7f3ba3e00700 1 -- 172.21.15.90:0/2185840186 --> [v2:172.21.15.149:6800/3257067419,v1:172.21.15.149:6801/3257067419] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3ba406b0b0 con 0x7f3b880ad530 2024-04-24T09:19:53.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.618+0000 7f3ba8a00700 1 -- 172.21.15.90:0/2185840186 <== osd.2 v2:172.21.15.149:6800/3257067419 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3ba406b0b0 con 0x7f3b880ad530 2024-04-24T09:19:53.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.619+0000 7f3ba3e00700 1 -- 172.21.15.90:0/2185840186 >> [v2:172.21.15.149:6800/3257067419,v1:172.21.15.149:6801/3257067419] conn(0x7f3b880ad530 msgr2=0x7f3b880af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:53.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.619+0000 7f3ba3e00700 1 --2- 172.21.15.90:0/2185840186 >> [v2:172.21.15.149:6800/3257067419,v1:172.21.15.149:6801/3257067419] conn(0x7f3b880ad530 0x7f3b880af950 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:53.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.619+0000 7f3ba3e00700 1 -- 172.21.15.90:0/2185840186 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f3b880a3d50 msgr2=0x7f3b880a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:53.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.619+0000 7f3ba3e00700 1 --2- 172.21.15.90:0/2185840186 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f3b880a3d50 0x7f3b880a6210 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3b94011590 tx=0x7f3b94009260 comp rx=0 tx=0).stop 2024-04-24T09:19:53.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.619+0000 7f3ba3e00700 1 -- 172.21.15.90:0/2185840186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ba410d010 msgr2=0x7f3ba419fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:53.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.619+0000 7f3ba3e00700 1 --2- 172.21.15.90:0/2185840186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ba410d010 0x7f3ba419fc20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c00b0a0 tx=0x7f3b9c004d00 comp rx=0 tx=0).stop 2024-04-24T09:19:53.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.619+0000 7f3ba3400700 1 -- 172.21.15.90:0/2185840186 reap_dead start 2024-04-24T09:19:53.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.619+0000 7f3ba3e00700 1 -- 172.21.15.90:0/2185840186 shutdown_connections 2024-04-24T09:19:53.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.619+0000 7f3ba3e00700 1 -- 172.21.15.90:0/2185840186 >> 172.21.15.90:0/2185840186 conn(0x7f3ba40fd410 msgr2=0x7f3ba410f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:53.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.619+0000 7f3ba3e00700 1 -- 172.21.15.90:0/2185840186 shutdown_connections 2024-04-24T09:19:53.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.619+0000 7f3ba3e00700 1 -- 172.21.15.90:0/2185840186 wait complete. 2024-04-24T09:19:53.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.392+0000 7f0916200700 1 -- 172.21.15.90:0/3630688137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0910103500 msgr2=0x7f091010f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:53.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.392+0000 7f0916200700 1 --2- 172.21.15.90:0/3630688137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0910103500 0x7f091010f960 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f090800b0a0 tx=0x7f090800b3b0 comp rx=0 tx=0).stop 2024-04-24T09:19:53.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.392+0000 7f0916200700 1 -- 172.21.15.90:0/3630688137 shutdown_connections 2024-04-24T09:19:53.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.392+0000 7f0916200700 1 --2- 172.21.15.90:0/3630688137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0910103500 0x7f091010f960 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:53.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.392+0000 7f0916200700 1 --2- 172.21.15.90:0/3630688137 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0910108b90 0x7f0910108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:53.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.392+0000 7f0916200700 1 --2- 172.21.15.90:0/3630688137 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0910102b40 0x7f0910102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:53.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.392+0000 7f0916200700 1 -- 172.21.15.90:0/3630688137 >> 172.21.15.90:0/3630688137 conn(0x7f0910079c50 msgr2=0x7f091007a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:53.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.393+0000 7f0916200700 1 -- 172.21.15.90:0/3630688137 shutdown_connections 2024-04-24T09:19:53.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.393+0000 7f0916200700 1 -- 172.21.15.90:0/3630688137 wait complete. 2024-04-24T09:19:53.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.394+0000 7f0916200700 1 Processor -- start 2024-04-24T09:19:53.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.394+0000 7f0916200700 1 -- start start 2024-04-24T09:19:53.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.394+0000 7f0916200700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0910102b40 0x7f091019b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:53.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.394+0000 7f0916200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0910103500 0x7f091019ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:53.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.394+0000 7f0916200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0910108b90 0x7f091019fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:53.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.394+0000 7f0916200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0910112df0 con 0x7f0910108b90 2024-04-24T09:19:53.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.394+0000 7f0916200700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0910112c70 con 0x7f0910102b40 2024-04-24T09:19:53.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.394+0000 7f0916200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0910112f70 con 0x7f0910103500 2024-04-24T09:19:53.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.395+0000 7f090fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0910108b90 0x7f091019fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:53.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.395+0000 7f090ea00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0910103500 0x7f091019ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:53.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.395+0000 7f090f400700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0910102b40 0x7f091019b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:53.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.395+0000 7f090fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0910108b90 0x7f091019fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57812/0 (socket says 172.21.15.90:57812) 2024-04-24T09:19:53.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.395+0000 7f090fe00700 1 -- 172.21.15.90:0/983300097 learned_addr learned my addr 172.21.15.90:0/983300097 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:19:53.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.395+0000 7f090fe00700 1 -- 172.21.15.90:0/983300097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0910103500 msgr2=0x7f091019ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:53.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.395+0000 7f090fe00700 1 --2- 172.21.15.90:0/983300097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0910103500 0x7f091019ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:53.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.395+0000 7f090fe00700 1 -- 172.21.15.90:0/983300097 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0910102b40 msgr2=0x7f091019b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:53.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.395+0000 7f090fe00700 1 --2- 172.21.15.90:0/983300097 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0910102b40 0x7f091019b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:53.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.395+0000 7f090fe00700 1 -- 172.21.15.90:0/983300097 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09101a04e0 con 0x7f0910108b90 2024-04-24T09:19:53.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.395+0000 7f090fe00700 1 --2- 172.21.15.90:0/983300097 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0910108b90 0x7f091019fda0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f090800b780 tx=0x7f0908004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:53.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.396+0000 7f0907e00700 1 -- 172.21.15.90:0/983300097 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0908021070 con 0x7f0910108b90 2024-04-24T09:19:53.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.396+0000 7f0907e00700 1 -- 172.21.15.90:0/983300097 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0908007b80 con 0x7f0910108b90 2024-04-24T09:19:53.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.396+0000 7f0916200700 1 -- 172.21.15.90:0/983300097 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09101a0770 con 0x7f0910108b90 2024-04-24T09:19:53.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.396+0000 7f0907e00700 1 -- 172.21.15.90:0/983300097 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09080052a0 con 0x7f0910108b90 2024-04-24T09:19:53.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.396+0000 7f0916200700 1 -- 172.21.15.90:0/983300097 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09101a8050 con 0x7f0910108b90 2024-04-24T09:19:53.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.397+0000 7f0916200700 1 -- 172.21.15.90:0/983300097 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0910076710 con 0x7f0910108b90 2024-04-24T09:19:53.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.399+0000 7f0907e00700 1 -- 172.21.15.90:0/983300097 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0908024070 con 0x7f0910108b90 2024-04-24T09:19:53.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.401+0000 7f0907e00700 1 --2- 172.21.15.90:0/983300097 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f08f40a3cd0 0x7f08f40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:53.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.401+0000 7f0907e00700 1 -- 172.21.15.90:0/983300097 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f090800f020 con 0x7f0910108b90 2024-04-24T09:19:53.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.401+0000 7f090f400700 1 --2- 172.21.15.90:0/983300097 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f08f40a3cd0 0x7f08f40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:53.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.401+0000 7f0907e00700 1 --2- 172.21.15.90:0/983300097 >> [v2:172.21.15.114:6824/1719802140,v1:172.21.15.114:6825/1719802140] conn(0x7f08f40ad380 0x7f08f40af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:53.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.401+0000 7f0907e00700 1 -- 172.21.15.90:0/983300097 --> [v2:172.21.15.114:6824/1719802140,v1:172.21.15.114:6825/1719802140] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f090801fe80 con 0x7f08f40ad380 2024-04-24T09:19:53.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.401+0000 7f0907e00700 1 -- 172.21.15.90:0/983300097 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f09080d1f20 con 0x7f0910108b90 2024-04-24T09:19:53.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.402+0000 7f090ea00700 1 --2- 172.21.15.90:0/983300097 >> [v2:172.21.15.114:6824/1719802140,v1:172.21.15.114:6825/1719802140] conn(0x7f08f40ad380 0x7f08f40af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:53.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.402+0000 7f090f400700 1 --2- 172.21.15.90:0/983300097 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f08f40a3cd0 0x7f08f40a6190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0900009be0 tx=0x7f0900009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:53.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.402+0000 7f090ea00700 1 --2- 172.21.15.90:0/983300097 >> [v2:172.21.15.114:6824/1719802140,v1:172.21.15.114:6825/1719802140] conn(0x7f08f40ad380 0x7f08f40af7a0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:53.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.403+0000 7f0907e00700 1 -- 172.21.15.90:0/983300097 <== osd.10 v2:172.21.15.114:6824/1719802140 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f090801fe80 con 0x7f08f40ad380 2024-04-24T09:19:53.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.437+0000 7f0916200700 1 -- 172.21.15.90:0/983300097 --> [v2:172.21.15.114:6824/1719802140,v1:172.21.15.114:6825/1719802140] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f09101041f0 con 0x7f08f40ad380 2024-04-24T09:19:53.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.438+0000 7f0907e00700 1 -- 172.21.15.90:0/983300097 <== osd.10 v2:172.21.15.114:6824/1719802140 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f09101041f0 con 0x7f08f40ad380 2024-04-24T09:19:53.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.438+0000 7f0916200700 1 -- 172.21.15.90:0/983300097 >> [v2:172.21.15.114:6824/1719802140,v1:172.21.15.114:6825/1719802140] conn(0x7f08f40ad380 msgr2=0x7f08f40af7a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:53.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.438+0000 7f0916200700 1 --2- 172.21.15.90:0/983300097 >> [v2:172.21.15.114:6824/1719802140,v1:172.21.15.114:6825/1719802140] conn(0x7f08f40ad380 0x7f08f40af7a0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:53.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.438+0000 7f0916200700 1 -- 172.21.15.90:0/983300097 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f08f40a3cd0 msgr2=0x7f08f40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:53.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.438+0000 7f0916200700 1 --2- 172.21.15.90:0/983300097 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f08f40a3cd0 0x7f08f40a6190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0900009be0 tx=0x7f0900009260 comp rx=0 tx=0).stop 2024-04-24T09:19:53.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.438+0000 7f0916200700 1 -- 172.21.15.90:0/983300097 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0910108b90 msgr2=0x7f091019fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:53.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.438+0000 7f0916200700 1 --2- 172.21.15.90:0/983300097 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0910108b90 0x7f091019fda0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f090800b780 tx=0x7f0908004d00 comp rx=0 tx=0).stop 2024-04-24T09:19:53.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.438+0000 7f090fe00700 1 -- 172.21.15.90:0/983300097 reap_dead start 2024-04-24T09:19:53.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.439+0000 7f0916200700 1 -- 172.21.15.90:0/983300097 shutdown_connections 2024-04-24T09:19:53.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.439+0000 7f0916200700 1 -- 172.21.15.90:0/983300097 >> 172.21.15.90:0/983300097 conn(0x7f0910079c50 msgr2=0x7f0910074d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:53.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.439+0000 7f0916200700 1 -- 172.21.15.90:0/983300097 shutdown_connections 2024-04-24T09:19:53.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.439+0000 7f0916200700 1 -- 172.21.15.90:0/983300097 wait complete. 2024-04-24T09:19:54.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.000+0000 7fdcb7200700 1 -- 172.21.15.90:0/3450616410 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdcb8109f60 msgr2=0x7fdcb8111b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:54.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.000+0000 7fdcb7200700 1 --2- 172.21.15.90:0/3450616410 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdcb8109f60 0x7fdcb8111b30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdcb000b0d0 tx=0x7fdcb000b3e0 comp rx=0 tx=0).stop 2024-04-24T09:19:54.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.000+0000 7fdcb7200700 1 -- 172.21.15.90:0/3450616410 shutdown_connections 2024-04-24T09:19:54.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.000+0000 7fdcb7200700 1 --2- 172.21.15.90:0/3450616410 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdcb8109f60 0x7fdcb8111b30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:54.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.000+0000 7fdcb7200700 1 --2- 172.21.15.90:0/3450616410 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdcb8104e80 0x7fdcb8105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:54.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.000+0000 7fdcb7200700 1 --2- 172.21.15.90:0/3450616410 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcb8105830 0x7fdcb8109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:54.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.000+0000 7fdcb7200700 1 -- 172.21.15.90:0/3450616410 >> 172.21.15.90:0/3450616410 conn(0x7fdcb8100710 msgr2=0x7fdcb8102b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:54.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.000+0000 7fdcb7200700 1 -- 172.21.15.90:0/3450616410 shutdown_connections 2024-04-24T09:19:54.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.000+0000 7fdcb7200700 1 -- 172.21.15.90:0/3450616410 wait complete. 2024-04-24T09:19:54.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.001+0000 7fdcb7200700 1 Processor -- start 2024-04-24T09:19:54.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.001+0000 7fdcb7200700 1 -- start start 2024-04-24T09:19:54.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.001+0000 7fdcb7200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdcb8104e80 0x7fdcb819b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:54.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.001+0000 7fdcb7200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcb8105830 0x7fdcb819b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:54.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.002+0000 7fdcb7200700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdcb8109f60 0x7fdcb819fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:54.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.002+0000 7fdcb7200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb8114fa0 con 0x7fdcb8104e80 2024-04-24T09:19:54.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.002+0000 7fdcb7200700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb8114e20 con 0x7fdcb8109f60 2024-04-24T09:19:54.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.002+0000 7fdcb7200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb8115120 con 0x7fdcb8105830 2024-04-24T09:19:54.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.002+0000 7fdcb5e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdcb8104e80 0x7fdcb819b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:54.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.002+0000 7fdcb5e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdcb8104e80 0x7fdcb819b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57790/0 (socket says 172.21.15.90:57790) 2024-04-24T09:19:54.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.002+0000 7fdcb6800700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdcb8109f60 0x7fdcb819fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:54.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.002+0000 7fdcb5e00700 1 -- 172.21.15.90:0/2014039724 learned_addr learned my addr 172.21.15.90:0/2014039724 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:19:54.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.002+0000 7fdcb5400700 1 --2- 172.21.15.90:0/2014039724 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcb8105830 0x7fdcb819b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:54.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.002+0000 7fdcb5e00700 1 -- 172.21.15.90:0/2014039724 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcb8105830 msgr2=0x7fdcb819b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:54.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.002+0000 7fdcb5e00700 1 --2- 172.21.15.90:0/2014039724 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcb8105830 0x7fdcb819b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:54.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.002+0000 7fdcb5e00700 1 -- 172.21.15.90:0/2014039724 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdcb8109f60 msgr2=0x7fdcb819fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:54.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.002+0000 7fdcb5e00700 1 --2- 172.21.15.90:0/2014039724 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdcb8109f60 0x7fdcb819fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:54.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.002+0000 7fdcb5e00700 1 -- 172.21.15.90:0/2014039724 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcb81a0350 con 0x7fdcb8104e80 2024-04-24T09:19:54.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.002+0000 7fdcb6800700 1 --2- 172.21.15.90:0/2014039724 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdcb8109f60 0x7fdcb819fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:19:54.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.003+0000 7fdcb5e00700 1 --2- 172.21.15.90:0/2014039724 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdcb8104e80 0x7fdcb819b390 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdca800e9a0 tx=0x7fdca800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:54.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.003+0000 7fdcaea00700 1 -- 172.21.15.90:0/2014039724 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca800ca10 con 0x7fdcb8104e80 2024-04-24T09:19:54.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.003+0000 7fdcaea00700 1 -- 172.21.15.90:0/2014039724 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdca800cbb0 con 0x7fdcb8104e80 2024-04-24T09:19:54.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.003+0000 7fdcb7200700 1 -- 172.21.15.90:0/2014039724 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcb81a0640 con 0x7fdcb8104e80 2024-04-24T09:19:54.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.003+0000 7fdcaea00700 1 -- 172.21.15.90:0/2014039724 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca8018680 con 0x7fdcb8104e80 2024-04-24T09:19:54.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.003+0000 7fdcb7200700 1 -- 172.21.15.90:0/2014039724 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcb81a7ec0 con 0x7fdcb8104e80 2024-04-24T09:19:54.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.004+0000 7fdcb7200700 1 -- 172.21.15.90:0/2014039724 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdcb8101b50 con 0x7fdcb8104e80 2024-04-24T09:19:54.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.006+0000 7fdcaea00700 1 -- 172.21.15.90:0/2014039724 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fdca8010430 con 0x7fdcb8104e80 2024-04-24T09:19:54.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.008+0000 7fdcaea00700 1 --2- 172.21.15.90:0/2014039724 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fdc9c0a3cd0 0x7fdc9c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:54.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.008+0000 7fdcaea00700 1 -- 172.21.15.90:0/2014039724 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fdca80d06d0 con 0x7fdcb8104e80 2024-04-24T09:19:54.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.008+0000 7fdcb5400700 1 --2- 172.21.15.90:0/2014039724 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fdc9c0a3cd0 0x7fdc9c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:54.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.008+0000 7fdcaea00700 1 --2- 172.21.15.90:0/2014039724 >> [v2:172.21.15.90:6802/2864288066,v1:172.21.15.90:6803/2864288066] conn(0x7fdc9c0ad4b0 0x7fdc9c0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:54.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.008+0000 7fdcaea00700 1 -- 172.21.15.90:0/2014039724 --> [v2:172.21.15.90:6802/2864288066,v1:172.21.15.90:6803/2864288066] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdca8015e80 con 0x7fdc9c0ad4b0 2024-04-24T09:19:54.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.008+0000 7fdcaea00700 1 -- 172.21.15.90:0/2014039724 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fdca80d7050 con 0x7fdcb8104e80 2024-04-24T09:19:54.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.008+0000 7fdcb6800700 1 --2- 172.21.15.90:0/2014039724 >> [v2:172.21.15.90:6802/2864288066,v1:172.21.15.90:6803/2864288066] conn(0x7fdc9c0ad4b0 0x7fdc9c0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:54.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.009+0000 7fdcb5400700 1 --2- 172.21.15.90:0/2014039724 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fdc9c0a3cd0 0x7fdc9c0a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdcb810b490 tx=0x7fdca40073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:54.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.009+0000 7fdcb6800700 1 --2- 172.21.15.90:0/2014039724 >> [v2:172.21.15.90:6802/2864288066,v1:172.21.15.90:6803/2864288066] conn(0x7fdc9c0ad4b0 0x7fdc9c0af8d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:54.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.010+0000 7fdcaea00700 1 -- 172.21.15.90:0/2014039724 <== osd.0 v2:172.21.15.90:6802/2864288066 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdca8015e80 con 0x7fdc9c0ad4b0 2024-04-24T09:19:54.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.028+0000 7fdcb7200700 1 -- 172.21.15.90:0/2014039724 --> [v2:172.21.15.90:6802/2864288066,v1:172.21.15.90:6803/2864288066] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdcb8106360 con 0x7fdc9c0ad4b0 2024-04-24T09:19:54.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.029+0000 7fdcaea00700 1 -- 172.21.15.90:0/2014039724 <== osd.0 v2:172.21.15.90:6802/2864288066 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fdcb8106360 con 0x7fdc9c0ad4b0 2024-04-24T09:19:54.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.029+0000 7fdcb7200700 1 -- 172.21.15.90:0/2014039724 >> [v2:172.21.15.90:6802/2864288066,v1:172.21.15.90:6803/2864288066] conn(0x7fdc9c0ad4b0 msgr2=0x7fdc9c0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:54.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.029+0000 7fdcb7200700 1 --2- 172.21.15.90:0/2014039724 >> [v2:172.21.15.90:6802/2864288066,v1:172.21.15.90:6803/2864288066] conn(0x7fdc9c0ad4b0 0x7fdc9c0af8d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:54.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.029+0000 7fdcb7200700 1 -- 172.21.15.90:0/2014039724 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fdc9c0a3cd0 msgr2=0x7fdc9c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:54.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.029+0000 7fdcb7200700 1 --2- 172.21.15.90:0/2014039724 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fdc9c0a3cd0 0x7fdc9c0a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdcb810b490 tx=0x7fdca40073d0 comp rx=0 tx=0).stop 2024-04-24T09:19:54.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.029+0000 7fdcb7200700 1 -- 172.21.15.90:0/2014039724 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdcb8104e80 msgr2=0x7fdcb819b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:54.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.029+0000 7fdcb7200700 1 --2- 172.21.15.90:0/2014039724 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdcb8104e80 0x7fdcb819b390 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdca800e9a0 tx=0x7fdca800ecb0 comp rx=0 tx=0).stop 2024-04-24T09:19:54.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.029+0000 7fdcb6800700 1 -- 172.21.15.90:0/2014039724 reap_dead start 2024-04-24T09:19:54.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.030+0000 7fdcb7200700 1 -- 172.21.15.90:0/2014039724 shutdown_connections 2024-04-24T09:19:54.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.030+0000 7fdcb7200700 1 -- 172.21.15.90:0/2014039724 >> 172.21.15.90:0/2014039724 conn(0x7fdcb8100710 msgr2=0x7fdcb8101130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:54.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.030+0000 7fdcb7200700 1 -- 172.21.15.90:0/2014039724 shutdown_connections 2024-04-24T09:19:54.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:53.030+0000 7fdcb7200700 1 -- 172.21.15.90:0/2014039724 wait complete. 2024-04-24T09:19:54.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:54 smithi090 ceph-mon[27825]: pgmap v344: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-24T09:19:54.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:54 smithi149 ceph-mon[32592]: pgmap v344: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-24T09:19:54.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:54 smithi114 ceph-mon[32501]: pgmap v344: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-24T09:19:56.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.244+0000 7f59a3200700 1 -- 172.21.15.90:0/2228514518 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f59a4109f40 msgr2=0x7f59a4111b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:56.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.244+0000 7f59a3200700 1 --2- 172.21.15.90:0/2228514518 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f59a4109f40 0x7f59a4111b10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f599c00b0a0 tx=0x7f599c00b3b0 comp rx=0 tx=0).stop 2024-04-24T09:19:56.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.244+0000 7f59a3200700 1 -- 172.21.15.90:0/2228514518 shutdown_connections 2024-04-24T09:19:56.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.244+0000 7f59a3200700 1 --2- 172.21.15.90:0/2228514518 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f59a4109f40 0x7f59a4111b10 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:56.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.244+0000 7f59a3200700 1 --2- 172.21.15.90:0/2228514518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f59a4104e60 0x7f59a4105240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:56.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.244+0000 7f59a3200700 1 --2- 172.21.15.90:0/2228514518 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f59a4105810 0x7f59a4109860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:56.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.244+0000 7f59a3200700 1 -- 172.21.15.90:0/2228514518 >> 172.21.15.90:0/2228514518 conn(0x7f59a4100730 msgr2=0x7f59a4102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:56.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.244+0000 7f59a3200700 1 -- 172.21.15.90:0/2228514518 shutdown_connections 2024-04-24T09:19:56.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.244+0000 7f59a3200700 1 -- 172.21.15.90:0/2228514518 wait complete. 2024-04-24T09:19:56.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.245+0000 7f59a3200700 1 Processor -- start 2024-04-24T09:19:56.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.245+0000 7f59a3200700 1 -- start start 2024-04-24T09:19:56.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.246+0000 7f59a3200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f59a4104e60 0x7f59a419b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:56.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.246+0000 7f59a3200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f59a4105810 0x7f59a419b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:56.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.246+0000 7f59a3200700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f59a4109f40 0x7f59a419fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:56.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.246+0000 7f59a3200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f59a4114fa0 con 0x7f59a4105810 2024-04-24T09:19:56.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.246+0000 7f59a3200700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f59a4114e20 con 0x7f59a4109f40 2024-04-24T09:19:56.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.246+0000 7f59a3200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f59a4115120 con 0x7f59a4104e60 2024-04-24T09:19:56.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.246+0000 7f59a1400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f59a4105810 0x7f59a419b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:56.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.246+0000 7f59a1400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f59a4105810 0x7f59a419b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57768/0 (socket says 172.21.15.90:57768) 2024-04-24T09:19:56.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.246+0000 7f59a1e00700 1 --2- 172.21.15.90:0/3985874060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f59a4104e60 0x7f59a419b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:56.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.246+0000 7f59a2800700 1 --2- 172.21.15.90:0/3985874060 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f59a4109f40 0x7f59a419fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:56.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.246+0000 7f59a1400700 1 -- 172.21.15.90:0/3985874060 learned_addr learned my addr 172.21.15.90:0/3985874060 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:19:56.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.247+0000 7f59a2800700 1 -- 172.21.15.90:0/3985874060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f59a4104e60 msgr2=0x7f59a419b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:56.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.247+0000 7f59a2800700 1 --2- 172.21.15.90:0/3985874060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f59a4104e60 0x7f59a419b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:56.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.247+0000 7f59a2800700 1 -- 172.21.15.90:0/3985874060 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f59a4105810 msgr2=0x7f59a419b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:56.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.247+0000 7f59a2800700 1 --2- 172.21.15.90:0/3985874060 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f59a4105810 0x7f59a419b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:56.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.247+0000 7f59a2800700 1 -- 172.21.15.90:0/3985874060 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59a41a03d0 con 0x7f59a4109f40 2024-04-24T09:19:56.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.247+0000 7f59a1e00700 1 --2- 172.21.15.90:0/3985874060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f59a4104e60 0x7f59a419b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:19:56.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.247+0000 7f59a1400700 1 --2- 172.21.15.90:0/3985874060 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f59a4105810 0x7f59a419b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:19:56.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.247+0000 7f59a2800700 1 --2- 172.21.15.90:0/3985874060 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f59a4109f40 0x7f59a419fc90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f599c0062a0 tx=0x7f599c004c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:56.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.247+0000 7f599aa00700 1 -- 172.21.15.90:0/3985874060 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c021070 con 0x7f59a4109f40 2024-04-24T09:19:56.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.247+0000 7f599aa00700 1 -- 172.21.15.90:0/3985874060 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f599c003c00 con 0x7f59a4109f40 2024-04-24T09:19:56.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.247+0000 7f59a3200700 1 -- 172.21.15.90:0/3985874060 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59a41a0660 con 0x7f59a4109f40 2024-04-24T09:19:56.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.247+0000 7f599aa00700 1 -- 172.21.15.90:0/3985874060 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c0045a0 con 0x7f59a4109f40 2024-04-24T09:19:56.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.247+0000 7f59a3200700 1 -- 172.21.15.90:0/3985874060 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59a41a7f40 con 0x7f59a4109f40 2024-04-24T09:19:56.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.248+0000 7f59a3200700 1 -- 172.21.15.90:0/3985874060 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f59a4063180 con 0x7f59a4109f40 2024-04-24T09:19:56.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.250+0000 7f599aa00700 1 -- 172.21.15.90:0/3985874060 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f599c004740 con 0x7f59a4109f40 2024-04-24T09:19:56.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.252+0000 7f599aa00700 1 --2- 172.21.15.90:0/3985874060 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f59880a3d50 0x7f59880a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:56.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.252+0000 7f599aa00700 1 -- 172.21.15.90:0/3985874060 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f599c00f020 con 0x7f59a4109f40 2024-04-24T09:19:56.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.252+0000 7f59a1e00700 1 --2- 172.21.15.90:0/3985874060 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f59880a3d50 0x7f59880a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:56.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.252+0000 7f599aa00700 1 --2- 172.21.15.90:0/3985874060 >> [v2:172.21.15.114:6800/344222414,v1:172.21.15.114:6801/344222414] conn(0x7f59880ad530 0x7f59880af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:56.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.252+0000 7f599aa00700 1 -- 172.21.15.90:0/3985874060 --> [v2:172.21.15.114:6800/344222414,v1:172.21.15.114:6801/344222414] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f599c009ce0 con 0x7f59880ad530 2024-04-24T09:19:56.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.252+0000 7f599aa00700 1 -- 172.21.15.90:0/3985874060 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f599c0d1c10 con 0x7f59a4109f40 2024-04-24T09:19:56.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.253+0000 7f59a1400700 1 --2- 172.21.15.90:0/3985874060 >> [v2:172.21.15.114:6800/344222414,v1:172.21.15.114:6801/344222414] conn(0x7f59880ad530 0x7f59880af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:56.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.253+0000 7f59a1e00700 1 --2- 172.21.15.90:0/3985874060 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f59880a3d50 0x7f59880a6210 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5994011490 tx=0x7f5994009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:56.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.253+0000 7f59a1400700 1 --2- 172.21.15.90:0/3985874060 >> [v2:172.21.15.114:6800/344222414,v1:172.21.15.114:6801/344222414] conn(0x7f59880ad530 0x7f59880af950 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:56.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.254+0000 7f599aa00700 1 -- 172.21.15.90:0/3985874060 <== osd.1 v2:172.21.15.114:6800/344222414 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f599c009ce0 con 0x7f59880ad530 2024-04-24T09:19:56.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.274+0000 7f59a3200700 1 -- 172.21.15.90:0/3985874060 --> [v2:172.21.15.114:6800/344222414,v1:172.21.15.114:6801/344222414] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f59a4063580 con 0x7f59880ad530 2024-04-24T09:19:56.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.274+0000 7f599aa00700 1 -- 172.21.15.90:0/3985874060 <== osd.1 v2:172.21.15.114:6800/344222414 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f59a4063580 con 0x7f59880ad530 2024-04-24T09:19:56.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.275+0000 7f59a3200700 1 -- 172.21.15.90:0/3985874060 >> [v2:172.21.15.114:6800/344222414,v1:172.21.15.114:6801/344222414] conn(0x7f59880ad530 msgr2=0x7f59880af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:56.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.275+0000 7f59a3200700 1 --2- 172.21.15.90:0/3985874060 >> [v2:172.21.15.114:6800/344222414,v1:172.21.15.114:6801/344222414] conn(0x7f59880ad530 0x7f59880af950 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:56.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.275+0000 7f59a3200700 1 -- 172.21.15.90:0/3985874060 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f59880a3d50 msgr2=0x7f59880a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:56.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.275+0000 7f59a3200700 1 --2- 172.21.15.90:0/3985874060 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f59880a3d50 0x7f59880a6210 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5994011490 tx=0x7f5994009260 comp rx=0 tx=0).stop 2024-04-24T09:19:56.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.275+0000 7f59a3200700 1 -- 172.21.15.90:0/3985874060 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f59a4109f40 msgr2=0x7f59a419fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:56.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.275+0000 7f59a3200700 1 --2- 172.21.15.90:0/3985874060 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f59a4109f40 0x7f59a419fc90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f599c0062a0 tx=0x7f599c004c60 comp rx=0 tx=0).stop 2024-04-24T09:19:56.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.275+0000 7f59a2800700 1 -- 172.21.15.90:0/3985874060 reap_dead start 2024-04-24T09:19:56.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.275+0000 7f59a3200700 1 -- 172.21.15.90:0/3985874060 shutdown_connections 2024-04-24T09:19:56.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.275+0000 7f59a3200700 1 -- 172.21.15.90:0/3985874060 >> 172.21.15.90:0/3985874060 conn(0x7f59a4100730 msgr2=0x7f59a4101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:56.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.275+0000 7f59a3200700 1 -- 172.21.15.90:0/3985874060 shutdown_connections 2024-04-24T09:19:56.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:52.275+0000 7f59a3200700 1 -- 172.21.15.90:0/3985874060 wait complete. 2024-04-24T09:19:56.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:56 smithi090 ceph-mon[27825]: pgmap v345: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-24T09:19:56.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:56 smithi149 ceph-mon[32592]: pgmap v345: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-24T09:19:56.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:56 smithi114 ceph-mon[32501]: pgmap v345: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-24T09:19:57.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.843+0000 7f2966800700 1 -- 172.21.15.90:0/2770457446 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2960109380 msgr2=0x7f2960109760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:57.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.843+0000 7f2966800700 1 --2- 172.21.15.90:0/2770457446 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2960109380 0x7f2960109760 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f29500099c0 tx=0x7f2950009cd0 comp rx=0 tx=0).stop 2024-04-24T09:19:57.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.843+0000 7f2966800700 1 -- 172.21.15.90:0/2770457446 shutdown_connections 2024-04-24T09:19:57.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.843+0000 7f2966800700 1 --2- 172.21.15.90:0/2770457446 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f296010e460 0x7f2960116030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:57.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.843+0000 7f2966800700 1 --2- 172.21.15.90:0/2770457446 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2960109380 0x7f2960109760 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:57.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.843+0000 7f2966800700 1 --2- 172.21.15.90:0/2770457446 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2960109d30 0x7f296010dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:57.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.843+0000 7f2966800700 1 -- 172.21.15.90:0/2770457446 >> 172.21.15.90:0/2770457446 conn(0x7f29600762c0 msgr2=0x7f2960107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:57.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.843+0000 7f2966800700 1 -- 172.21.15.90:0/2770457446 shutdown_connections 2024-04-24T09:19:57.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.843+0000 7f2966800700 1 -- 172.21.15.90:0/2770457446 wait complete. 2024-04-24T09:19:57.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.844+0000 7f2966800700 1 Processor -- start 2024-04-24T09:19:57.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.844+0000 7f2966800700 1 -- start start 2024-04-24T09:19:57.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.845+0000 7f2966800700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2960109380 0x7f29601a5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:57.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.845+0000 7f2966800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2960109d30 0x7f29601a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:57.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.845+0000 7f2966800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f296010e460 0x7f296019f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:57.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.845+0000 7f2966800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2960119490 con 0x7f2960109d30 2024-04-24T09:19:57.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.845+0000 7f2966800700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2960119310 con 0x7f2960109380 2024-04-24T09:19:57.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.845+0000 7f2966800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2960119610 con 0x7f296010e460 2024-04-24T09:19:57.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.845+0000 7f295ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2960109d30 0x7f29601a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:57.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.845+0000 7f295ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2960109d30 0x7f29601a5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57746/0 (socket says 172.21.15.90:57746) 2024-04-24T09:19:57.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.845+0000 7f295ea00700 1 -- 172.21.15.90:0/3390505037 learned_addr learned my addr 172.21.15.90:0/3390505037 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:19:57.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.845+0000 7f295fe00700 1 --2- 172.21.15.90:0/3390505037 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f296010e460 0x7f296019f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:57.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.845+0000 7f295f400700 1 --2- 172.21.15.90:0/3390505037 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2960109380 0x7f29601a5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:57.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.845+0000 7f295ea00700 1 -- 172.21.15.90:0/3390505037 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f296010e460 msgr2=0x7f296019f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:57.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.846+0000 7f295ea00700 1 --2- 172.21.15.90:0/3390505037 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f296010e460 0x7f296019f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:57.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.846+0000 7f295ea00700 1 -- 172.21.15.90:0/3390505037 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2960109380 msgr2=0x7f29601a5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:57.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.846+0000 7f295ea00700 1 --2- 172.21.15.90:0/3390505037 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2960109380 0x7f29601a5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:57.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.846+0000 7f295ea00700 1 -- 172.21.15.90:0/3390505037 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f296019ffd0 con 0x7f2960109d30 2024-04-24T09:19:57.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.846+0000 7f295fe00700 1 --2- 172.21.15.90:0/3390505037 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f296010e460 0x7f296019f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:19:57.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.846+0000 7f295ea00700 1 --2- 172.21.15.90:0/3390505037 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2960109d30 0x7f29601a5b60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f294c00e970 tx=0x7f294c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:57.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.846+0000 7f2957e00700 1 -- 172.21.15.90:0/3390505037 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c004a30 con 0x7f2960109d30 2024-04-24T09:19:57.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.846+0000 7f2966800700 1 -- 172.21.15.90:0/3390505037 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29601a02c0 con 0x7f2960109d30 2024-04-24T09:19:57.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.846+0000 7f2957e00700 1 -- 172.21.15.90:0/3390505037 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f294c00ccc0 con 0x7f2960109d30 2024-04-24T09:19:57.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.846+0000 7f2957e00700 1 -- 172.21.15.90:0/3390505037 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c005650 con 0x7f2960109d30 2024-04-24T09:19:57.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.846+0000 7f2966800700 1 -- 172.21.15.90:0/3390505037 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29601ac5a0 con 0x7f2960109d30 2024-04-24T09:19:57.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.847+0000 7f2966800700 1 -- 172.21.15.90:0/3390505037 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2960063240 con 0x7f2960109d30 2024-04-24T09:19:57.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.849+0000 7f2957e00700 1 -- 172.21.15.90:0/3390505037 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f294c004bd0 con 0x7f2960109d30 2024-04-24T09:19:57.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.851+0000 7f2957e00700 1 --2- 172.21.15.90:0/3390505037 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f29440a3d50 0x7f29440a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:57.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.851+0000 7f2957e00700 1 -- 172.21.15.90:0/3390505037 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f294c0d0a90 con 0x7f2960109d30 2024-04-24T09:19:57.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.851+0000 7f295f400700 1 --2- 172.21.15.90:0/3390505037 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f29440a3d50 0x7f29440a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:57.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.851+0000 7f2957e00700 1 --2- 172.21.15.90:0/3390505037 >> [v2:172.21.15.90:6810/3068524970,v1:172.21.15.90:6811/3068524970] conn(0x7f29440ad530 0x7f29440af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:57.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.851+0000 7f2957e00700 1 -- 172.21.15.90:0/3390505037 --> [v2:172.21.15.90:6810/3068524970,v1:172.21.15.90:6811/3068524970] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f294c01fe50 con 0x7f29440ad530 2024-04-24T09:19:57.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.851+0000 7f2957e00700 1 -- 172.21.15.90:0/3390505037 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f294c0d0e40 con 0x7f2960109d30 2024-04-24T09:19:57.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.852+0000 7f295fe00700 1 --2- 172.21.15.90:0/3390505037 >> [v2:172.21.15.90:6810/3068524970,v1:172.21.15.90:6811/3068524970] conn(0x7f29440ad530 0x7f29440af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:57.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.852+0000 7f295f400700 1 --2- 172.21.15.90:0/3390505037 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f29440a3d50 0x7f29440a6210 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2950000c00 tx=0x7f2950005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:57.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.852+0000 7f295fe00700 1 --2- 172.21.15.90:0/3390505037 >> [v2:172.21.15.90:6810/3068524970,v1:172.21.15.90:6811/3068524970] conn(0x7f29440ad530 0x7f29440af950 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:57.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.853+0000 7f2957e00700 1 -- 172.21.15.90:0/3390505037 <== osd.3 v2:172.21.15.90:6810/3068524970 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f294c01fe50 con 0x7f29440ad530 2024-04-24T09:19:57.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.871+0000 7f2966800700 1 -- 172.21.15.90:0/3390505037 --> [v2:172.21.15.90:6810/3068524970,v1:172.21.15.90:6811/3068524970] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2960063640 con 0x7f29440ad530 2024-04-24T09:19:57.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.872+0000 7f2957e00700 1 -- 172.21.15.90:0/3390505037 <== osd.3 v2:172.21.15.90:6810/3068524970 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2960063640 con 0x7f29440ad530 2024-04-24T09:19:57.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.872+0000 7f2966800700 1 -- 172.21.15.90:0/3390505037 >> [v2:172.21.15.90:6810/3068524970,v1:172.21.15.90:6811/3068524970] conn(0x7f29440ad530 msgr2=0x7f29440af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:57.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.872+0000 7f2966800700 1 --2- 172.21.15.90:0/3390505037 >> [v2:172.21.15.90:6810/3068524970,v1:172.21.15.90:6811/3068524970] conn(0x7f29440ad530 0x7f29440af950 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:57.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.872+0000 7f2966800700 1 -- 172.21.15.90:0/3390505037 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f29440a3d50 msgr2=0x7f29440a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:57.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.872+0000 7f2966800700 1 --2- 172.21.15.90:0/3390505037 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f29440a3d50 0x7f29440a6210 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2950000c00 tx=0x7f2950005fb0 comp rx=0 tx=0).stop 2024-04-24T09:19:57.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.872+0000 7f2966800700 1 -- 172.21.15.90:0/3390505037 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2960109d30 msgr2=0x7f29601a5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:57.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.872+0000 7f2966800700 1 --2- 172.21.15.90:0/3390505037 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2960109d30 0x7f29601a5b60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f294c00e970 tx=0x7f294c00ec80 comp rx=0 tx=0).stop 2024-04-24T09:19:57.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.872+0000 7f295fe00700 1 -- 172.21.15.90:0/3390505037 reap_dead start 2024-04-24T09:19:57.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.872+0000 7f2966800700 1 -- 172.21.15.90:0/3390505037 shutdown_connections 2024-04-24T09:19:57.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.872+0000 7f2966800700 1 -- 172.21.15.90:0/3390505037 >> 172.21.15.90:0/3390505037 conn(0x7f29600762c0 msgr2=0x7f29600778b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:57.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.873+0000 7f2966800700 1 -- 172.21.15.90:0/3390505037 shutdown_connections 2024-04-24T09:19:57.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:50.873+0000 7f2966800700 1 -- 172.21.15.90:0/3390505037 wait complete. 2024-04-24T09:19:57.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:57 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-04-24T09:19:57.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:57 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-04-24T09:19:57.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:57 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-24T09:19:57.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:57 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-04-24T09:19:57.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:57 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-04-24T09:19:57.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:57 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-04-24T09:19:57.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:57 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-04-24T09:19:57.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:57 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-04-24T09:19:57.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:57 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-04-24T09:19:57.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:57 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-04-24T09:19:57.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:57 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-04-24T09:19:57.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:57 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-24T09:19:57.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:57 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-04-24T09:19:57.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:57 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-04-24T09:19:57.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:57 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-04-24T09:19:57.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:57 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-04-24T09:19:57.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:57 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-04-24T09:19:57.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:57 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-04-24T09:19:58.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:57 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-04-24T09:19:58.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:57 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-04-24T09:19:58.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:57 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-24T09:19:58.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:57 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-04-24T09:19:58.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:57 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-04-24T09:19:58.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:57 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-04-24T09:19:58.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:57 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-04-24T09:19:58.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:57 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-04-24T09:19:58.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:57 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-04-24T09:19:58.293 INFO:teuthology.orchestra.run.smithi090.stdout:55834574930 2024-04-24T09:19:58.293 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-24T09:19:58.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:58 smithi090 ceph-mon[27825]: pgmap v346: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-24T09:19:58.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:58 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-04-24T09:19:58.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:58 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-04-24T09:19:58.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:58 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-24T09:19:58.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:58 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-04-24T09:19:58.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:58 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-04-24T09:19:58.854 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:58 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-04-24T09:19:58.854 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:58 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-04-24T09:19:58.854 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:58 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-04-24T09:19:58.854 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:58 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-04-24T09:19:58.854 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:58 smithi090 ceph-mon[27825]: osdmap e87: 12 total, 12 up, 12 in 2024-04-24T09:19:58.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:58 smithi114 ceph-mon[32501]: pgmap v346: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-24T09:19:58.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:58 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-04-24T09:19:58.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:58 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-04-24T09:19:58.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:58 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-24T09:19:58.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:58 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-04-24T09:19:58.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:58 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-04-24T09:19:58.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:58 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-04-24T09:19:58.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:58 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-04-24T09:19:58.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:58 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-04-24T09:19:58.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:58 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-04-24T09:19:58.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:58 smithi114 ceph-mon[32501]: osdmap e87: 12 total, 12 up, 12 in 2024-04-24T09:19:59.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:58 smithi149 ceph-mon[32592]: pgmap v346: 97 pgs: 97 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-24T09:19:59.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:58 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-04-24T09:19:59.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:58 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-04-24T09:19:59.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:58 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-24T09:19:59.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:58 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-04-24T09:19:59.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:58 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-04-24T09:19:59.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:58 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-04-24T09:19:59.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:58 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-04-24T09:19:59.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:58 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-04-24T09:19:59.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:58 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-04-24T09:19:59.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:58 smithi149 ceph-mon[32592]: osdmap e87: 12 total, 12 up, 12 in 2024-04-24T09:19:59.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.511+0000 7f85d0e00700 1 -- 172.21.15.90:0/3040119132 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85cc102b40 msgr2=0x7f85cc102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:59.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.511+0000 7f85d0e00700 1 --2- 172.21.15.90:0/3040119132 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85cc102b40 0x7f85cc102fc0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f85b8009990 tx=0x7f85b8009ca0 comp rx=0 tx=0).stop 2024-04-24T09:19:59.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.512+0000 7f85d0e00700 1 -- 172.21.15.90:0/3040119132 shutdown_connections 2024-04-24T09:19:59.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.512+0000 7f85d0e00700 1 --2- 172.21.15.90:0/3040119132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85cc103500 0x7f85cc10f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:59.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.512+0000 7f85d0e00700 1 --2- 172.21.15.90:0/3040119132 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85cc108b90 0x7f85cc108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:59.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.512+0000 7f85d0e00700 1 --2- 172.21.15.90:0/3040119132 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85cc102b40 0x7f85cc102fc0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:59.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.512+0000 7f85d0e00700 1 -- 172.21.15.90:0/3040119132 >> 172.21.15.90:0/3040119132 conn(0x7f85cc079c50 msgr2=0x7f85cc07a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:59.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.512+0000 7f85d0e00700 1 -- 172.21.15.90:0/3040119132 shutdown_connections 2024-04-24T09:19:59.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.512+0000 7f85d0e00700 1 -- 172.21.15.90:0/3040119132 wait complete. 2024-04-24T09:19:59.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.513+0000 7f85d0e00700 1 Processor -- start 2024-04-24T09:19:59.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.513+0000 7f85d0e00700 1 -- start start 2024-04-24T09:19:59.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.513+0000 7f85d0e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85cc102b40 0x7f85cc199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:59.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.513+0000 7f85d0e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85cc103500 0x7f85cc199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:59.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.513+0000 7f85d0e00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85cc108b90 0x7f85cc19dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:59.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.513+0000 7f85d0e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f85cc112e50 con 0x7f85cc102b40 2024-04-24T09:19:59.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.513+0000 7f85d0e00700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f85cc112cd0 con 0x7f85cc108b90 2024-04-24T09:19:59.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.513+0000 7f85d0e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f85cc112fd0 con 0x7f85cc103500 2024-04-24T09:19:59.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.513+0000 7f85c9e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85cc102b40 0x7f85cc199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:59.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.513+0000 7f85c9e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85cc102b40 0x7f85cc199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58284/0 (socket says 172.21.15.90:58284) 2024-04-24T09:19:59.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.513+0000 7f85c9e00700 1 -- 172.21.15.90:0/480442483 learned_addr learned my addr 172.21.15.90:0/480442483 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:19:59.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.513+0000 7f85c9400700 1 --2- 172.21.15.90:0/480442483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85cc103500 0x7f85cc199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:59.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.514+0000 7f85ca800700 1 --2- 172.21.15.90:0/480442483 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85cc108b90 0x7f85cc19dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:59.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.514+0000 7f85c9400700 1 -- 172.21.15.90:0/480442483 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85cc108b90 msgr2=0x7f85cc19dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:59.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.514+0000 7f85c9400700 1 --2- 172.21.15.90:0/480442483 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85cc108b90 0x7f85cc19dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:59.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.514+0000 7f85c9400700 1 -- 172.21.15.90:0/480442483 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85cc102b40 msgr2=0x7f85cc199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:59.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.514+0000 7f85c9400700 1 --2- 172.21.15.90:0/480442483 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85cc102b40 0x7f85cc199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:59.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.514+0000 7f85c9400700 1 -- 172.21.15.90:0/480442483 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85cc19e300 con 0x7f85cc103500 2024-04-24T09:19:59.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.514+0000 7f85c9e00700 1 --2- 172.21.15.90:0/480442483 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85cc102b40 0x7f85cc199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:19:59.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.514+0000 7f85c9400700 1 --2- 172.21.15.90:0/480442483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85cc103500 0x7f85cc199880 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f85b80098e0 tx=0x7f85b800b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:59.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.515+0000 7f85c2a00700 1 -- 172.21.15.90:0/480442483 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b8027070 con 0x7f85cc103500 2024-04-24T09:19:59.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.515+0000 7f85d0e00700 1 -- 172.21.15.90:0/480442483 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85cc19e590 con 0x7f85cc103500 2024-04-24T09:19:59.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.515+0000 7f85c2a00700 1 -- 172.21.15.90:0/480442483 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85b8003b60 con 0x7f85cc103500 2024-04-24T09:19:59.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.515+0000 7f85c2a00700 1 -- 172.21.15.90:0/480442483 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b8022d50 con 0x7f85cc103500 2024-04-24T09:19:59.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.515+0000 7f85d0e00700 1 -- 172.21.15.90:0/480442483 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85cc1a5e70 con 0x7f85cc103500 2024-04-24T09:19:59.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.516+0000 7f85d0e00700 1 -- 172.21.15.90:0/480442483 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f85cc063280 con 0x7f85cc103500 2024-04-24T09:19:59.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.518+0000 7f85c2a00700 1 -- 172.21.15.90:0/480442483 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f85b802a070 con 0x7f85cc103500 2024-04-24T09:19:59.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.519+0000 7f85c2a00700 1 --2- 172.21.15.90:0/480442483 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f85b00a3cd0 0x7f85b00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:59.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.519+0000 7f85c2a00700 1 -- 172.21.15.90:0/480442483 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f85b80d1fa0 con 0x7f85cc103500 2024-04-24T09:19:59.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.519+0000 7f85c9e00700 1 --2- 172.21.15.90:0/480442483 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f85b00a3cd0 0x7f85b00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:59.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.519+0000 7f85c2a00700 1 --2- 172.21.15.90:0/480442483 >> [v2:172.21.15.149:6816/157615149,v1:172.21.15.149:6817/157615149] conn(0x7f85b00ad890 0x7f85b00afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:59.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.520+0000 7f85c2a00700 1 -- 172.21.15.90:0/480442483 --> [v2:172.21.15.149:6816/157615149,v1:172.21.15.149:6817/157615149] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f85b8009030 con 0x7f85b00ad890 2024-04-24T09:19:59.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.520+0000 7f85c2a00700 1 -- 172.21.15.90:0/480442483 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f85b80d2350 con 0x7f85cc103500 2024-04-24T09:19:59.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.520+0000 7f85ca800700 1 --2- 172.21.15.90:0/480442483 >> [v2:172.21.15.149:6816/157615149,v1:172.21.15.149:6817/157615149] conn(0x7f85b00ad890 0x7f85b00afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:59.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.520+0000 7f85c9e00700 1 --2- 172.21.15.90:0/480442483 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f85b00a3cd0 0x7f85b00a6190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f85bc009680 tx=0x7f85bc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:59.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.520+0000 7f85ca800700 1 --2- 172.21.15.90:0/480442483 >> [v2:172.21.15.149:6816/157615149,v1:172.21.15.149:6817/157615149] conn(0x7f85b00ad890 0x7f85b00afd30 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:59.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.521+0000 7f85c2a00700 1 -- 172.21.15.90:0/480442483 <== osd.8 v2:172.21.15.149:6816/157615149 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f85b8009030 con 0x7f85b00ad890 2024-04-24T09:19:59.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.538+0000 7f85d0e00700 1 -- 172.21.15.90:0/480442483 --> [v2:172.21.15.149:6816/157615149,v1:172.21.15.149:6817/157615149] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f85cc063680 con 0x7f85b00ad890 2024-04-24T09:19:59.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.539+0000 7f85c2a00700 1 -- 172.21.15.90:0/480442483 <== osd.8 v2:172.21.15.149:6816/157615149 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f85cc063680 con 0x7f85b00ad890 2024-04-24T09:19:59.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.540+0000 7f85d0e00700 1 -- 172.21.15.90:0/480442483 >> [v2:172.21.15.149:6816/157615149,v1:172.21.15.149:6817/157615149] conn(0x7f85b00ad890 msgr2=0x7f85b00afd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:59.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.540+0000 7f85d0e00700 1 --2- 172.21.15.90:0/480442483 >> [v2:172.21.15.149:6816/157615149,v1:172.21.15.149:6817/157615149] conn(0x7f85b00ad890 0x7f85b00afd30 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:59.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.540+0000 7f85d0e00700 1 -- 172.21.15.90:0/480442483 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f85b00a3cd0 msgr2=0x7f85b00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:59.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.540+0000 7f85d0e00700 1 --2- 172.21.15.90:0/480442483 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f85b00a3cd0 0x7f85b00a6190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f85bc009680 tx=0x7f85bc009260 comp rx=0 tx=0).stop 2024-04-24T09:19:59.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.540+0000 7f85d0e00700 1 -- 172.21.15.90:0/480442483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85cc103500 msgr2=0x7f85cc199880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:59.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.540+0000 7f85d0e00700 1 --2- 172.21.15.90:0/480442483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85cc103500 0x7f85cc199880 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f85b80098e0 tx=0x7f85b800b8a0 comp rx=0 tx=0).stop 2024-04-24T09:19:59.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.540+0000 7f85ca800700 1 -- 172.21.15.90:0/480442483 reap_dead start 2024-04-24T09:19:59.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.540+0000 7f85d0e00700 1 -- 172.21.15.90:0/480442483 shutdown_connections 2024-04-24T09:19:59.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.540+0000 7f85d0e00700 1 -- 172.21.15.90:0/480442483 >> 172.21.15.90:0/480442483 conn(0x7f85cc079c50 msgr2=0x7f85cc101810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:59.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.540+0000 7f85d0e00700 1 -- 172.21.15.90:0/480442483 shutdown_connections 2024-04-24T09:19:59.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.540+0000 7f85d0e00700 1 -- 172.21.15.90:0/480442483 wait complete. 2024-04-24T09:19:59.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:19:59 smithi114 ceph-mon[32501]: osdmap e88: 12 total, 12 up, 12 in 2024-04-24T09:19:59.965 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:19:59 smithi090 ceph-mon[27825]: osdmap e88: 12 total, 12 up, 12 in 2024-04-24T09:20:00.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:19:59 smithi149 ceph-mon[32592]: osdmap e88: 12 total, 12 up, 12 in 2024-04-24T09:20:00.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.394+0000 7f9070400700 1 -- 172.21.15.90:0/1411710123 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f906806b1a0 msgr2=0x7f906810f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:00.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.394+0000 7f9070400700 1 --2- 172.21.15.90:0/1411710123 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f906806b1a0 0x7f906810f9b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9060009f90 tx=0x7f906000b330 comp rx=0 tx=0).stop 2024-04-24T09:20:00.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.394+0000 7f9070400700 1 -- 172.21.15.90:0/1411710123 shutdown_connections 2024-04-24T09:20:00.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.394+0000 7f9070400700 1 --2- 172.21.15.90:0/1411710123 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f906806b1a0 0x7f906810f9b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:00.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.394+0000 7f9070400700 1 --2- 172.21.15.90:0/1411710123 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9068108b70 0x7f9068108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:00.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.394+0000 7f9070400700 1 --2- 172.21.15.90:0/1411710123 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f906806a800 0x7f906806ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:00.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.394+0000 7f9070400700 1 -- 172.21.15.90:0/1411710123 >> 172.21.15.90:0/1411710123 conn(0x7f90680789a0 msgr2=0x7f9068078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:20:00.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.394+0000 7f9070400700 1 -- 172.21.15.90:0/1411710123 shutdown_connections 2024-04-24T09:20:00.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.395+0000 7f9070400700 1 -- 172.21.15.90:0/1411710123 wait complete. 2024-04-24T09:20:00.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.396+0000 7f9070400700 1 Processor -- start 2024-04-24T09:20:00.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.396+0000 7f9070400700 1 -- start start 2024-04-24T09:20:00.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.396+0000 7f9070400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f906806a800 0x7f9068101ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:00.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.396+0000 7f9070400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f906806b1a0 0x7f9068102000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:00.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.396+0000 7f9070400700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9068108b70 0x7f9068102540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:00.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.396+0000 7f9070400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9068112e20 con 0x7f906806a800 2024-04-24T09:20:00.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.396+0000 7f9070400700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9068112ca0 con 0x7f9068108b70 2024-04-24T09:20:00.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.396+0000 7f9070400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9068112fa0 con 0x7f906806b1a0 2024-04-24T09:20:00.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.397+0000 7f906d000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f906806b1a0 0x7f9068102000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:00.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.397+0000 7f906da00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f906806a800 0x7f9068101ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:00.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.397+0000 7f906da00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f906806a800 0x7f9068101ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58258/0 (socket says 172.21.15.90:58258) 2024-04-24T09:20:00.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.397+0000 7f906da00700 1 -- 172.21.15.90:0/3917427532 learned_addr learned my addr 172.21.15.90:0/3917427532 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:20:00.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.397+0000 7f906e400700 1 --2- 172.21.15.90:0/3917427532 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9068108b70 0x7f9068102540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:00.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.397+0000 7f906d000700 1 -- 172.21.15.90:0/3917427532 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9068108b70 msgr2=0x7f9068102540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:00.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.397+0000 7f906d000700 1 --2- 172.21.15.90:0/3917427532 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9068108b70 0x7f9068102540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:00.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.397+0000 7f906d000700 1 -- 172.21.15.90:0/3917427532 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f906806a800 msgr2=0x7f9068101ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:00.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.397+0000 7f906d000700 1 --2- 172.21.15.90:0/3917427532 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f906806a800 0x7f9068101ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:00.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.397+0000 7f906d000700 1 -- 172.21.15.90:0/3917427532 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90681a77e0 con 0x7f906806b1a0 2024-04-24T09:20:00.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.397+0000 7f906da00700 1 --2- 172.21.15.90:0/3917427532 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f906806a800 0x7f9068101ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:20:00.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.398+0000 7f906d000700 1 --2- 172.21.15.90:0/3917427532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f906806b1a0 0x7f9068102000 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f905400e9c0 tx=0x7f905400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:20:00.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.398+0000 7f9066000700 1 -- 172.21.15.90:0/3917427532 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905400c9c0 con 0x7f906806b1a0 2024-04-24T09:20:00.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.398+0000 7f9070400700 1 -- 172.21.15.90:0/3917427532 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90681a7ad0 con 0x7f906806b1a0 2024-04-24T09:20:00.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.398+0000 7f9066000700 1 -- 172.21.15.90:0/3917427532 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f905400cb60 con 0x7f906806b1a0 2024-04-24T09:20:00.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.398+0000 7f9066000700 1 -- 172.21.15.90:0/3917427532 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9054018680 con 0x7f906806b1a0 2024-04-24T09:20:00.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.398+0000 7f9070400700 1 -- 172.21.15.90:0/3917427532 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90681a8060 con 0x7f906806b1a0 2024-04-24T09:20:00.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.399+0000 7f9070400700 1 -- 172.21.15.90:0/3917427532 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f90680631c0 con 0x7f906806b1a0 2024-04-24T09:20:00.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.401+0000 7f9066000700 1 -- 172.21.15.90:0/3917427532 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f905400cd10 con 0x7f906806b1a0 2024-04-24T09:20:00.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.402+0000 7f9066000700 1 --2- 172.21.15.90:0/3917427532 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f90500a3d50 0x7f90500a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:00.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.403+0000 7f9066000700 1 -- 172.21.15.90:0/3917427532 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f90540d0c00 con 0x7f906806b1a0 2024-04-24T09:20:00.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.403+0000 7f9066000700 1 --2- 172.21.15.90:0/3917427532 >> [v2:172.21.15.114:6808/2782015522,v1:172.21.15.114:6809/2782015522] conn(0x7f90500ad910 0x7f90500afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:00.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.403+0000 7f9066000700 1 -- 172.21.15.90:0/3917427532 --> [v2:172.21.15.114:6808/2782015522,v1:172.21.15.114:6809/2782015522] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f905400be50 con 0x7f90500ad910 2024-04-24T09:20:00.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.403+0000 7f9066000700 1 -- 172.21.15.90:0/3917427532 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f90540d0fb0 con 0x7f906806b1a0 2024-04-24T09:20:00.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.403+0000 7f906da00700 1 --2- 172.21.15.90:0/3917427532 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f90500a3d50 0x7f90500a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:00.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.403+0000 7f906e400700 1 --2- 172.21.15.90:0/3917427532 >> [v2:172.21.15.114:6808/2782015522,v1:172.21.15.114:6809/2782015522] conn(0x7f90500ad910 0x7f90500afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:00.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.404+0000 7f906da00700 1 --2- 172.21.15.90:0/3917427532 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f90500a3d50 0x7f90500a6210 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f905c011550 tx=0x7f905c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:20:00.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.404+0000 7f906e400700 1 --2- 172.21.15.90:0/3917427532 >> [v2:172.21.15.114:6808/2782015522,v1:172.21.15.114:6809/2782015522] conn(0x7f90500ad910 0x7f90500afdb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:20:00.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.405+0000 7f9066000700 1 -- 172.21.15.90:0/3917427532 <== osd.4 v2:172.21.15.114:6808/2782015522 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f905400be50 con 0x7f90500ad910 2024-04-24T09:20:00.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.423+0000 7f9070400700 1 -- 172.21.15.90:0/3917427532 --> [v2:172.21.15.114:6808/2782015522,v1:172.21.15.114:6809/2782015522] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f90680635c0 con 0x7f90500ad910 2024-04-24T09:20:00.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.424+0000 7f9066000700 1 -- 172.21.15.90:0/3917427532 <== osd.4 v2:172.21.15.114:6808/2782015522 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f90680635c0 con 0x7f90500ad910 2024-04-24T09:20:00.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.424+0000 7f9070400700 1 -- 172.21.15.90:0/3917427532 >> [v2:172.21.15.114:6808/2782015522,v1:172.21.15.114:6809/2782015522] conn(0x7f90500ad910 msgr2=0x7f90500afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:00.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.424+0000 7f9070400700 1 --2- 172.21.15.90:0/3917427532 >> [v2:172.21.15.114:6808/2782015522,v1:172.21.15.114:6809/2782015522] conn(0x7f90500ad910 0x7f90500afdb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:00.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.424+0000 7f9070400700 1 -- 172.21.15.90:0/3917427532 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f90500a3d50 msgr2=0x7f90500a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:00.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.424+0000 7f9070400700 1 --2- 172.21.15.90:0/3917427532 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f90500a3d50 0x7f90500a6210 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f905c011550 tx=0x7f905c009260 comp rx=0 tx=0).stop 2024-04-24T09:20:00.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.424+0000 7f9070400700 1 -- 172.21.15.90:0/3917427532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f906806b1a0 msgr2=0x7f9068102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:00.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.424+0000 7f9070400700 1 --2- 172.21.15.90:0/3917427532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f906806b1a0 0x7f9068102000 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f905400e9c0 tx=0x7f905400ecd0 comp rx=0 tx=0).stop 2024-04-24T09:20:00.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.424+0000 7f906e400700 1 -- 172.21.15.90:0/3917427532 reap_dead start 2024-04-24T09:20:00.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.425+0000 7f9070400700 1 -- 172.21.15.90:0/3917427532 shutdown_connections 2024-04-24T09:20:00.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.425+0000 7f9070400700 1 -- 172.21.15.90:0/3917427532 >> 172.21.15.90:0/3917427532 conn(0x7f90680789a0 msgr2=0x7f90680ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:20:00.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.425+0000 7f9070400700 1 -- 172.21.15.90:0/3917427532 shutdown_connections 2024-04-24T09:20:00.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:19:59.425+0000 7f9070400700 1 -- 172.21.15.90:0/3917427532 wait complete. 2024-04-24T09:20:00.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:00 smithi114 ceph-mon[32501]: pgmap v349: 97 pgs: 2 peering, 95 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 6 op/s; 146 B/s, 0 objects/s recovering 2024-04-24T09:20:00.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:00 smithi114 ceph-mon[32501]: overall HEALTH_OK 2024-04-24T09:20:01.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:00 smithi090 ceph-mon[27825]: pgmap v349: 97 pgs: 2 peering, 95 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 6 op/s; 146 B/s, 0 objects/s recovering 2024-04-24T09:20:01.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:00 smithi090 ceph-mon[27825]: overall HEALTH_OK 2024-04-24T09:20:01.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:00 smithi149 ceph-mon[32592]: pgmap v349: 97 pgs: 2 peering, 95 active+clean; 643 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 6 op/s; 146 B/s, 0 objects/s recovering 2024-04-24T09:20:01.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:00 smithi149 ceph-mon[32592]: overall HEALTH_OK 2024-04-24T09:20:01.302 INFO:teuthology.orchestra.run.smithi090.stdout:111669149765 2024-04-24T09:20:01.302 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-24T09:20:02.206 INFO:teuthology.orchestra.run.smithi090.stdout:167503724601 2024-04-24T09:20:02.206 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-24T09:20:02.723 INFO:teuthology.orchestra.run.smithi090.stdout:77309411404 2024-04-24T09:20:02.723 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-24T09:20:02.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:02 smithi090 ceph-mon[27825]: pgmap v350: 97 pgs: 2 peering, 95 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 3 op/s; 146 B/s, 0 objects/s recovering 2024-04-24T09:20:02.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:02 smithi114 ceph-mon[32501]: pgmap v350: 97 pgs: 2 peering, 95 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 3 op/s; 146 B/s, 0 objects/s recovering 2024-04-24T09:20:03.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:02 smithi149 ceph-mon[32592]: pgmap v350: 97 pgs: 2 peering, 95 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 3 op/s; 146 B/s, 0 objects/s recovering 2024-04-24T09:20:03.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.330+0000 7fc1ec200700 1 -- 172.21.15.90:0/3275432144 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc1e4103500 msgr2=0x7fc1e410f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:03.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.330+0000 7fc1ec200700 1 --2- 172.21.15.90:0/3275432144 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc1e4103500 0x7fc1e410f9e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc00b0a0 tx=0x7fc1dc00b3b0 comp rx=0 tx=0).stop 2024-04-24T09:20:03.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.331+0000 7fc1ec200700 1 -- 172.21.15.90:0/3275432144 shutdown_connections 2024-04-24T09:20:03.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.331+0000 7fc1ec200700 1 --2- 172.21.15.90:0/3275432144 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc1e4103500 0x7fc1e410f9e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:03.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.331+0000 7fc1ec200700 1 --2- 172.21.15.90:0/3275432144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc1e4108c50 0x7fc1e4109030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:03.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.331+0000 7fc1ec200700 1 --2- 172.21.15.90:0/3275432144 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1e4102b40 0x7fc1e4102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:03.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.331+0000 7fc1ec200700 1 -- 172.21.15.90:0/3275432144 >> 172.21.15.90:0/3275432144 conn(0x7fc1e4079ca0 msgr2=0x7fc1e407a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:20:03.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.331+0000 7fc1ec200700 1 -- 172.21.15.90:0/3275432144 shutdown_connections 2024-04-24T09:20:03.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.331+0000 7fc1ec200700 1 -- 172.21.15.90:0/3275432144 wait complete. 2024-04-24T09:20:03.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.332+0000 7fc1ec200700 1 Processor -- start 2024-04-24T09:20:03.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.332+0000 7fc1ec200700 1 -- start start 2024-04-24T09:20:03.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.333+0000 7fc1ec200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc1e4102b40 0x7fc1e4074df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:03.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.333+0000 7fc1ec200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1e4103500 0x7fc1e4075330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:03.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.333+0000 7fc1ec200700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc1e4108c50 0x7fc1e4075870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:03.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.333+0000 7fc1ec200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e4112ec0 con 0x7fc1e4103500 2024-04-24T09:20:03.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.333+0000 7fc1ec200700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e4112d40 con 0x7fc1e4108c50 2024-04-24T09:20:03.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.333+0000 7fc1ec200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e4113040 con 0x7fc1e4102b40 2024-04-24T09:20:03.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.333+0000 7fc1e8e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1e4103500 0x7fc1e4075330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:03.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.333+0000 7fc1e9800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc1e4102b40 0x7fc1e4074df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:03.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.333+0000 7fc1e8e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1e4103500 0x7fc1e4075330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58314/0 (socket says 172.21.15.90:58314) 2024-04-24T09:20:03.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.333+0000 7fc1e9800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc1e4102b40 0x7fc1e4074df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:35776/0 (socket says 172.21.15.90:35776) 2024-04-24T09:20:03.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.333+0000 7fc1e8e00700 1 -- 172.21.15.90:0/2589545769 learned_addr learned my addr 172.21.15.90:0/2589545769 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:20:03.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.333+0000 7fc1ea200700 1 --2- 172.21.15.90:0/2589545769 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc1e4108c50 0x7fc1e4075870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:03.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.334+0000 7fc1ea200700 1 -- 172.21.15.90:0/2589545769 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc1e4102b40 msgr2=0x7fc1e4074df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:03.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.334+0000 7fc1ea200700 1 --2- 172.21.15.90:0/2589545769 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc1e4102b40 0x7fc1e4074df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:03.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.334+0000 7fc1ea200700 1 -- 172.21.15.90:0/2589545769 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1e4103500 msgr2=0x7fc1e4075330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:03.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.334+0000 7fc1ea200700 1 --2- 172.21.15.90:0/2589545769 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1e4103500 0x7fc1e4075330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:03.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.334+0000 7fc1ea200700 1 -- 172.21.15.90:0/2589545769 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1e41a7720 con 0x7fc1e4108c50 2024-04-24T09:20:03.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.334+0000 7fc1e9800700 1 --2- 172.21.15.90:0/2589545769 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc1e4102b40 0x7fc1e4074df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:20:03.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.334+0000 7fc1ea200700 1 --2- 172.21.15.90:0/2589545769 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc1e4108c50 0x7fc1e4075870 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc00b070 tx=0x7fc1dc004c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:20:03.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.334+0000 7fc1e2000700 1 -- 172.21.15.90:0/2589545769 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1dc021070 con 0x7fc1e4108c50 2024-04-24T09:20:03.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.334+0000 7fc1e2000700 1 -- 172.21.15.90:0/2589545769 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1dc007b80 con 0x7fc1e4108c50 2024-04-24T09:20:03.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.334+0000 7fc1ec200700 1 -- 172.21.15.90:0/2589545769 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1e41a79b0 con 0x7fc1e4108c50 2024-04-24T09:20:03.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.334+0000 7fc1e2000700 1 -- 172.21.15.90:0/2589545769 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1dc005040 con 0x7fc1e4108c50 2024-04-24T09:20:03.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.335+0000 7fc1ec200700 1 -- 172.21.15.90:0/2589545769 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1e41a7e60 con 0x7fc1e4108c50 2024-04-24T09:20:03.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.335+0000 7fc1d7400700 1 -- 172.21.15.90:0/2589545769 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc1c0001030 con 0x7fc1e4108c50 2024-04-24T09:20:03.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.338+0000 7fc1e2000700 1 -- 172.21.15.90:0/2589545769 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc1dc024070 con 0x7fc1e4108c50 2024-04-24T09:20:03.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.339+0000 7fc1e2000700 1 --2- 172.21.15.90:0/2589545769 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fc1cc0a3cd0 0x7fc1cc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:03.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.339+0000 7fc1e2000700 1 -- 172.21.15.90:0/2589545769 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fc1dc00f020 con 0x7fc1e4108c50 2024-04-24T09:20:03.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.339+0000 7fc1e9800700 1 --2- 172.21.15.90:0/2589545769 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fc1cc0a3cd0 0x7fc1cc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:03.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.339+0000 7fc1e2000700 1 --2- 172.21.15.90:0/2589545769 >> [v2:172.21.15.114:6816/3222546280,v1:172.21.15.114:6817/3222546280] conn(0x7fc1cc0ad890 0x7fc1cc0afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:03.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.339+0000 7fc1e2000700 1 -- 172.21.15.90:0/2589545769 --> [v2:172.21.15.114:6816/3222546280,v1:172.21.15.114:6817/3222546280] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc1cc0b0400 con 0x7fc1cc0ad890 2024-04-24T09:20:03.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.339+0000 7fc1e2000700 1 -- 172.21.15.90:0/2589545769 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fc1dc0d1db0 con 0x7fc1e4108c50 2024-04-24T09:20:03.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.340+0000 7fc1e8e00700 1 --2- 172.21.15.90:0/2589545769 >> [v2:172.21.15.114:6816/3222546280,v1:172.21.15.114:6817/3222546280] conn(0x7fc1cc0ad890 0x7fc1cc0afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:03.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.340+0000 7fc1e9800700 1 --2- 172.21.15.90:0/2589545769 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fc1cc0a3cd0 0x7fc1cc0a6190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8009fd0 tx=0x7fc1d8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:20:03.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.340+0000 7fc1e8e00700 1 --2- 172.21.15.90:0/2589545769 >> [v2:172.21.15.114:6816/3222546280,v1:172.21.15.114:6817/3222546280] conn(0x7fc1cc0ad890 0x7fc1cc0afd30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:20:03.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.341+0000 7fc1e2000700 1 -- 172.21.15.90:0/2589545769 <== osd.7 v2:172.21.15.114:6816/3222546280 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc1cc0b0400 con 0x7fc1cc0ad890 2024-04-24T09:20:03.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.359+0000 7fc1d7400700 1 -- 172.21.15.90:0/2589545769 --> [v2:172.21.15.114:6816/3222546280,v1:172.21.15.114:6817/3222546280] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc1c0002e60 con 0x7fc1cc0ad890 2024-04-24T09:20:03.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.360+0000 7fc1e2000700 1 -- 172.21.15.90:0/2589545769 <== osd.7 v2:172.21.15.114:6816/3222546280 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc1c0002e60 con 0x7fc1cc0ad890 2024-04-24T09:20:03.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.361+0000 7fc1d7400700 1 -- 172.21.15.90:0/2589545769 >> [v2:172.21.15.114:6816/3222546280,v1:172.21.15.114:6817/3222546280] conn(0x7fc1cc0ad890 msgr2=0x7fc1cc0afd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:03.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.361+0000 7fc1d7400700 1 --2- 172.21.15.90:0/2589545769 >> [v2:172.21.15.114:6816/3222546280,v1:172.21.15.114:6817/3222546280] conn(0x7fc1cc0ad890 0x7fc1cc0afd30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:03.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.361+0000 7fc1d7400700 1 -- 172.21.15.90:0/2589545769 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fc1cc0a3cd0 msgr2=0x7fc1cc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:03.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.361+0000 7fc1d7400700 1 --2- 172.21.15.90:0/2589545769 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fc1cc0a3cd0 0x7fc1cc0a6190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8009fd0 tx=0x7fc1d8009260 comp rx=0 tx=0).stop 2024-04-24T09:20:03.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.361+0000 7fc1d7400700 1 -- 172.21.15.90:0/2589545769 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc1e4108c50 msgr2=0x7fc1e4075870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:03.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.361+0000 7fc1d7400700 1 --2- 172.21.15.90:0/2589545769 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc1e4108c50 0x7fc1e4075870 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc00b070 tx=0x7fc1dc004c80 comp rx=0 tx=0).stop 2024-04-24T09:20:03.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.361+0000 7fc1ea200700 1 -- 172.21.15.90:0/2589545769 reap_dead start 2024-04-24T09:20:03.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.361+0000 7fc1d7400700 1 -- 172.21.15.90:0/2589545769 shutdown_connections 2024-04-24T09:20:03.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.361+0000 7fc1d7400700 1 -- 172.21.15.90:0/2589545769 >> 172.21.15.90:0/2589545769 conn(0x7fc1e4079ca0 msgr2=0x7fc1e41016e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:20:03.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.362+0000 7fc1d7400700 1 -- 172.21.15.90:0/2589545769 shutdown_connections 2024-04-24T09:20:03.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.362+0000 7fc1d7400700 1 -- 172.21.15.90:0/2589545769 wait complete. 2024-04-24T09:20:03.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.672+0000 7f8b64a00700 1 -- 172.21.15.90:0/4149128924 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b60105910 msgr2=0x7f8b60109960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:03.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.672+0000 7f8b64a00700 1 --2- 172.21.15.90:0/4149128924 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b60105910 0x7f8b60109960 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c0099c0 tx=0x7f8b4c009cd0 comp rx=0 tx=0).stop 2024-04-24T09:20:03.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.672+0000 7f8b64a00700 1 -- 172.21.15.90:0/4149128924 shutdown_connections 2024-04-24T09:20:03.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.672+0000 7f8b64a00700 1 --2- 172.21.15.90:0/4149128924 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8b6010a040 0x7f8b60111c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:03.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.672+0000 7f8b64a00700 1 --2- 172.21.15.90:0/4149128924 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b60104f60 0x7f8b60105340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:03.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.672+0000 7f8b64a00700 1 --2- 172.21.15.90:0/4149128924 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b60105910 0x7f8b60109960 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:03.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.672+0000 7f8b64a00700 1 -- 172.21.15.90:0/4149128924 >> 172.21.15.90:0/4149128924 conn(0x7f8b601007f0 msgr2=0x7f8b60102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:20:03.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.673+0000 7f8b64a00700 1 -- 172.21.15.90:0/4149128924 shutdown_connections 2024-04-24T09:20:03.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.673+0000 7f8b64a00700 1 -- 172.21.15.90:0/4149128924 wait complete. 2024-04-24T09:20:03.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.674+0000 7f8b64a00700 1 Processor -- start 2024-04-24T09:20:03.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.674+0000 7f8b64a00700 1 -- start start 2024-04-24T09:20:03.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.675+0000 7f8b64a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b60104f60 0x7f8b6019b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:03.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.675+0000 7f8b64a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8b60105910 0x7f8b6019ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:03.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.675+0000 7f8b64a00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b6010a040 0x7f8b6019fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:03.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.675+0000 7f8b64a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8b60115070 con 0x7f8b60104f60 2024-04-24T09:20:03.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.675+0000 7f8b64a00700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8b60114ef0 con 0x7f8b6010a040 2024-04-24T09:20:03.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.675+0000 7f8b64a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8b601151f0 con 0x7f8b60105910 2024-04-24T09:20:03.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.675+0000 7f8b5de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b60104f60 0x7f8b6019b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:03.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.675+0000 7f8b5de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b60104f60 0x7f8b6019b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58328/0 (socket says 172.21.15.90:58328) 2024-04-24T09:20:03.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.675+0000 7f8b5de00700 1 -- 172.21.15.90:0/1950139600 learned_addr learned my addr 172.21.15.90:0/1950139600 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:20:03.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.675+0000 7f8b5e800700 1 --2- 172.21.15.90:0/1950139600 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b6010a040 0x7f8b6019fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:03.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.675+0000 7f8b5d400700 1 --2- 172.21.15.90:0/1950139600 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8b60105910 0x7f8b6019ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:03.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.676+0000 7f8b5e800700 1 -- 172.21.15.90:0/1950139600 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8b60105910 msgr2=0x7f8b6019ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:03.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.676+0000 7f8b5e800700 1 --2- 172.21.15.90:0/1950139600 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8b60105910 0x7f8b6019ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:03.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.676+0000 7f8b5e800700 1 -- 172.21.15.90:0/1950139600 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b60104f60 msgr2=0x7f8b6019b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:03.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.676+0000 7f8b5e800700 1 --2- 172.21.15.90:0/1950139600 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b60104f60 0x7f8b6019b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:03.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.676+0000 7f8b5e800700 1 -- 172.21.15.90:0/1950139600 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b601a04f0 con 0x7f8b6010a040 2024-04-24T09:20:03.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.676+0000 7f8b5d400700 1 --2- 172.21.15.90:0/1950139600 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8b60105910 0x7f8b6019ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:20:03.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.676+0000 7f8b5e800700 1 --2- 172.21.15.90:0/1950139600 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b6010a040 0x7f8b6019fdb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8b5800efc0 tx=0x7f8b5800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:20:03.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.676+0000 7f8b56a00700 1 -- 172.21.15.90:0/1950139600 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b580092e0 con 0x7f8b6010a040 2024-04-24T09:20:03.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.676+0000 7f8b64a00700 1 -- 172.21.15.90:0/1950139600 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b601a0780 con 0x7f8b6010a040 2024-04-24T09:20:03.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.676+0000 7f8b56a00700 1 -- 172.21.15.90:0/1950139600 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b58004bb0 con 0x7f8b6010a040 2024-04-24T09:20:03.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.676+0000 7f8b56a00700 1 -- 172.21.15.90:0/1950139600 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b58004280 con 0x7f8b6010a040 2024-04-24T09:20:03.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.676+0000 7f8b64a00700 1 -- 172.21.15.90:0/1950139600 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b601a8060 con 0x7f8b6010a040 2024-04-24T09:20:03.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.677+0000 7f8b64a00700 1 -- 172.21.15.90:0/1950139600 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8b60063280 con 0x7f8b6010a040 2024-04-24T09:20:03.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.679+0000 7f8b56a00700 1 -- 172.21.15.90:0/1950139600 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8b5801b030 con 0x7f8b6010a040 2024-04-24T09:20:03.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.681+0000 7f8b56a00700 1 --2- 172.21.15.90:0/1950139600 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f8b440a3d50 0x7f8b440a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:03.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.681+0000 7f8b56a00700 1 -- 172.21.15.90:0/1950139600 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f8b580d0190 con 0x7f8b6010a040 2024-04-24T09:20:03.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.681+0000 7f8b5de00700 1 --2- 172.21.15.90:0/1950139600 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f8b440a3d50 0x7f8b440a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:03.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.682+0000 7f8b56a00700 1 --2- 172.21.15.90:0/1950139600 >> [v2:172.21.15.90:6818/1121461276,v1:172.21.15.90:6819/1121461276] conn(0x7f8b440ad910 0x7f8b440afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:03.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.682+0000 7f8b56a00700 1 -- 172.21.15.90:0/1950139600 --> [v2:172.21.15.90:6818/1121461276,v1:172.21.15.90:6819/1121461276] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8b580048a0 con 0x7f8b440ad910 2024-04-24T09:20:03.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.682+0000 7f8b56a00700 1 -- 172.21.15.90:0/1950139600 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f8b580d0540 con 0x7f8b6010a040 2024-04-24T09:20:03.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.682+0000 7f8b5d400700 1 --2- 172.21.15.90:0/1950139600 >> [v2:172.21.15.90:6818/1121461276,v1:172.21.15.90:6819/1121461276] conn(0x7f8b440ad910 0x7f8b440afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:03.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.682+0000 7f8b5de00700 1 --2- 172.21.15.90:0/1950139600 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f8b440a3d50 0x7f8b440a6210 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8b50009680 tx=0x7f8b50009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:20:03.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.683+0000 7f8b5d400700 1 --2- 172.21.15.90:0/1950139600 >> [v2:172.21.15.90:6818/1121461276,v1:172.21.15.90:6819/1121461276] conn(0x7f8b440ad910 0x7f8b440afdb0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:20:03.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.684+0000 7f8b56a00700 1 -- 172.21.15.90:0/1950139600 <== osd.6 v2:172.21.15.90:6818/1121461276 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8b580048a0 con 0x7f8b440ad910 2024-04-24T09:20:03.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.702+0000 7f8b64a00700 1 -- 172.21.15.90:0/1950139600 --> [v2:172.21.15.90:6818/1121461276,v1:172.21.15.90:6819/1121461276] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8b60063700 con 0x7f8b440ad910 2024-04-24T09:20:03.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.702+0000 7f8b56a00700 1 -- 172.21.15.90:0/1950139600 <== osd.6 v2:172.21.15.90:6818/1121461276 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8b60063700 con 0x7f8b440ad910 2024-04-24T09:20:03.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.703+0000 7f8b64a00700 1 -- 172.21.15.90:0/1950139600 >> [v2:172.21.15.90:6818/1121461276,v1:172.21.15.90:6819/1121461276] conn(0x7f8b440ad910 msgr2=0x7f8b440afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:03.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.703+0000 7f8b64a00700 1 --2- 172.21.15.90:0/1950139600 >> [v2:172.21.15.90:6818/1121461276,v1:172.21.15.90:6819/1121461276] conn(0x7f8b440ad910 0x7f8b440afdb0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:03.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.703+0000 7f8b64a00700 1 -- 172.21.15.90:0/1950139600 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f8b440a3d50 msgr2=0x7f8b440a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:03.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.703+0000 7f8b64a00700 1 --2- 172.21.15.90:0/1950139600 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f8b440a3d50 0x7f8b440a6210 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8b50009680 tx=0x7f8b50009260 comp rx=0 tx=0).stop 2024-04-24T09:20:03.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.703+0000 7f8b64a00700 1 -- 172.21.15.90:0/1950139600 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b6010a040 msgr2=0x7f8b6019fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:03.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.703+0000 7f8b64a00700 1 --2- 172.21.15.90:0/1950139600 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b6010a040 0x7f8b6019fdb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8b5800efc0 tx=0x7f8b5800c3a0 comp rx=0 tx=0).stop 2024-04-24T09:20:03.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.703+0000 7f8b5e800700 1 -- 172.21.15.90:0/1950139600 reap_dead start 2024-04-24T09:20:03.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.703+0000 7f8b64a00700 1 -- 172.21.15.90:0/1950139600 shutdown_connections 2024-04-24T09:20:03.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.703+0000 7f8b64a00700 1 -- 172.21.15.90:0/1950139600 >> 172.21.15.90:0/1950139600 conn(0x7f8b601007f0 msgr2=0x7f8b601013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:20:03.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.703+0000 7f8b64a00700 1 -- 172.21.15.90:0/1950139600 shutdown_connections 2024-04-24T09:20:03.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:03.703+0000 7f8b64a00700 1 -- 172.21.15.90:0/1950139600 wait complete. 2024-04-24T09:20:04.635 INFO:teuthology.orchestra.run.smithi090.stdout:313532612635 2024-04-24T09:20:04.636 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-24T09:20:04.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.640+0000 7f8a59a00700 1 -- 172.21.15.90:0/618108364 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a54074d20 msgr2=0x7f8a54075100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:04.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.640+0000 7f8a59a00700 1 --2- 172.21.15.90:0/618108364 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a54074d20 0x7f8a54075100 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8a44009990 tx=0x7f8a44009ca0 comp rx=0 tx=0).stop 2024-04-24T09:20:04.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.640+0000 7f8a59a00700 1 -- 172.21.15.90:0/618108364 shutdown_connections 2024-04-24T09:20:04.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.640+0000 7f8a59a00700 1 --2- 172.21.15.90:0/618108364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a5410f550 0x7f8a54111940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:04.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.640+0000 7f8a59a00700 1 --2- 172.21.15.90:0/618108364 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a54074d20 0x7f8a54075100 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:04.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.640+0000 7f8a59a00700 1 --2- 172.21.15.90:0/618108364 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a54075640 0x7f8a5410f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:04.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.640+0000 7f8a59a00700 1 -- 172.21.15.90:0/618108364 >> 172.21.15.90:0/618108364 conn(0x7f8a54079c50 msgr2=0x7f8a5407a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:20:04.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.640+0000 7f8a59a00700 1 -- 172.21.15.90:0/618108364 shutdown_connections 2024-04-24T09:20:04.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.640+0000 7f8a59a00700 1 -- 172.21.15.90:0/618108364 wait complete. 2024-04-24T09:20:04.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.642+0000 7f8a59a00700 1 Processor -- start 2024-04-24T09:20:04.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.642+0000 7f8a59a00700 1 -- start start 2024-04-24T09:20:04.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.642+0000 7f8a59a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a54074d20 0x7f8a5419b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:04.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.642+0000 7f8a59a00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a54075640 0x7f8a5419b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:04.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.643+0000 7f8a59a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a5410f550 0x7f8a5419fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:04.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.643+0000 7f8a59a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8a541150a0 con 0x7f8a54074d20 2024-04-24T09:20:04.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.643+0000 7f8a59a00700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8a54114f20 con 0x7f8a54075640 2024-04-24T09:20:04.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.643+0000 7f8a59a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8a54115220 con 0x7f8a5410f550 2024-04-24T09:20:04.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.643+0000 7f8a52000700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a54075640 0x7f8a5419b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:04.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.643+0000 7f8a52000700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a54075640 0x7f8a5419b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.90:33894/0 (socket says 172.21.15.90:33894) 2024-04-24T09:20:04.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.643+0000 7f8a52000700 1 -- 172.21.15.90:0/3101183361 learned_addr learned my addr 172.21.15.90:0/3101183361 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:20:04.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.643+0000 7f8a52a00700 1 --2- 172.21.15.90:0/3101183361 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a54074d20 0x7f8a5419b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:04.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.643+0000 7f8a53400700 1 --2- 172.21.15.90:0/3101183361 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a5410f550 0x7f8a5419fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:04.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.643+0000 7f8a52000700 1 -- 172.21.15.90:0/3101183361 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a5410f550 msgr2=0x7f8a5419fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:04.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.643+0000 7f8a52000700 1 --2- 172.21.15.90:0/3101183361 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a5410f550 0x7f8a5419fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:04.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.643+0000 7f8a52000700 1 -- 172.21.15.90:0/3101183361 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a54074d20 msgr2=0x7f8a5419b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:04.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.643+0000 7f8a52000700 1 --2- 172.21.15.90:0/3101183361 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a54074d20 0x7f8a5419b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:04.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.643+0000 7f8a52000700 1 -- 172.21.15.90:0/3101183361 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a541a0480 con 0x7f8a54075640 2024-04-24T09:20:04.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.644+0000 7f8a52a00700 1 --2- 172.21.15.90:0/3101183361 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a54074d20 0x7f8a5419b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:20:04.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.644+0000 7f8a52000700 1 --2- 172.21.15.90:0/3101183361 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a54075640 0x7f8a5419b9b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8a4000ea50 tx=0x7f8a4000ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:20:04.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.645+0000 7f8a4b400700 1 -- 172.21.15.90:0/3101183361 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4000cbd0 con 0x7f8a54075640 2024-04-24T09:20:04.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.645+0000 7f8a4b400700 1 -- 172.21.15.90:0/3101183361 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a4000cd70 con 0x7f8a54075640 2024-04-24T09:20:04.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.645+0000 7f8a4b400700 1 -- 172.21.15.90:0/3101183361 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a400187e0 con 0x7f8a54075640 2024-04-24T09:20:04.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.645+0000 7f8a59a00700 1 -- 172.21.15.90:0/3101183361 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a541a0770 con 0x7f8a54075640 2024-04-24T09:20:04.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.645+0000 7f8a59a00700 1 -- 172.21.15.90:0/3101183361 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a541a7ff0 con 0x7f8a54075640 2024-04-24T09:20:04.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.646+0000 7f8a59a00700 1 -- 172.21.15.90:0/3101183361 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8a541006a0 con 0x7f8a54075640 2024-04-24T09:20:04.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.648+0000 7f8a4b400700 1 -- 172.21.15.90:0/3101183361 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8a4001f020 con 0x7f8a54075640 2024-04-24T09:20:04.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.650+0000 7f8a4b400700 1 --2- 172.21.15.90:0/3101183361 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f8a380a3d50 0x7f8a380a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:04.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.650+0000 7f8a4b400700 1 -- 172.21.15.90:0/3101183361 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f8a400d05e0 con 0x7f8a54075640 2024-04-24T09:20:04.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.650+0000 7f8a52a00700 1 --2- 172.21.15.90:0/3101183361 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f8a380a3d50 0x7f8a380a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:04.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.650+0000 7f8a4b400700 1 --2- 172.21.15.90:0/3101183361 >> [v2:172.21.15.90:6826/1307318076,v1:172.21.15.90:6827/1307318076] conn(0x7f8a380ad910 0x7f8a380afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:04.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.650+0000 7f8a4b400700 1 -- 172.21.15.90:0/3101183361 --> [v2:172.21.15.90:6826/1307318076,v1:172.21.15.90:6827/1307318076] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8a40009500 con 0x7f8a380ad910 2024-04-24T09:20:04.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.650+0000 7f8a4b400700 1 -- 172.21.15.90:0/3101183361 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f8a400d7050 con 0x7f8a54075640 2024-04-24T09:20:04.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.651+0000 7f8a53400700 1 --2- 172.21.15.90:0/3101183361 >> [v2:172.21.15.90:6826/1307318076,v1:172.21.15.90:6827/1307318076] conn(0x7f8a380ad910 0x7f8a380afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:04.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.651+0000 7f8a52a00700 1 --2- 172.21.15.90:0/3101183361 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f8a380a3d50 0x7f8a380a6210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8a44006010 tx=0x7f8a4400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:20:04.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.651+0000 7f8a53400700 1 --2- 172.21.15.90:0/3101183361 >> [v2:172.21.15.90:6826/1307318076,v1:172.21.15.90:6827/1307318076] conn(0x7f8a380ad910 0x7f8a380afdb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:20:04.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.652+0000 7f8a4b400700 1 -- 172.21.15.90:0/3101183361 <== osd.9 v2:172.21.15.90:6826/1307318076 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8a40009500 con 0x7f8a380ad910 2024-04-24T09:20:04.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.670+0000 7f8a59a00700 1 -- 172.21.15.90:0/3101183361 --> [v2:172.21.15.90:6826/1307318076,v1:172.21.15.90:6827/1307318076] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8a54076400 con 0x7f8a380ad910 2024-04-24T09:20:04.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.671+0000 7f8a4b400700 1 -- 172.21.15.90:0/3101183361 <== osd.9 v2:172.21.15.90:6826/1307318076 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8a54076400 con 0x7f8a380ad910 2024-04-24T09:20:04.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.671+0000 7f8a59a00700 1 -- 172.21.15.90:0/3101183361 >> [v2:172.21.15.90:6826/1307318076,v1:172.21.15.90:6827/1307318076] conn(0x7f8a380ad910 msgr2=0x7f8a380afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:04.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.671+0000 7f8a59a00700 1 --2- 172.21.15.90:0/3101183361 >> [v2:172.21.15.90:6826/1307318076,v1:172.21.15.90:6827/1307318076] conn(0x7f8a380ad910 0x7f8a380afdb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:04.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.672+0000 7f8a59a00700 1 -- 172.21.15.90:0/3101183361 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f8a380a3d50 msgr2=0x7f8a380a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:04.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.672+0000 7f8a59a00700 1 --2- 172.21.15.90:0/3101183361 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f8a380a3d50 0x7f8a380a6210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8a44006010 tx=0x7f8a4400b260 comp rx=0 tx=0).stop 2024-04-24T09:20:04.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.672+0000 7f8a59a00700 1 -- 172.21.15.90:0/3101183361 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a54075640 msgr2=0x7f8a5419b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:04.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.672+0000 7f8a59a00700 1 --2- 172.21.15.90:0/3101183361 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a54075640 0x7f8a5419b9b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8a4000ea50 tx=0x7f8a4000ed60 comp rx=0 tx=0).stop 2024-04-24T09:20:04.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.672+0000 7f8a53400700 1 -- 172.21.15.90:0/3101183361 reap_dead start 2024-04-24T09:20:04.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.672+0000 7f8a59a00700 1 -- 172.21.15.90:0/3101183361 shutdown_connections 2024-04-24T09:20:04.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.672+0000 7f8a59a00700 1 -- 172.21.15.90:0/3101183361 >> 172.21.15.90:0/3101183361 conn(0x7f8a54079c50 msgr2=0x7f8a540ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:20:04.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.672+0000 7f8a59a00700 1 -- 172.21.15.90:0/3101183361 shutdown_connections 2024-04-24T09:20:04.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:04.672+0000 7f8a59a00700 1 -- 172.21.15.90:0/3101183361 wait complete. 2024-04-24T09:20:04.743 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:04 smithi090 ceph-mon[27825]: pgmap v351: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 3 op/s; 160 B/s, 0 keys/s, 1 objects/s recovering 2024-04-24T09:20:04.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:04 smithi114 ceph-mon[32501]: pgmap v351: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 3 op/s; 160 B/s, 0 keys/s, 1 objects/s recovering 2024-04-24T09:20:05.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:04 smithi149 ceph-mon[32592]: pgmap v351: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 3 op/s; 160 B/s, 0 keys/s, 1 objects/s recovering 2024-04-24T09:20:06.032 INFO:teuthology.orchestra.run.smithi090.stdout:223338299441 2024-04-24T09:20:06.033 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-24T09:20:06.385 INFO:teuthology.orchestra.run.smithi090.stdout:146028888129 2024-04-24T09:20:06.386 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-24T09:20:06.841 INFO:teuthology.orchestra.run.smithi090.stdout:34359738455 2024-04-24T09:20:06.842 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-24T09:20:06.910 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:06 smithi090 ceph-mon[27825]: pgmap v352: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 4 op/s; 160 B/s, 0 keys/s, 1 objects/s recovering 2024-04-24T09:20:06.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:06 smithi114 ceph-mon[32501]: pgmap v352: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 4 op/s; 160 B/s, 0 keys/s, 1 objects/s recovering 2024-04-24T09:20:06.973 INFO:teuthology.orchestra.run.smithi090.stdout:257698037803 2024-04-24T09:20:06.973 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-24T09:20:07.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:06 smithi149 ceph-mon[32592]: pgmap v352: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 4 op/s; 160 B/s, 0 keys/s, 1 objects/s recovering 2024-04-24T09:20:07.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.567+0000 7fc73c800700 1 -- 172.21.15.90:0/1309353055 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc734103500 msgr2=0x7fc73410f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:07.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.567+0000 7fc73c800700 1 --2- 172.21.15.90:0/1309353055 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc734103500 0x7fc73410f960 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc73000b0a0 tx=0x7fc73000b3b0 comp rx=0 tx=0).stop 2024-04-24T09:20:07.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.567+0000 7fc73c800700 1 -- 172.21.15.90:0/1309353055 shutdown_connections 2024-04-24T09:20:07.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.567+0000 7fc73c800700 1 --2- 172.21.15.90:0/1309353055 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc734103500 0x7fc73410f960 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:07.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.567+0000 7fc73c800700 1 --2- 172.21.15.90:0/1309353055 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc734108b90 0x7fc734108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:07.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.567+0000 7fc73c800700 1 --2- 172.21.15.90:0/1309353055 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc734102b40 0x7fc734102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:07.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.567+0000 7fc73c800700 1 -- 172.21.15.90:0/1309353055 >> 172.21.15.90:0/1309353055 conn(0x7fc734079c50 msgr2=0x7fc73407a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:20:07.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.567+0000 7fc73c800700 1 -- 172.21.15.90:0/1309353055 shutdown_connections 2024-04-24T09:20:07.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.568+0000 7fc73c800700 1 -- 172.21.15.90:0/1309353055 wait complete. 2024-04-24T09:20:07.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.569+0000 7fc73c800700 1 Processor -- start 2024-04-24T09:20:07.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.569+0000 7fc73c800700 1 -- start start 2024-04-24T09:20:07.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.569+0000 7fc73c800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc734102b40 0x7fc734074df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:07.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.569+0000 7fc73c800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc734103500 0x7fc734075330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:07.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.569+0000 7fc73c800700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc734108b90 0x7fc734075870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:07.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.569+0000 7fc73c800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc734112e40 con 0x7fc734103500 2024-04-24T09:20:07.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.569+0000 7fc73c800700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc734112cc0 con 0x7fc734108b90 2024-04-24T09:20:07.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.569+0000 7fc73c800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc734112fc0 con 0x7fc734102b40 2024-04-24T09:20:07.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.570+0000 7fc739e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc734102b40 0x7fc734074df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:07.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.570+0000 7fc739e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc734102b40 0x7fc734074df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:48908/0 (socket says 172.21.15.90:48908) 2024-04-24T09:20:07.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.570+0000 7fc739e00700 1 -- 172.21.15.90:0/809100611 learned_addr learned my addr 172.21.15.90:0/809100611 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:20:07.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.570+0000 7fc73a800700 1 --2- 172.21.15.90:0/809100611 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc734108b90 0x7fc734075870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:07.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.570+0000 7fc739400700 1 --2- 172.21.15.90:0/809100611 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc734103500 0x7fc734075330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:07.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.570+0000 7fc739e00700 1 -- 172.21.15.90:0/809100611 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc734108b90 msgr2=0x7fc734075870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:07.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.570+0000 7fc739e00700 1 --2- 172.21.15.90:0/809100611 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc734108b90 0x7fc734075870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:07.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.570+0000 7fc739e00700 1 -- 172.21.15.90:0/809100611 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc734103500 msgr2=0x7fc734075330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:07.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.570+0000 7fc739e00700 1 --2- 172.21.15.90:0/809100611 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc734103500 0x7fc734075330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:07.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.570+0000 7fc739e00700 1 -- 172.21.15.90:0/809100611 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7341a77e0 con 0x7fc734102b40 2024-04-24T09:20:07.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.570+0000 7fc739e00700 1 --2- 172.21.15.90:0/809100611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc734102b40 0x7fc734074df0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc72800e9a0 tx=0x7fc72800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:20:07.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.571+0000 7fc72ea00700 1 -- 172.21.15.90:0/809100611 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72800ca10 con 0x7fc734102b40 2024-04-24T09:20:07.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.571+0000 7fc72ea00700 1 -- 172.21.15.90:0/809100611 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc72800cbb0 con 0x7fc734102b40 2024-04-24T09:20:07.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.571+0000 7fc73c800700 1 -- 172.21.15.90:0/809100611 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7341a7ad0 con 0x7fc734102b40 2024-04-24T09:20:07.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.571+0000 7fc72ea00700 1 -- 172.21.15.90:0/809100611 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc728018680 con 0x7fc734102b40 2024-04-24T09:20:07.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.571+0000 7fc73c800700 1 -- 172.21.15.90:0/809100611 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7341a8010 con 0x7fc734102b40 2024-04-24T09:20:07.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.572+0000 7fc73c800700 1 -- 172.21.15.90:0/809100611 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc7340631c0 con 0x7fc734102b40 2024-04-24T09:20:07.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.574+0000 7fc72ea00700 1 -- 172.21.15.90:0/809100611 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc728010430 con 0x7fc734102b40 2024-04-24T09:20:07.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.575+0000 7fc72ea00700 1 --2- 172.21.15.90:0/809100611 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fc71c0a3cd0 0x7fc71c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:07.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.575+0000 7fc72ea00700 1 -- 172.21.15.90:0/809100611 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fc7280d0f50 con 0x7fc734102b40 2024-04-24T09:20:07.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.575+0000 7fc739400700 1 --2- 172.21.15.90:0/809100611 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fc71c0a3cd0 0x7fc71c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:07.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.576+0000 7fc72ea00700 1 --2- 172.21.15.90:0/809100611 >> [v2:172.21.15.149:6824/1737275166,v1:172.21.15.149:6825/1737275166] conn(0x7fc71c0ad890 0x7fc71c0afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:07.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.576+0000 7fc72ea00700 1 -- 172.21.15.90:0/809100611 --> [v2:172.21.15.149:6824/1737275166,v1:172.21.15.149:6825/1737275166] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc728027e50 con 0x7fc71c0ad890 2024-04-24T09:20:07.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.576+0000 7fc72ea00700 1 -- 172.21.15.90:0/809100611 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fc7280d1300 con 0x7fc734102b40 2024-04-24T09:20:07.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.576+0000 7fc73a800700 1 --2- 172.21.15.90:0/809100611 >> [v2:172.21.15.149:6824/1737275166,v1:172.21.15.149:6825/1737275166] conn(0x7fc71c0ad890 0x7fc71c0afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:07.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.576+0000 7fc739400700 1 --2- 172.21.15.90:0/809100611 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fc71c0a3cd0 0x7fc71c0a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc73407a1e0 tx=0x7fc724009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:20:07.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.577+0000 7fc73a800700 1 --2- 172.21.15.90:0/809100611 >> [v2:172.21.15.149:6824/1737275166,v1:172.21.15.149:6825/1737275166] conn(0x7fc71c0ad890 0x7fc71c0afd30 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:20:07.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.578+0000 7fc72ea00700 1 -- 172.21.15.90:0/809100611 <== osd.11 v2:172.21.15.149:6824/1737275166 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc728027e50 con 0x7fc71c0ad890 2024-04-24T09:20:07.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.595+0000 7fc73c800700 1 -- 172.21.15.90:0/809100611 --> [v2:172.21.15.149:6824/1737275166,v1:172.21.15.149:6825/1737275166] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc734063640 con 0x7fc71c0ad890 2024-04-24T09:20:07.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.596+0000 7fc72ea00700 1 -- 172.21.15.90:0/809100611 <== osd.11 v2:172.21.15.149:6824/1737275166 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc734063640 con 0x7fc71c0ad890 2024-04-24T09:20:07.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.596+0000 7fc73c800700 1 -- 172.21.15.90:0/809100611 >> [v2:172.21.15.149:6824/1737275166,v1:172.21.15.149:6825/1737275166] conn(0x7fc71c0ad890 msgr2=0x7fc71c0afd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:07.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.596+0000 7fc73c800700 1 --2- 172.21.15.90:0/809100611 >> [v2:172.21.15.149:6824/1737275166,v1:172.21.15.149:6825/1737275166] conn(0x7fc71c0ad890 0x7fc71c0afd30 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:07.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.597+0000 7fc73c800700 1 -- 172.21.15.90:0/809100611 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fc71c0a3cd0 msgr2=0x7fc71c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:07.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.597+0000 7fc73c800700 1 --2- 172.21.15.90:0/809100611 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fc71c0a3cd0 0x7fc71c0a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc73407a1e0 tx=0x7fc724009260 comp rx=0 tx=0).stop 2024-04-24T09:20:07.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.597+0000 7fc73c800700 1 -- 172.21.15.90:0/809100611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc734102b40 msgr2=0x7fc734074df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:07.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.597+0000 7fc73c800700 1 --2- 172.21.15.90:0/809100611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc734102b40 0x7fc734074df0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc72800e9a0 tx=0x7fc72800ecb0 comp rx=0 tx=0).stop 2024-04-24T09:20:07.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.597+0000 7fc73a800700 1 -- 172.21.15.90:0/809100611 reap_dead start 2024-04-24T09:20:07.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.597+0000 7fc73c800700 1 -- 172.21.15.90:0/809100611 shutdown_connections 2024-04-24T09:20:07.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.597+0000 7fc73c800700 1 -- 172.21.15.90:0/809100611 >> 172.21.15.90:0/809100611 conn(0x7fc734079c50 msgr2=0x7fc7341016c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:20:07.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.597+0000 7fc73c800700 1 -- 172.21.15.90:0/809100611 shutdown_connections 2024-04-24T09:20:07.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:07.597+0000 7fc73c800700 1 -- 172.21.15.90:0/809100611 wait complete. 2024-04-24T09:20:08.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:08 smithi114 ceph-mon[32501]: pgmap v353: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 5 op/s; 129 B/s, 0 keys/s, 1 objects/s recovering 2024-04-24T09:20:08.997 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:08 smithi090 ceph-mon[27825]: pgmap v353: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 5 op/s; 129 B/s, 0 keys/s, 1 objects/s recovering 2024-04-24T09:20:09.065 INFO:teuthology.orchestra.run.smithi090.stdout:292057776165 2024-04-24T09:20:09.065 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-24T09:20:09.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:08 smithi149 ceph-mon[32592]: pgmap v353: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 5 op/s; 129 B/s, 0 keys/s, 1 objects/s recovering 2024-04-24T09:20:09.375 INFO:teuthology.orchestra.run.smithi090.stdout:347892351003 2024-04-24T09:20:09.376 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-24T09:20:09.650 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:20:09.692 INFO:teuthology.orchestra.run.smithi090.stdout:201863462967 2024-04-24T09:20:09.692 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-24T09:20:10.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:10 smithi090 ceph-mon[27825]: pgmap v354: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 6 op/s; 117 B/s, 0 keys/s, 1 objects/s recovering 2024-04-24T09:20:10.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:10 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:20:10.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:10 smithi114 ceph-mon[32501]: pgmap v354: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 6 op/s; 117 B/s, 0 keys/s, 1 objects/s recovering 2024-04-24T09:20:10.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:10 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:20:11.088 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:10 smithi149 ceph-mon[32592]: pgmap v354: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 6 op/s; 117 B/s, 0 keys/s, 1 objects/s recovering 2024-04-24T09:20:11.088 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:10 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:20:11.447 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:20:11.468 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:20:11.470 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:20:11.472 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:20:11.475 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:20:11.479 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:20:11.486 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:20:11.489 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:20:11.493 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:20:12.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:12 smithi114 ceph-mon[32501]: pgmap v355: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 9 B/s, 0 keys/s, 0 objects/s recovering 2024-04-24T09:20:12.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:12 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:20:12.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:12 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:20:12.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:12 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:20:12.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:12 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:20:13.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:12 smithi090 ceph-mon[27825]: pgmap v355: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 9 B/s, 0 keys/s, 0 objects/s recovering 2024-04-24T09:20:13.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:12 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:20:13.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:12 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:20:13.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:12 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:20:13.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:12 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:20:13.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:12 smithi149 ceph-mon[32592]: pgmap v355: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 9 B/s, 0 keys/s, 0 objects/s recovering 2024-04-24T09:20:13.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:12 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:20:13.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:12 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:20:13.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:12 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:20:13.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:12 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:20:14.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:14 smithi090 ceph-mon[27825]: pgmap v356: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 9 B/s, 0 keys/s, 0 objects/s recovering 2024-04-24T09:20:14.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:14 smithi114 ceph-mon[32501]: pgmap v356: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 9 B/s, 0 keys/s, 0 objects/s recovering 2024-04-24T09:20:15.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:14 smithi149 ceph-mon[32592]: pgmap v356: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 9 B/s, 0 keys/s, 0 objects/s recovering 2024-04-24T09:20:16.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:16 smithi114 ceph-mon[32501]: pgmap v357: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-04-24T09:20:16.937 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:16 smithi090 ceph-mon[27825]: pgmap v357: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-04-24T09:20:17.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:16 smithi149 ceph-mon[32592]: pgmap v357: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-04-24T09:20:17.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:17 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/2923832572' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-24T09:20:18.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:17 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/2923832572' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-24T09:20:18.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:17 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/2923832572' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-24T09:20:18.499 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:20:18.733 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:18 smithi090 ceph-mon[27825]: pgmap v358: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-24T09:20:19.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:18 smithi149 ceph-mon[32592]: pgmap v358: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-24T09:20:19.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:18 smithi114 ceph-mon[32501]: pgmap v358: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-24T09:20:19.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.601+0000 7fdca0600700 1 -- 172.21.15.90:0/40978041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc98103500 msgr2=0x7fdc9810f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:19.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.601+0000 7fdca0600700 1 --2- 172.21.15.90:0/40978041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc98103500 0x7fdc9810f960 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdc90009f90 tx=0x7fdc9000b330 comp rx=0 tx=0).stop 2024-04-24T09:20:19.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.601+0000 7fdca0600700 1 -- 172.21.15.90:0/40978041 shutdown_connections 2024-04-24T09:20:19.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.601+0000 7fdca0600700 1 --2- 172.21.15.90:0/40978041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc98103500 0x7fdc9810f960 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:19.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.601+0000 7fdca0600700 1 --2- 172.21.15.90:0/40978041 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc98108b90 0x7fdc98108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:19.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.601+0000 7fdca0600700 1 --2- 172.21.15.90:0/40978041 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdc98102b40 0x7fdc98102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:19.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.601+0000 7fdca0600700 1 -- 172.21.15.90:0/40978041 >> 172.21.15.90:0/40978041 conn(0x7fdc98079c50 msgr2=0x7fdc9807a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:20:19.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.601+0000 7fdca0600700 1 -- 172.21.15.90:0/40978041 shutdown_connections 2024-04-24T09:20:19.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.601+0000 7fdca0600700 1 -- 172.21.15.90:0/40978041 wait complete. 2024-04-24T09:20:19.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.603+0000 7fdca0600700 1 Processor -- start 2024-04-24T09:20:19.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.603+0000 7fdca0600700 1 -- start start 2024-04-24T09:20:19.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.603+0000 7fdca0600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc98102b40 0x7fdc9819b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:19.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.603+0000 7fdca0600700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdc98103500 0x7fdc9819ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:19.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.604+0000 7fdca0600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc98108b90 0x7fdc9819fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:19.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.604+0000 7fdca0600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdc98112e60 con 0x7fdc98108b90 2024-04-24T09:20:19.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.604+0000 7fdca0600700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fdc98112ce0 con 0x7fdc98103500 2024-04-24T09:20:19.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.604+0000 7fdca0600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fdc98112fe0 con 0x7fdc98102b40 2024-04-24T09:20:19.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.604+0000 7fdc9e600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc98108b90 0x7fdc9819fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:19.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.604+0000 7fdc9e600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc98108b90 0x7fdc9819fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46762/0 (socket says 172.21.15.90:46762) 2024-04-24T09:20:19.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.604+0000 7fdc9dc00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc98102b40 0x7fdc9819b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:19.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.604+0000 7fdc9e600700 1 -- 172.21.15.90:0/2923832572 learned_addr learned my addr 172.21.15.90:0/2923832572 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:20:19.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.604+0000 7fdc9d200700 1 --2- 172.21.15.90:0/2923832572 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdc98103500 0x7fdc9819ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:19.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.604+0000 7fdc9e600700 1 -- 172.21.15.90:0/2923832572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc98102b40 msgr2=0x7fdc9819b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:19.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.604+0000 7fdc9e600700 1 --2- 172.21.15.90:0/2923832572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc98102b40 0x7fdc9819b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:19.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.604+0000 7fdc9e600700 1 -- 172.21.15.90:0/2923832572 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdc98103500 msgr2=0x7fdc9819ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:19.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.604+0000 7fdc9e600700 1 --2- 172.21.15.90:0/2923832572 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdc98103500 0x7fdc9819ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:19.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.604+0000 7fdc9e600700 1 -- 172.21.15.90:0/2923832572 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc981a0520 con 0x7fdc98108b90 2024-04-24T09:20:19.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.604+0000 7fdc9d200700 1 --2- 172.21.15.90:0/2923832572 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdc98103500 0x7fdc9819ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:20:19.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.605+0000 7fdc9e600700 1 --2- 172.21.15.90:0/2923832572 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc98108b90 0x7fdc9819fde0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdc90009550 tx=0x7fdc9000cfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:20:19.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.605+0000 7fdc96000700 1 -- 172.21.15.90:0/2923832572 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc90021070 con 0x7fdc98108b90 2024-04-24T09:20:19.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.605+0000 7fdca0600700 1 -- 172.21.15.90:0/2923832572 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc981a07b0 con 0x7fdc98108b90 2024-04-24T09:20:19.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.605+0000 7fdc96000700 1 -- 172.21.15.90:0/2923832572 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc90007ac0 con 0x7fdc98108b90 2024-04-24T09:20:19.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.605+0000 7fdc96000700 1 -- 172.21.15.90:0/2923832572 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc900052a0 con 0x7fdc98108b90 2024-04-24T09:20:19.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.605+0000 7fdca0600700 1 -- 172.21.15.90:0/2923832572 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc981a8090 con 0x7fdc98108b90 2024-04-24T09:20:19.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.606+0000 7fdca0600700 1 -- 172.21.15.90:0/2923832572 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc980697e0 con 0x7fdc98108b90 2024-04-24T09:20:19.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.609+0000 7fdc96000700 1 -- 172.21.15.90:0/2923832572 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fdc90024070 con 0x7fdc98108b90 2024-04-24T09:20:19.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.610+0000 7fdc96000700 1 --2- 172.21.15.90:0/2923832572 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fdc800a3cd0 0x7fdc800a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:19.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.610+0000 7fdc96000700 1 -- 172.21.15.90:0/2923832572 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fdc9000f020 con 0x7fdc98108b90 2024-04-24T09:20:19.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.610+0000 7fdc9dc00700 1 --2- 172.21.15.90:0/2923832572 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fdc800a3cd0 0x7fdc800a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:19.782 INFO:teuthology.orchestra.run.smithi090.stdout:55834574934 2024-04-24T09:20:19.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.611+0000 7fdc9dc00700 1 --2- 172.21.15.90:0/2923832572 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fdc800a3cd0 0x7fdc800a6190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c009be0 tx=0x7fdc8c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:20:19.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.613+0000 7fdc96000700 1 -- 172.21.15.90:0/2923832572 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdc9002b020 con 0x7fdc98108b90 2024-04-24T09:20:19.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.850+0000 7fdca0600700 1 -- 172.21.15.90:0/2923832572 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fdc981a8260 con 0x7fdc98108b90 2024-04-24T09:20:19.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.851+0000 7fdc96000700 1 -- 172.21.15.90:0/2923832572 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdc9009e7f0 con 0x7fdc98108b90 2024-04-24T09:20:19.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.854+0000 7fdca0600700 1 -- 172.21.15.90:0/2923832572 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fdc800a3cd0 msgr2=0x7fdc800a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:19.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.854+0000 7fdca0600700 1 --2- 172.21.15.90:0/2923832572 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fdc800a3cd0 0x7fdc800a6190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c009be0 tx=0x7fdc8c009260 comp rx=0 tx=0).stop 2024-04-24T09:20:19.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.854+0000 7fdca0600700 1 -- 172.21.15.90:0/2923832572 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc98108b90 msgr2=0x7fdc9819fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:19.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.854+0000 7fdca0600700 1 --2- 172.21.15.90:0/2923832572 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc98108b90 0x7fdc9819fde0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdc90009550 tx=0x7fdc9000cfa0 comp rx=0 tx=0).stop 2024-04-24T09:20:19.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.854+0000 7fdca0600700 1 -- 172.21.15.90:0/2923832572 shutdown_connections 2024-04-24T09:20:19.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.854+0000 7fdca0600700 1 --2- 172.21.15.90:0/2923832572 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fdc800a3cd0 0x7fdc800a6190 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:19.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.854+0000 7fdca0600700 1 --2- 172.21.15.90:0/2923832572 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc98108b90 0x7fdc9819fde0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:19.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.854+0000 7fdca0600700 1 --2- 172.21.15.90:0/2923832572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc98102b40 0x7fdc9819b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:19.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.854+0000 7fdca0600700 1 --2- 172.21.15.90:0/2923832572 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdc98103500 0x7fdc9819ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:19.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.854+0000 7fdca0600700 1 -- 172.21.15.90:0/2923832572 >> 172.21.15.90:0/2923832572 conn(0x7fdc98079c50 msgr2=0x7fdc9810df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:20:19.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.854+0000 7fdca0600700 1 -- 172.21.15.90:0/2923832572 shutdown_connections 2024-04-24T09:20:19.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:16.854+0000 7fdca0600700 1 -- 172.21.15.90:0/2923832572 wait complete. 2024-04-24T09:20:20.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:20 smithi090 ceph-mon[27825]: pgmap v359: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-24T09:20:21.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:20 smithi149 ceph-mon[32592]: pgmap v359: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-24T09:20:21.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:20 smithi114 ceph-mon[32501]: pgmap v359: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-24T09:20:22.119 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:20:23.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:22 smithi090 ceph-mon[27825]: pgmap v360: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T09:20:23.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:22 smithi149 ceph-mon[32592]: pgmap v360: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T09:20:23.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:22 smithi114 ceph-mon[32501]: pgmap v360: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T09:20:24.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:24 smithi090 ceph-mon[27825]: pgmap v361: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:20:25.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:24 smithi149 ceph-mon[32592]: pgmap v361: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:20:25.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:24 smithi114 ceph-mon[32501]: pgmap v361: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:20:25.226 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574930 got 55834574934 for osd.1 2024-04-24T09:20:25.226 DEBUG:teuthology.parallel:result is None 2024-04-24T09:20:25.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.477+0000 7f89c2800700 1 -- 172.21.15.90:0/2888567807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89bc108ad0 msgr2=0x7f89bc108eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:25.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.477+0000 7f89c2800700 1 --2- 172.21.15.90:0/2888567807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89bc108ad0 0x7f89bc108eb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f89b0009f90 tx=0x7f89b000b330 comp rx=0 tx=0).stop 2024-04-24T09:20:25.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.477+0000 7f89c2800700 1 -- 172.21.15.90:0/2888567807 shutdown_connections 2024-04-24T09:20:25.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.477+0000 7f89c2800700 1 --2- 172.21.15.90:0/2888567807 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89bc06ba20 0x7f89bc10f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:25.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.477+0000 7f89c2800700 1 --2- 172.21.15.90:0/2888567807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89bc108ad0 0x7f89bc108eb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:25.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.477+0000 7f89c2800700 1 --2- 172.21.15.90:0/2888567807 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89bc06b060 0x7f89bc06b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:25.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.477+0000 7f89c2800700 1 -- 172.21.15.90:0/2888567807 >> 172.21.15.90:0/2888567807 conn(0x7f89bc078830 msgr2=0x7f89bc078c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:20:25.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.478+0000 7f89c2800700 1 -- 172.21.15.90:0/2888567807 shutdown_connections 2024-04-24T09:20:25.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.478+0000 7f89c2800700 1 -- 172.21.15.90:0/2888567807 wait complete. 2024-04-24T09:20:25.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.479+0000 7f89c2800700 1 Processor -- start 2024-04-24T09:20:25.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.479+0000 7f89c2800700 1 -- start start 2024-04-24T09:20:25.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.480+0000 7f89c2800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89bc06b060 0x7f89bc101c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:25.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.480+0000 7f89c2800700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89bc06ba20 0x7f89bc1021c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:25.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.480+0000 7f89c2800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89bc108ad0 0x7f89bc102700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:25.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.480+0000 7f89c2800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc112dc0 con 0x7f89bc06b060 2024-04-24T09:20:25.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.480+0000 7f89c2800700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc112c40 con 0x7f89bc06ba20 2024-04-24T09:20:25.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.480+0000 7f89c2800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc112f40 con 0x7f89bc108ad0 2024-04-24T09:20:25.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.480+0000 7f89c1400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89bc06b060 0x7f89bc101c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:25.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.480+0000 7f89c0a00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89bc06ba20 0x7f89bc1021c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:25.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.480+0000 7f89c0a00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89bc06ba20 0x7f89bc1021c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.90:44558/0 (socket says 172.21.15.90:44558) 2024-04-24T09:20:25.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.480+0000 7f89c0a00700 1 -- 172.21.15.90:0/4154054443 learned_addr learned my addr 172.21.15.90:0/4154054443 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:20:25.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.480+0000 7f89c1400700 1 --2- 172.21.15.90:0/4154054443 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89bc06b060 0x7f89bc101c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46778/0 (socket says 172.21.15.90:46778) 2024-04-24T09:20:25.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.480+0000 7f89c1e00700 1 --2- 172.21.15.90:0/4154054443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89bc108ad0 0x7f89bc102700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:25.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.480+0000 7f89c0a00700 1 -- 172.21.15.90:0/4154054443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89bc108ad0 msgr2=0x7f89bc102700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:25.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.480+0000 7f89c0a00700 1 --2- 172.21.15.90:0/4154054443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89bc108ad0 0x7f89bc102700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:25.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.480+0000 7f89c0a00700 1 -- 172.21.15.90:0/4154054443 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89bc06b060 msgr2=0x7f89bc101c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:25.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.480+0000 7f89c0a00700 1 --2- 172.21.15.90:0/4154054443 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89bc06b060 0x7f89bc101c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:25.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.480+0000 7f89c0a00700 1 -- 172.21.15.90:0/4154054443 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89bc1a76d0 con 0x7f89bc06ba20 2024-04-24T09:20:25.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.480+0000 7f89c1400700 1 --2- 172.21.15.90:0/4154054443 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89bc06b060 0x7f89bc101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:20:25.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.481+0000 7f89c0a00700 1 --2- 172.21.15.90:0/4154054443 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89bc06ba20 0x7f89bc1021c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f89a800e9c0 tx=0x7f89a800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:20:25.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.481+0000 7f89ba000700 1 -- 172.21.15.90:0/4154054443 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a800cbd0 con 0x7f89bc06ba20 2024-04-24T09:20:25.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.481+0000 7f89ba000700 1 -- 172.21.15.90:0/4154054443 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89a800ee60 con 0x7f89bc06ba20 2024-04-24T09:20:25.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.481+0000 7f89c2800700 1 -- 172.21.15.90:0/4154054443 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89bc1a79c0 con 0x7f89bc06ba20 2024-04-24T09:20:25.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.481+0000 7f89ba000700 1 -- 172.21.15.90:0/4154054443 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a8005630 con 0x7f89bc06ba20 2024-04-24T09:20:25.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.481+0000 7f89c2800700 1 -- 172.21.15.90:0/4154054443 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89bc1a7eb0 con 0x7f89bc06ba20 2024-04-24T09:20:25.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.482+0000 7f89c2800700 1 -- 172.21.15.90:0/4154054443 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89bc10afb0 con 0x7f89bc06ba20 2024-04-24T09:20:25.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.484+0000 7f89ba000700 1 -- 172.21.15.90:0/4154054443 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f89a8004750 con 0x7f89bc06ba20 2024-04-24T09:20:25.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.486+0000 7f89ba000700 1 --2- 172.21.15.90:0/4154054443 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f89a40a3cd0 0x7f89a40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:25.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.486+0000 7f89ba000700 1 -- 172.21.15.90:0/4154054443 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f89a80d09b0 con 0x7f89bc06ba20 2024-04-24T09:20:25.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.486+0000 7f89c1400700 1 --2- 172.21.15.90:0/4154054443 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f89a40a3cd0 0x7f89a40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:25.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.487+0000 7f89c1400700 1 --2- 172.21.15.90:0/4154054443 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f89a40a3cd0 0x7f89a40a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f89b000b6c0 tx=0x7f89b00094b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:20:25.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.488+0000 7f89ba000700 1 -- 172.21.15.90:0/4154054443 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f89a809d1f0 con 0x7f89bc06ba20 2024-04-24T09:20:25.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.729+0000 7f89c2800700 1 -- 172.21.15.90:0/4154054443 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f89bc10a990 con 0x7f89bc06ba20 2024-04-24T09:20:25.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.730+0000 7f89ba000700 1 -- 172.21.15.90:0/4154054443 <== mon.2 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f89a809caf0 con 0x7f89bc06ba20 2024-04-24T09:20:25.730 INFO:teuthology.orchestra.run.smithi090.stdout:167503724609 2024-04-24T09:20:25.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.733+0000 7f89c2800700 1 -- 172.21.15.90:0/4154054443 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f89a40a3cd0 msgr2=0x7f89a40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:25.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.733+0000 7f89c2800700 1 --2- 172.21.15.90:0/4154054443 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f89a40a3cd0 0x7f89a40a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f89b000b6c0 tx=0x7f89b00094b0 comp rx=0 tx=0).stop 2024-04-24T09:20:25.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.733+0000 7f89c2800700 1 -- 172.21.15.90:0/4154054443 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89bc06ba20 msgr2=0x7f89bc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:25.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.733+0000 7f89c2800700 1 --2- 172.21.15.90:0/4154054443 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89bc06ba20 0x7f89bc1021c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f89a800e9c0 tx=0x7f89a800ecd0 comp rx=0 tx=0).stop 2024-04-24T09:20:25.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.734+0000 7f89c2800700 1 -- 172.21.15.90:0/4154054443 shutdown_connections 2024-04-24T09:20:25.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.734+0000 7f89c2800700 1 --2- 172.21.15.90:0/4154054443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89bc108ad0 0x7f89bc102700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:25.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.734+0000 7f89c2800700 1 --2- 172.21.15.90:0/4154054443 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f89a40a3cd0 0x7f89a40a6190 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:25.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.734+0000 7f89c2800700 1 --2- 172.21.15.90:0/4154054443 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89bc06b060 0x7f89bc101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:25.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.734+0000 7f89c2800700 1 --2- 172.21.15.90:0/4154054443 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89bc06ba20 0x7f89bc1021c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:25.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.734+0000 7f89c2800700 1 -- 172.21.15.90:0/4154054443 >> 172.21.15.90:0/4154054443 conn(0x7f89bc078830 msgr2=0x7f89bc0ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:20:25.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.734+0000 7f89c2800700 1 -- 172.21.15.90:0/4154054443 shutdown_connections 2024-04-24T09:20:25.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:25.734+0000 7f89c2800700 1 -- 172.21.15.90:0/4154054443 wait complete. 2024-04-24T09:20:27.092 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:26 smithi090 ceph-mon[27825]: pgmap v362: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-24T09:20:27.092 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:26 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/4154054443' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-24T09:20:27.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:26 smithi149 ceph-mon[32592]: pgmap v362: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-24T09:20:27.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:26 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/4154054443' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-24T09:20:27.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:26 smithi114 ceph-mon[32501]: pgmap v362: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-24T09:20:27.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:26 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/4154054443' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-24T09:20:28.048 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:27 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/3123615945' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-24T09:20:28.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:27 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/3123615945' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-24T09:20:28.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:27 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/3123615945' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-24T09:20:28.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.764+0000 7f6b86200700 1 -- 172.21.15.90:0/2421678155 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6b8006b1a0 msgr2=0x7f6b8010f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:28.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.764+0000 7f6b86200700 1 --2- 172.21.15.90:0/2421678155 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6b8006b1a0 0x7f6b8010f9b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6b7800b0a0 tx=0x7f6b7800b3b0 comp rx=0 tx=0).stop 2024-04-24T09:20:28.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.764+0000 7f6b86200700 1 -- 172.21.15.90:0/2421678155 shutdown_connections 2024-04-24T09:20:28.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.764+0000 7f6b86200700 1 --2- 172.21.15.90:0/2421678155 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6b8006b1a0 0x7f6b8010f9b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:28.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.764+0000 7f6b86200700 1 --2- 172.21.15.90:0/2421678155 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6b80108b70 0x7f6b80108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:28.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.764+0000 7f6b86200700 1 --2- 172.21.15.90:0/2421678155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6b8006a800 0x7f6b8006ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:28.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.764+0000 7f6b86200700 1 -- 172.21.15.90:0/2421678155 >> 172.21.15.90:0/2421678155 conn(0x7f6b800789a0 msgr2=0x7f6b80078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:20:28.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.764+0000 7f6b86200700 1 -- 172.21.15.90:0/2421678155 shutdown_connections 2024-04-24T09:20:28.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.764+0000 7f6b86200700 1 -- 172.21.15.90:0/2421678155 wait complete. 2024-04-24T09:20:28.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.765+0000 7f6b86200700 1 Processor -- start 2024-04-24T09:20:28.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.765+0000 7f6b86200700 1 -- start start 2024-04-24T09:20:28.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.766+0000 7f6b86200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6b8006a800 0x7f6b80101ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:28.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.766+0000 7f6b86200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6b8006b1a0 0x7f6b80102000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:28.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.766+0000 7f6b86200700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6b80108b70 0x7f6b80102540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:28.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.766+0000 7f6b86200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6b80112e10 con 0x7f6b8006b1a0 2024-04-24T09:20:28.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.766+0000 7f6b86200700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6b80112c90 con 0x7f6b80108b70 2024-04-24T09:20:28.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.766+0000 7f6b86200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6b80112f90 con 0x7f6b8006a800 2024-04-24T09:20:28.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.766+0000 7f6b7f400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6b8006a800 0x7f6b80101ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:28.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.766+0000 7f6b7ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6b8006b1a0 0x7f6b80102000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:28.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.766+0000 7f6b7f400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6b8006a800 0x7f6b80101ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:40034/0 (socket says 172.21.15.90:40034) 2024-04-24T09:20:28.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.766+0000 7f6b7f400700 1 -- 172.21.15.90:0/59735876 learned_addr learned my addr 172.21.15.90:0/59735876 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:20:28.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.766+0000 7f6b7fe00700 1 --2- 172.21.15.90:0/59735876 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6b80108b70 0x7f6b80102540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:28.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.767+0000 7f6b7f400700 1 -- 172.21.15.90:0/59735876 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6b80108b70 msgr2=0x7f6b80102540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:28.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.767+0000 7f6b7f400700 1 --2- 172.21.15.90:0/59735876 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6b80108b70 0x7f6b80102540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:28.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.767+0000 7f6b7f400700 1 -- 172.21.15.90:0/59735876 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6b8006b1a0 msgr2=0x7f6b80102000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:28.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.767+0000 7f6b7f400700 1 --2- 172.21.15.90:0/59735876 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6b8006b1a0 0x7f6b80102000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:28.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.767+0000 7f6b7f400700 1 -- 172.21.15.90:0/59735876 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b801a77a0 con 0x7f6b8006a800 2024-04-24T09:20:28.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.767+0000 7f6b7f400700 1 --2- 172.21.15.90:0/59735876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6b8006a800 0x7f6b80101ac0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6b7000eb30 tx=0x7f6b7000ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:20:28.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.767+0000 7f6b77e00700 1 -- 172.21.15.90:0/59735876 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7000cbd0 con 0x7f6b8006a800 2024-04-24T09:20:28.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.767+0000 7f6b77e00700 1 -- 172.21.15.90:0/59735876 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b70004500 con 0x7f6b8006a800 2024-04-24T09:20:28.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.767+0000 7f6b86200700 1 -- 172.21.15.90:0/59735876 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b801a7a90 con 0x7f6b8006a800 2024-04-24T09:20:28.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.767+0000 7f6b77e00700 1 -- 172.21.15.90:0/59735876 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b70010430 con 0x7f6b8006a800 2024-04-24T09:20:28.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.768+0000 7f6b86200700 1 -- 172.21.15.90:0/59735876 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b801a7fd0 con 0x7f6b8006a800 2024-04-24T09:20:28.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.769+0000 7f6b86200700 1 -- 172.21.15.90:0/59735876 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b800697e0 con 0x7f6b8006a800 2024-04-24T09:20:28.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.770+0000 7f6b77e00700 1 -- 172.21.15.90:0/59735876 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6b70003ed0 con 0x7f6b8006a800 2024-04-24T09:20:28.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.771+0000 7f6b77e00700 1 --2- 172.21.15.90:0/59735876 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f6b640a3c40 0x7f6b640a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:28.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.772+0000 7f6b77e00700 1 -- 172.21.15.90:0/59735876 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f6b700d06e0 con 0x7f6b8006a800 2024-04-24T09:20:28.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.772+0000 7f6b7ea00700 1 --2- 172.21.15.90:0/59735876 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f6b640a3c40 0x7f6b640a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:28.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.772+0000 7f6b7ea00700 1 --2- 172.21.15.90:0/59735876 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f6b640a3c40 0x7f6b640a6100 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6b80078f10 tx=0x7f6b6c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:20:28.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.776+0000 7f6b77e00700 1 -- 172.21.15.90:0/59735876 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6b7009d080 con 0x7f6b8006a800 2024-04-24T09:20:28.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:28 smithi090 ceph-mon[27825]: pgmap v363: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-24T09:20:28.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.872+0000 7fdd54000700 1 -- 172.21.15.90:0/162035615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd4c102b40 msgr2=0x7fdd4c102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:28.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.872+0000 7fdd54000700 1 --2- 172.21.15.90:0/162035615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd4c102b40 0x7fdd4c102fc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdd380098f0 tx=0x7fdd38009c00 comp rx=0 tx=0).stop 2024-04-24T09:20:28.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.872+0000 7fdd54000700 1 -- 172.21.15.90:0/162035615 shutdown_connections 2024-04-24T09:20:28.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.872+0000 7fdd54000700 1 --2- 172.21.15.90:0/162035615 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdd4c103500 0x7fdd4c10f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:28.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.872+0000 7fdd54000700 1 --2- 172.21.15.90:0/162035615 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd4c108b90 0x7fdd4c108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:28.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.872+0000 7fdd54000700 1 --2- 172.21.15.90:0/162035615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd4c102b40 0x7fdd4c102fc0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:28.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.872+0000 7fdd54000700 1 -- 172.21.15.90:0/162035615 >> 172.21.15.90:0/162035615 conn(0x7fdd4c079c50 msgr2=0x7fdd4c07a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:20:28.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.872+0000 7fdd54000700 1 -- 172.21.15.90:0/162035615 shutdown_connections 2024-04-24T09:20:28.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.872+0000 7fdd54000700 1 -- 172.21.15.90:0/162035615 wait complete. 2024-04-24T09:20:28.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.873+0000 7fdd54000700 1 Processor -- start 2024-04-24T09:20:28.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.873+0000 7fdd54000700 1 -- start start 2024-04-24T09:20:28.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.873+0000 7fdd54000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd4c102b40 0x7fdd4c074df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:28.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.873+0000 7fdd54000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd4c103500 0x7fdd4c075330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:28.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.873+0000 7fdd54000700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdd4c108b90 0x7fdd4c075870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:28.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.874+0000 7fdd54000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4c112e40 con 0x7fdd4c102b40 2024-04-24T09:20:28.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.874+0000 7fdd54000700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4c112cc0 con 0x7fdd4c108b90 2024-04-24T09:20:28.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.874+0000 7fdd54000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4c112fc0 con 0x7fdd4c103500 2024-04-24T09:20:28.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.874+0000 7fdd51600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd4c102b40 0x7fdd4c074df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:28.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.874+0000 7fdd50c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd4c103500 0x7fdd4c075330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:28.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.874+0000 7fdd50c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd4c103500 0x7fdd4c075330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:40052/0 (socket says 172.21.15.90:40052) 2024-04-24T09:20:28.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.874+0000 7fdd50c00700 1 -- 172.21.15.90:0/3270815459 learned_addr learned my addr 172.21.15.90:0/3270815459 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:20:28.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.874+0000 7fdd51600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd4c102b40 0x7fdd4c074df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55656/0 (socket says 172.21.15.90:55656) 2024-04-24T09:20:28.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.874+0000 7fdd52000700 1 --2- 172.21.15.90:0/3270815459 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdd4c108b90 0x7fdd4c075870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:28.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.874+0000 7fdd50c00700 1 -- 172.21.15.90:0/3270815459 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdd4c108b90 msgr2=0x7fdd4c075870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:28.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.874+0000 7fdd50c00700 1 --2- 172.21.15.90:0/3270815459 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdd4c108b90 0x7fdd4c075870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:28.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.874+0000 7fdd50c00700 1 -- 172.21.15.90:0/3270815459 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd4c102b40 msgr2=0x7fdd4c074df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:28.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.874+0000 7fdd50c00700 1 --2- 172.21.15.90:0/3270815459 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd4c102b40 0x7fdd4c074df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:28.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.874+0000 7fdd50c00700 1 -- 172.21.15.90:0/3270815459 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd4c1a78c0 con 0x7fdd4c103500 2024-04-24T09:20:28.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.874+0000 7fdd51600700 1 --2- 172.21.15.90:0/3270815459 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd4c102b40 0x7fdd4c074df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:20:28.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.875+0000 7fdd50c00700 1 --2- 172.21.15.90:0/3270815459 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd4c103500 0x7fdd4c075330 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdd38005950 tx=0x7fdd38005040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:20:28.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.875+0000 7fdd4a000700 1 -- 172.21.15.90:0/3270815459 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd380052e0 con 0x7fdd4c103500 2024-04-24T09:20:28.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.875+0000 7fdd4a000700 1 -- 172.21.15.90:0/3270815459 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd3800ba40 con 0x7fdd4c103500 2024-04-24T09:20:28.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.875+0000 7fdd4a000700 1 -- 172.21.15.90:0/3270815459 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd38003b80 con 0x7fdd4c103500 2024-04-24T09:20:28.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.875+0000 7fdd54000700 1 -- 172.21.15.90:0/3270815459 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd4c1a7b50 con 0x7fdd4c103500 2024-04-24T09:20:28.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.875+0000 7fdd54000700 1 -- 172.21.15.90:0/3270815459 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd4c1a7fe0 con 0x7fdd4c103500 2024-04-24T09:20:28.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.876+0000 7fdd54000700 1 -- 172.21.15.90:0/3270815459 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd4c0697e0 con 0x7fdd4c103500 2024-04-24T09:20:28.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.878+0000 7fdd4a000700 1 -- 172.21.15.90:0/3270815459 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fdd38005480 con 0x7fdd4c103500 2024-04-24T09:20:28.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.879+0000 7fdd4a000700 1 --2- 172.21.15.90:0/3270815459 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fdd340a3cc0 0x7fdd340a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:28.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.879+0000 7fdd4a000700 1 -- 172.21.15.90:0/3270815459 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fdd380d13a0 con 0x7fdd4c103500 2024-04-24T09:20:28.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.879+0000 7fdd51600700 1 --2- 172.21.15.90:0/3270815459 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fdd340a3cc0 0x7fdd340a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:28.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.880+0000 7fdd51600700 1 --2- 172.21.15.90:0/3270815459 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fdd340a3cc0 0x7fdd340a6180 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdd40005fd0 tx=0x7fdd40009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:20:28.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:28.883+0000 7fdd4a000700 1 -- 172.21.15.90:0/3270815459 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdd38013030 con 0x7fdd4c103500 2024-04-24T09:20:29.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.018+0000 7f6b86200700 1 -- 172.21.15.90:0/59735876 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f6b8002dfe0 con 0x7f6b8006a800 2024-04-24T09:20:29.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.018+0000 7f6b77e00700 1 -- 172.21.15.90:0/59735876 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6b7009c980 con 0x7f6b8006a800 2024-04-24T09:20:29.018 INFO:teuthology.orchestra.run.smithi090.stdout:292057776170 2024-04-24T09:20:29.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.021+0000 7f6b86200700 1 -- 172.21.15.90:0/59735876 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f6b640a3c40 msgr2=0x7f6b640a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:29.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.021+0000 7f6b86200700 1 --2- 172.21.15.90:0/59735876 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f6b640a3c40 0x7f6b640a6100 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6b80078f10 tx=0x7f6b6c009260 comp rx=0 tx=0).stop 2024-04-24T09:20:29.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.021+0000 7f6b86200700 1 -- 172.21.15.90:0/59735876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6b8006a800 msgr2=0x7f6b80101ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:29.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.021+0000 7f6b86200700 1 --2- 172.21.15.90:0/59735876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6b8006a800 0x7f6b80101ac0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6b7000eb30 tx=0x7f6b7000ee40 comp rx=0 tx=0).stop 2024-04-24T09:20:29.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.021+0000 7f6b86200700 1 -- 172.21.15.90:0/59735876 shutdown_connections 2024-04-24T09:20:29.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.021+0000 7f6b86200700 1 --2- 172.21.15.90:0/59735876 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6b80108b70 0x7f6b80102540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:29.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.021+0000 7f6b86200700 1 --2- 172.21.15.90:0/59735876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6b8006a800 0x7f6b80101ac0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:29.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.021+0000 7f6b86200700 1 --2- 172.21.15.90:0/59735876 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f6b640a3c40 0x7f6b640a6100 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:29.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.021+0000 7f6b86200700 1 --2- 172.21.15.90:0/59735876 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6b8006b1a0 0x7f6b80102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:29.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.021+0000 7f6b86200700 1 -- 172.21.15.90:0/59735876 >> 172.21.15.90:0/59735876 conn(0x7f6b800789a0 msgr2=0x7f6b800ff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:20:29.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.021+0000 7f6b86200700 1 -- 172.21.15.90:0/59735876 shutdown_connections 2024-04-24T09:20:29.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.022+0000 7f6b86200700 1 -- 172.21.15.90:0/59735876 wait complete. 2024-04-24T09:20:29.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:28 smithi149 ceph-mon[32592]: pgmap v363: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-24T09:20:29.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.119+0000 7fdd54000700 1 -- 172.21.15.90:0/3270815459 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fdd4c02e240 con 0x7fdd4c103500 2024-04-24T09:20:29.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.120+0000 7fdd4a000700 1 -- 172.21.15.90:0/3270815459 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fdd3809ddd0 con 0x7fdd4c103500 2024-04-24T09:20:29.120 INFO:teuthology.orchestra.run.smithi090.stdout:313532612642 2024-04-24T09:20:29.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.123+0000 7fdd54000700 1 -- 172.21.15.90:0/3270815459 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fdd340a3cc0 msgr2=0x7fdd340a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:29.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.123+0000 7fdd54000700 1 --2- 172.21.15.90:0/3270815459 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fdd340a3cc0 0x7fdd340a6180 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdd40005fd0 tx=0x7fdd40009310 comp rx=0 tx=0).stop 2024-04-24T09:20:29.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.123+0000 7fdd54000700 1 -- 172.21.15.90:0/3270815459 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd4c103500 msgr2=0x7fdd4c075330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:29.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.123+0000 7fdd54000700 1 --2- 172.21.15.90:0/3270815459 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd4c103500 0x7fdd4c075330 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdd38005950 tx=0x7fdd38005040 comp rx=0 tx=0).stop 2024-04-24T09:20:29.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.123+0000 7fdd54000700 1 -- 172.21.15.90:0/3270815459 shutdown_connections 2024-04-24T09:20:29.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.123+0000 7fdd54000700 1 --2- 172.21.15.90:0/3270815459 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdd4c108b90 0x7fdd4c075870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:29.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.123+0000 7fdd54000700 1 --2- 172.21.15.90:0/3270815459 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fdd340a3cc0 0x7fdd340a6180 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:29.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.123+0000 7fdd54000700 1 --2- 172.21.15.90:0/3270815459 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd4c102b40 0x7fdd4c074df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:29.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.123+0000 7fdd54000700 1 --2- 172.21.15.90:0/3270815459 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd4c103500 0x7fdd4c075330 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:29.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.123+0000 7fdd54000700 1 -- 172.21.15.90:0/3270815459 >> 172.21.15.90:0/3270815459 conn(0x7fdd4c079c50 msgr2=0x7fdd4c1019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:20:29.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.123+0000 7fdd54000700 1 -- 172.21.15.90:0/3270815459 shutdown_connections 2024-04-24T09:20:29.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.123+0000 7fdd54000700 1 -- 172.21.15.90:0/3270815459 wait complete. 2024-04-24T09:20:29.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:28 smithi114 ceph-mon[32501]: pgmap v363: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-24T09:20:29.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.638+0000 7fdb7c600700 1 -- 172.21.15.90:0/563834576 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb74074d20 msgr2=0x7fdb74075100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:29.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.638+0000 7fdb7c600700 1 --2- 172.21.15.90:0/563834576 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb74074d20 0x7fdb74075100 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdb68009990 tx=0x7fdb68009ca0 comp rx=0 tx=0).stop 2024-04-24T09:20:29.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.638+0000 7fdb7c600700 1 -- 172.21.15.90:0/563834576 shutdown_connections 2024-04-24T09:20:29.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.638+0000 7fdb7c600700 1 --2- 172.21.15.90:0/563834576 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb7410f4f0 0x7fdb741118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:29.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.638+0000 7fdb7c600700 1 --2- 172.21.15.90:0/563834576 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb74074d20 0x7fdb74075100 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:29.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.638+0000 7fdb7c600700 1 --2- 172.21.15.90:0/563834576 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb74075640 0x7fdb7410efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:29.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.638+0000 7fdb7c600700 1 -- 172.21.15.90:0/563834576 >> 172.21.15.90:0/563834576 conn(0x7fdb74079c50 msgr2=0x7fdb7407a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:20:29.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.638+0000 7fdb7c600700 1 -- 172.21.15.90:0/563834576 shutdown_connections 2024-04-24T09:20:29.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.638+0000 7fdb7c600700 1 -- 172.21.15.90:0/563834576 wait complete. 2024-04-24T09:20:29.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.639+0000 7fdb7c600700 1 Processor -- start 2024-04-24T09:20:29.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.640+0000 7fdb7c600700 1 -- start start 2024-04-24T09:20:29.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.640+0000 7fdb7c600700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb74074d20 0x7fdb7419b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:29.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.640+0000 7fdb7c600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb74075640 0x7fdb7419b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:29.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.640+0000 7fdb7c600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb7410f4f0 0x7fdb7419fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:29.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.640+0000 7fdb7c600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdb74115020 con 0x7fdb7410f4f0 2024-04-24T09:20:29.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.640+0000 7fdb7c600700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fdb74114ea0 con 0x7fdb74074d20 2024-04-24T09:20:29.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.640+0000 7fdb7c600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fdb741151a0 con 0x7fdb74075640 2024-04-24T09:20:29.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.640+0000 7fdb7a600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb7410f4f0 0x7fdb7419fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:29.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.640+0000 7fdb7a600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb7410f4f0 0x7fdb7419fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55668/0 (socket says 172.21.15.90:55668) 2024-04-24T09:20:29.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.640+0000 7fdb7a600700 1 -- 172.21.15.90:0/3193324694 learned_addr learned my addr 172.21.15.90:0/3193324694 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:20:29.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.640+0000 7fdb79c00700 1 --2- 172.21.15.90:0/3193324694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb74074d20 0x7fdb7419b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:29.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.640+0000 7fdb79200700 1 --2- 172.21.15.90:0/3193324694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb74075640 0x7fdb7419b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:29.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.641+0000 7fdb79c00700 1 -- 172.21.15.90:0/3193324694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb74075640 msgr2=0x7fdb7419b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:29.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.641+0000 7fdb79c00700 1 --2- 172.21.15.90:0/3193324694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb74075640 0x7fdb7419b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:29.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.641+0000 7fdb79c00700 1 -- 172.21.15.90:0/3193324694 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb7410f4f0 msgr2=0x7fdb7419fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:29.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.641+0000 7fdb79c00700 1 --2- 172.21.15.90:0/3193324694 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb7410f4f0 0x7fdb7419fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:29.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.641+0000 7fdb79c00700 1 -- 172.21.15.90:0/3193324694 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb741a04a0 con 0x7fdb74074d20 2024-04-24T09:20:29.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.641+0000 7fdb7a600700 1 --2- 172.21.15.90:0/3193324694 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb7410f4f0 0x7fdb7419fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:20:29.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.641+0000 7fdb79c00700 1 --2- 172.21.15.90:0/3193324694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb74074d20 0x7fdb7419b490 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdb680098e0 tx=0x7fdb6800b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:20:29.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.641+0000 7fdb72000700 1 -- 172.21.15.90:0/3193324694 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb68027070 con 0x7fdb74074d20 2024-04-24T09:20:29.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.641+0000 7fdb72000700 1 -- 172.21.15.90:0/3193324694 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb68003b60 con 0x7fdb74074d20 2024-04-24T09:20:29.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.641+0000 7fdb7c600700 1 -- 172.21.15.90:0/3193324694 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb741a0730 con 0x7fdb74074d20 2024-04-24T09:20:29.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.641+0000 7fdb72000700 1 -- 172.21.15.90:0/3193324694 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb68022d50 con 0x7fdb74074d20 2024-04-24T09:20:29.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.642+0000 7fdb7c600700 1 -- 172.21.15.90:0/3193324694 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb741a8010 con 0x7fdb74074d20 2024-04-24T09:20:29.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.643+0000 7fdb7c600700 1 -- 172.21.15.90:0/3193324694 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb74076410 con 0x7fdb74074d20 2024-04-24T09:20:29.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.645+0000 7fdb72000700 1 -- 172.21.15.90:0/3193324694 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fdb6802a070 con 0x7fdb74074d20 2024-04-24T09:20:29.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.646+0000 7fdb72000700 1 --2- 172.21.15.90:0/3193324694 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fdb5c0a3c40 0x7fdb5c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:29.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.646+0000 7fdb72000700 1 -- 172.21.15.90:0/3193324694 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fdb680d1ae0 con 0x7fdb74074d20 2024-04-24T09:20:29.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.646+0000 7fdb79200700 1 --2- 172.21.15.90:0/3193324694 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fdb5c0a3c40 0x7fdb5c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:29.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.647+0000 7fdb79200700 1 --2- 172.21.15.90:0/3193324694 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fdb5c0a3c40 0x7fdb5c0a6100 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdb74101cf0 tx=0x7fdb60009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:20:29.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.648+0000 7fdb72000700 1 -- 172.21.15.90:0/3193324694 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdb68031020 con 0x7fdb74074d20 2024-04-24T09:20:29.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.876+0000 7fdb7c600700 1 -- 172.21.15.90:0/3193324694 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fdb740697e0 con 0x7fdb74074d20 2024-04-24T09:20:29.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.876+0000 7fdb72000700 1 -- 172.21.15.90:0/3193324694 <== mon.2 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fdb740697e0 con 0x7fdb74074d20 2024-04-24T09:20:29.877 INFO:teuthology.orchestra.run.smithi090.stdout:111669149773 2024-04-24T09:20:29.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.879+0000 7fdb7c600700 1 -- 172.21.15.90:0/3193324694 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fdb5c0a3c40 msgr2=0x7fdb5c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:29.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.879+0000 7fdb7c600700 1 --2- 172.21.15.90:0/3193324694 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fdb5c0a3c40 0x7fdb5c0a6100 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdb74101cf0 tx=0x7fdb60009260 comp rx=0 tx=0).stop 2024-04-24T09:20:29.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.879+0000 7fdb7c600700 1 -- 172.21.15.90:0/3193324694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb74074d20 msgr2=0x7fdb7419b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:29.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.879+0000 7fdb7c600700 1 --2- 172.21.15.90:0/3193324694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb74074d20 0x7fdb7419b490 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdb680098e0 tx=0x7fdb6800b8a0 comp rx=0 tx=0).stop 2024-04-24T09:20:29.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.879+0000 7fdb7c600700 1 -- 172.21.15.90:0/3193324694 shutdown_connections 2024-04-24T09:20:29.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.879+0000 7fdb7c600700 1 --2- 172.21.15.90:0/3193324694 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fdb5c0a3c40 0x7fdb5c0a6100 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:29.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.879+0000 7fdb7c600700 1 --2- 172.21.15.90:0/3193324694 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb7410f4f0 0x7fdb7419fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:29.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.879+0000 7fdb7c600700 1 --2- 172.21.15.90:0/3193324694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdb74074d20 0x7fdb7419b490 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:29.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.879+0000 7fdb7c600700 1 --2- 172.21.15.90:0/3193324694 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb74075640 0x7fdb7419b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:29.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.879+0000 7fdb7c600700 1 -- 172.21.15.90:0/3193324694 >> 172.21.15.90:0/3193324694 conn(0x7fdb74079c50 msgr2=0x7fdb74100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:20:29.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.879+0000 7fdb7c600700 1 -- 172.21.15.90:0/3193324694 shutdown_connections 2024-04-24T09:20:29.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:29.879+0000 7fdb7c600700 1 -- 172.21.15.90:0/3193324694 wait complete. 2024-04-24T09:20:30.075 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:29 smithi090 ceph-mon[27825]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-24T09:20:30.075 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:29 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/59735876' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-24T09:20:30.075 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:29 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/3270815459' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-24T09:20:30.075 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:29 smithi090 ceph-mon[27825]: pgmap v364: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-24T09:20:30.077 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:29 smithi114 ceph-mon[32501]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-24T09:20:30.077 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:29 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/59735876' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-24T09:20:30.077 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:29 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/3270815459' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-24T09:20:30.077 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:29 smithi114 ceph-mon[32501]: pgmap v364: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-24T09:20:30.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:29 smithi149 ceph-mon[32592]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-24T09:20:30.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:29 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/59735876' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-24T09:20:30.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:29 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/3270815459' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-24T09:20:30.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:29 smithi149 ceph-mon[32592]: pgmap v364: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-24T09:20:30.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.102+0000 7fb92e200700 1 -- 172.21.15.90:0/1717402265 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb928108a20 msgr2=0x7fb928108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:30.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.102+0000 7fb92e200700 1 --2- 172.21.15.90:0/1717402265 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb928108a20 0x7fb928108e00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb91c009990 tx=0x7fb91c009ca0 comp rx=0 tx=0).stop 2024-04-24T09:20:30.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.102+0000 7fb92e200700 1 -- 172.21.15.90:0/1717402265 shutdown_connections 2024-04-24T09:20:30.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.102+0000 7fb92e200700 1 --2- 172.21.15.90:0/1717402265 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb9281033e0 0x7fb92810f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:30.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.102+0000 7fb92e200700 1 --2- 172.21.15.90:0/1717402265 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb928108a20 0x7fb928108e00 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:30.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.102+0000 7fb92e200700 1 --2- 172.21.15.90:0/1717402265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb928102a20 0x7fb928102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:30.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.102+0000 7fb92e200700 1 -- 172.21.15.90:0/1717402265 >> 172.21.15.90:0/1717402265 conn(0x7fb9280fe570 msgr2=0x7fb928100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:20:30.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.102+0000 7fb92e200700 1 -- 172.21.15.90:0/1717402265 shutdown_connections 2024-04-24T09:20:30.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.102+0000 7fb92e200700 1 -- 172.21.15.90:0/1717402265 wait complete. 2024-04-24T09:20:30.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.103+0000 7fb92e200700 1 Processor -- start 2024-04-24T09:20:30.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.103+0000 7fb92e200700 1 -- start start 2024-04-24T09:20:30.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.104+0000 7fb92e200700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb928102a20 0x7fb9281991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:30.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.104+0000 7fb92e200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9281033e0 0x7fb928199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:30.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.104+0000 7fb92e200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb928108a20 0x7fb92819daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:30.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.104+0000 7fb92e200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb928112da0 con 0x7fb928108a20 2024-04-24T09:20:30.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.104+0000 7fb92e200700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb928112c20 con 0x7fb928102a20 2024-04-24T09:20:30.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.104+0000 7fb92e200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb928112f20 con 0x7fb9281033e0 2024-04-24T09:20:30.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.104+0000 7fb92d800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb928108a20 0x7fb92819daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:30.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.104+0000 7fb92ce00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb928102a20 0x7fb9281991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:30.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.104+0000 7fb92d800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb928108a20 0x7fb92819daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55604/0 (socket says 172.21.15.90:55604) 2024-04-24T09:20:30.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.104+0000 7fb92d800700 1 -- 172.21.15.90:0/3123615945 learned_addr learned my addr 172.21.15.90:0/3123615945 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:20:30.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.104+0000 7fb927e00700 1 --2- 172.21.15.90:0/3123615945 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9281033e0 0x7fb928199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:30.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.105+0000 7fb92ce00700 1 -- 172.21.15.90:0/3123615945 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9281033e0 msgr2=0x7fb928199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:30.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.105+0000 7fb92ce00700 1 --2- 172.21.15.90:0/3123615945 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9281033e0 0x7fb928199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:30.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.105+0000 7fb92ce00700 1 -- 172.21.15.90:0/3123615945 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb928108a20 msgr2=0x7fb92819daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:30.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.105+0000 7fb92ce00700 1 --2- 172.21.15.90:0/3123615945 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb928108a20 0x7fb92819daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:30.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.105+0000 7fb92ce00700 1 -- 172.21.15.90:0/3123615945 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb92819e1e0 con 0x7fb928102a20 2024-04-24T09:20:30.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.105+0000 7fb927e00700 1 --2- 172.21.15.90:0/3123615945 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9281033e0 0x7fb928199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:20:30.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.105+0000 7fb92d800700 1 --2- 172.21.15.90:0/3123615945 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb928108a20 0x7fb92819daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:20:30.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.105+0000 7fb92ce00700 1 --2- 172.21.15.90:0/3123615945 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb928102a20 0x7fb9281991d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb91c0098e0 tx=0x7fb91c00b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:20:30.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.105+0000 7fb925600700 1 -- 172.21.15.90:0/3123615945 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91c027070 con 0x7fb928102a20 2024-04-24T09:20:30.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.105+0000 7fb92e200700 1 -- 172.21.15.90:0/3123615945 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb92819e470 con 0x7fb928102a20 2024-04-24T09:20:30.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.106+0000 7fb925600700 1 -- 172.21.15.90:0/3123615945 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb91c003b60 con 0x7fb928102a20 2024-04-24T09:20:30.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.106+0000 7fb925600700 1 -- 172.21.15.90:0/3123615945 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91c022d50 con 0x7fb928102a20 2024-04-24T09:20:30.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.106+0000 7fb92e200700 1 -- 172.21.15.90:0/3123615945 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9281a5d50 con 0x7fb928102a20 2024-04-24T09:20:30.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.107+0000 7fb92e200700 1 -- 172.21.15.90:0/3123615945 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9280697e0 con 0x7fb928102a20 2024-04-24T09:20:30.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.109+0000 7fb925600700 1 -- 172.21.15.90:0/3123615945 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb91c02a070 con 0x7fb928102a20 2024-04-24T09:20:30.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.110+0000 7fb925600700 1 --2- 172.21.15.90:0/3123615945 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fb9100a3d50 0x7fb9100a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:30.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.110+0000 7fb925600700 1 -- 172.21.15.90:0/3123615945 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fb91c0d1210 con 0x7fb928102a20 2024-04-24T09:20:30.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.110+0000 7fb927e00700 1 --2- 172.21.15.90:0/3123615945 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fb9100a3d50 0x7fb9100a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:30.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.111+0000 7fb927e00700 1 --2- 172.21.15.90:0/3123615945 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fb9100a3d50 0x7fb9100a6210 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb928100a90 tx=0x7fb914009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:20:30.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.114+0000 7fb925600700 1 -- 172.21.15.90:0/3123615945 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb91c013070 con 0x7fb928102a20 2024-04-24T09:20:30.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.350+0000 7fb92e200700 1 -- 172.21.15.90:0/3123615945 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fb92802dfe0 con 0x7fb928102a20 2024-04-24T09:20:30.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.351+0000 7fb925600700 1 -- 172.21.15.90:0/3123615945 <== mon.2 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb91c09dae0 con 0x7fb928102a20 2024-04-24T09:20:30.481 INFO:teuthology.orchestra.run.smithi090.stdout:257698037808 2024-04-24T09:20:30.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.354+0000 7fb92e200700 1 -- 172.21.15.90:0/3123615945 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fb9100a3d50 msgr2=0x7fb9100a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:30.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.354+0000 7fb92e200700 1 --2- 172.21.15.90:0/3123615945 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fb9100a3d50 0x7fb9100a6210 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb928100a90 tx=0x7fb914009260 comp rx=0 tx=0).stop 2024-04-24T09:20:30.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.354+0000 7fb92e200700 1 -- 172.21.15.90:0/3123615945 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb928102a20 msgr2=0x7fb9281991d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:30.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.354+0000 7fb92e200700 1 --2- 172.21.15.90:0/3123615945 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb928102a20 0x7fb9281991d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb91c0098e0 tx=0x7fb91c00b8a0 comp rx=0 tx=0).stop 2024-04-24T09:20:30.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.354+0000 7fb92e200700 1 -- 172.21.15.90:0/3123615945 shutdown_connections 2024-04-24T09:20:30.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.354+0000 7fb92e200700 1 --2- 172.21.15.90:0/3123615945 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fb9100a3d50 0x7fb9100a6210 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:30.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.354+0000 7fb92e200700 1 --2- 172.21.15.90:0/3123615945 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb928108a20 0x7fb92819daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:30.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.354+0000 7fb92e200700 1 --2- 172.21.15.90:0/3123615945 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb928102a20 0x7fb9281991d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:30.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.354+0000 7fb92e200700 1 --2- 172.21.15.90:0/3123615945 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9281033e0 0x7fb928199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:30.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.354+0000 7fb92e200700 1 -- 172.21.15.90:0/3123615945 >> 172.21.15.90:0/3123615945 conn(0x7fb9280fe570 msgr2=0x7fb9280feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:20:30.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.354+0000 7fb92e200700 1 -- 172.21.15.90:0/3123615945 shutdown_connections 2024-04-24T09:20:30.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:27.354+0000 7fb92e200700 1 -- 172.21.15.90:0/3123615945 wait complete. 2024-04-24T09:20:31.023 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:30 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/3193324694' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-24T09:20:31.093 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:30 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/3193324694' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-24T09:20:31.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:30 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/3193324694' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-24T09:20:31.453 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037803 got 257698037808 for osd.8 2024-04-24T09:20:31.453 DEBUG:teuthology.parallel:result is None 2024-04-24T09:20:31.951 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:31 smithi090 ceph-mon[27825]: pgmap v365: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-24T09:20:32.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:31 smithi149 ceph-mon[32592]: pgmap v365: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-24T09:20:32.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:31 smithi114 ceph-mon[32501]: pgmap v365: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-24T09:20:32.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.363+0000 7f37bfe00700 1 -- 172.21.15.90:0/3358773649 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37c01099f0 msgr2=0x7f37c0115e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:32.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.363+0000 7f37bfe00700 1 --2- 172.21.15.90:0/3358773649 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37c01099f0 0x7f37c0115e90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f37b800b0a0 tx=0x7f37b800b3b0 comp rx=0 tx=0).stop 2024-04-24T09:20:32.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.363+0000 7f37bfe00700 1 -- 172.21.15.90:0/3358773649 shutdown_connections 2024-04-24T09:20:32.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.363+0000 7f37bfe00700 1 --2- 172.21.15.90:0/3358773649 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37c01099f0 0x7f37c0115e90 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:32.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.363+0000 7f37bfe00700 1 --2- 172.21.15.90:0/3358773649 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37c010f0e0 0x7f37c010f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:32.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.363+0000 7f37bfe00700 1 --2- 172.21.15.90:0/3358773649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f37c0109030 0x7f37c01094b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:32.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.363+0000 7f37bfe00700 1 -- 172.21.15.90:0/3358773649 >> 172.21.15.90:0/3358773649 conn(0x7f37c0078880 msgr2=0x7f37c0107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:20:32.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.363+0000 7f37bfe00700 1 -- 172.21.15.90:0/3358773649 shutdown_connections 2024-04-24T09:20:32.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.363+0000 7f37bfe00700 1 -- 172.21.15.90:0/3358773649 wait complete. 2024-04-24T09:20:32.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.364+0000 7f37bfe00700 1 Processor -- start 2024-04-24T09:20:32.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.364+0000 7f37bfe00700 1 -- start start 2024-04-24T09:20:32.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.365+0000 7f37bfe00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f37c0109030 0x7f37c019f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:32.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.365+0000 7f37bfe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37c01099f0 0x7f37c019fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:32.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.365+0000 7f37bfe00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37c010f0e0 0x7f37c01a3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:32.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.365+0000 7f37bfe00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f37c0119370 con 0x7f37c01099f0 2024-04-24T09:20:32.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.365+0000 7f37bfe00700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f37c01191f0 con 0x7f37c010f0e0 2024-04-24T09:20:32.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.365+0000 7f37bfe00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f37c01194f0 con 0x7f37c0109030 2024-04-24T09:20:32.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.365+0000 7f37bea00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f37c0109030 0x7f37c019f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:32.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.365+0000 7f37bf400700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37c010f0e0 0x7f37c01a3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:32.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.365+0000 7f37b7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37c01099f0 0x7f37c019fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:32.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.365+0000 7f37bea00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f37c0109030 0x7f37c019f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:40092/0 (socket says 172.21.15.90:40092) 2024-04-24T09:20:32.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.365+0000 7f37bf400700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37c010f0e0 0x7f37c01a3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.90:58482/0 (socket says 172.21.15.90:58482) 2024-04-24T09:20:32.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.365+0000 7f37bea00700 1 -- 172.21.15.90:0/2026355239 learned_addr learned my addr 172.21.15.90:0/2026355239 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:20:32.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.365+0000 7f37bf400700 1 -- 172.21.15.90:0/2026355239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f37c0109030 msgr2=0x7f37c019f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:32.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.365+0000 7f37bf400700 1 --2- 172.21.15.90:0/2026355239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f37c0109030 0x7f37c019f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:32.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.365+0000 7f37bf400700 1 -- 172.21.15.90:0/2026355239 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37c01099f0 msgr2=0x7f37c019fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:32.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.365+0000 7f37bf400700 1 --2- 172.21.15.90:0/2026355239 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37c01099f0 0x7f37c019fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:32.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.365+0000 7f37bf400700 1 -- 172.21.15.90:0/2026355239 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37c01a4700 con 0x7f37c010f0e0 2024-04-24T09:20:32.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.366+0000 7f37bf400700 1 --2- 172.21.15.90:0/2026355239 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37c010f0e0 0x7f37c01a3fc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f37b800b070 tx=0x7f37b8004c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:20:32.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.366+0000 7f37b7400700 1 -- 172.21.15.90:0/2026355239 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b8021070 con 0x7f37c010f0e0 2024-04-24T09:20:32.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.366+0000 7f37b7400700 1 -- 172.21.15.90:0/2026355239 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37b8007b80 con 0x7f37c010f0e0 2024-04-24T09:20:32.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.366+0000 7f37b7400700 1 -- 172.21.15.90:0/2026355239 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b8005040 con 0x7f37c010f0e0 2024-04-24T09:20:32.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.366+0000 7f37bfe00700 1 -- 172.21.15.90:0/2026355239 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37c01a4990 con 0x7f37c010f0e0 2024-04-24T09:20:32.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.366+0000 7f37bfe00700 1 -- 172.21.15.90:0/2026355239 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37c01ac270 con 0x7f37c010f0e0 2024-04-24T09:20:32.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.367+0000 7f37bfe00700 1 -- 172.21.15.90:0/2026355239 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37c0111430 con 0x7f37c010f0e0 2024-04-24T09:20:32.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.369+0000 7f37b7400700 1 -- 172.21.15.90:0/2026355239 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f37b8024070 con 0x7f37c010f0e0 2024-04-24T09:20:32.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.370+0000 7f37b7400700 1 --2- 172.21.15.90:0/2026355239 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f37a40a3cc0 0x7f37a40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:32.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.371+0000 7f37b7400700 1 -- 172.21.15.90:0/2026355239 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f37b800f020 con 0x7f37c010f0e0 2024-04-24T09:20:32.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.371+0000 7f37bea00700 1 --2- 172.21.15.90:0/2026355239 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f37a40a3cc0 0x7f37a40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:32.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.371+0000 7f37bea00700 1 --2- 172.21.15.90:0/2026355239 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f37a40a3cc0 0x7f37a40a6180 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f37b0009680 tx=0x7f37b0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:20:32.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.374+0000 7f37b7400700 1 -- 172.21.15.90:0/2026355239 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f37b802b020 con 0x7f37c010f0e0 2024-04-24T09:20:32.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.611+0000 7f37bfe00700 1 -- 172.21.15.90:0/2026355239 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f37c01a0a50 con 0x7f37c010f0e0 2024-04-24T09:20:32.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.612+0000 7f37b7400700 1 -- 172.21.15.90:0/2026355239 <== mon.2 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f37b8007d30 con 0x7f37c010f0e0 2024-04-24T09:20:32.612 INFO:teuthology.orchestra.run.smithi090.stdout:34359738463 2024-04-24T09:20:32.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.615+0000 7f37bfe00700 1 -- 172.21.15.90:0/2026355239 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f37a40a3cc0 msgr2=0x7f37a40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:32.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.615+0000 7f37bfe00700 1 --2- 172.21.15.90:0/2026355239 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f37a40a3cc0 0x7f37a40a6180 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f37b0009680 tx=0x7f37b0009260 comp rx=0 tx=0).stop 2024-04-24T09:20:32.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.615+0000 7f37bfe00700 1 -- 172.21.15.90:0/2026355239 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37c010f0e0 msgr2=0x7f37c01a3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:32.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.615+0000 7f37bfe00700 1 --2- 172.21.15.90:0/2026355239 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37c010f0e0 0x7f37c01a3fc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f37b800b070 tx=0x7f37b8004c80 comp rx=0 tx=0).stop 2024-04-24T09:20:32.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.615+0000 7f37bfe00700 1 -- 172.21.15.90:0/2026355239 shutdown_connections 2024-04-24T09:20:32.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.615+0000 7f37bfe00700 1 --2- 172.21.15.90:0/2026355239 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37c010f0e0 0x7f37c01a3fc0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:32.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.615+0000 7f37bfe00700 1 --2- 172.21.15.90:0/2026355239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f37c0109030 0x7f37c019f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:32.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.615+0000 7f37bfe00700 1 --2- 172.21.15.90:0/2026355239 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f37a40a3cc0 0x7f37a40a6180 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:32.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.615+0000 7f37bfe00700 1 --2- 172.21.15.90:0/2026355239 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37c01099f0 0x7f37c019fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:32.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.615+0000 7f37bfe00700 1 -- 172.21.15.90:0/2026355239 >> 172.21.15.90:0/2026355239 conn(0x7f37c0078880 msgr2=0x7f37c0079990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:20:32.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.615+0000 7f37bfe00700 1 -- 172.21.15.90:0/2026355239 shutdown_connections 2024-04-24T09:20:32.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.615+0000 7f37bfe00700 1 -- 172.21.15.90:0/2026355239 wait complete. 2024-04-24T09:20:33.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:32 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/2026355239' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-24T09:20:33.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:32 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/2026355239' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-24T09:20:33.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:32 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/2026355239' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-24T09:20:33.925 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:33 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/2379561428' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-24T09:20:33.925 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:33 smithi090 ceph-mon[27825]: pgmap v366: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-24T09:20:34.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:33 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/2379561428' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-24T09:20:34.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:33 smithi149 ceph-mon[32592]: pgmap v366: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-24T09:20:34.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:33 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/2379561428' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-24T09:20:34.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:33 smithi114 ceph-mon[32501]: pgmap v366: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-24T09:20:34.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:34 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/1126676966' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-24T09:20:35.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:34 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/1126676966' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-24T09:20:35.132 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149765 got 111669149773 for osd.3 2024-04-24T09:20:35.132 DEBUG:teuthology.parallel:result is None 2024-04-24T09:20:35.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:34 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/1126676966' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-24T09:20:35.249 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724601 got 167503724609 for osd.5 2024-04-24T09:20:35.249 DEBUG:teuthology.parallel:result is None 2024-04-24T09:20:35.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.617+0000 7f507f200700 1 -- 172.21.15.90:0/898094657 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5078104500 msgr2=0x7f507810c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:35.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.617+0000 7f507f200700 1 --2- 172.21.15.90:0/898094657 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5078104500 0x7f507810c9f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f50680098f0 tx=0x7f5068009c00 comp rx=0 tx=0).stop 2024-04-24T09:20:35.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.617+0000 7f507f200700 1 -- 172.21.15.90:0/898094657 shutdown_connections 2024-04-24T09:20:35.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.617+0000 7f507f200700 1 --2- 172.21.15.90:0/898094657 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f507810d010 0x7f507810f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:35.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.617+0000 7f507f200700 1 --2- 172.21.15.90:0/898094657 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5078103be0 0x7f5078103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:35.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.617+0000 7f507f200700 1 --2- 172.21.15.90:0/898094657 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5078104500 0x7f507810c9f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:35.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.617+0000 7f507f200700 1 -- 172.21.15.90:0/898094657 >> 172.21.15.90:0/898094657 conn(0x7f50780fd410 msgr2=0x7f50780ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:20:35.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.617+0000 7f507f200700 1 -- 172.21.15.90:0/898094657 shutdown_connections 2024-04-24T09:20:35.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.617+0000 7f507f200700 1 -- 172.21.15.90:0/898094657 wait complete. 2024-04-24T09:20:35.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.618+0000 7f507f200700 1 Processor -- start 2024-04-24T09:20:35.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.618+0000 7f507f200700 1 -- start start 2024-04-24T09:20:35.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.619+0000 7f507f200700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5078103be0 0x7f507819b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:35.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.619+0000 7f507f200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5078104500 0x7f507819b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:35.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.619+0000 7f507f200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f507810d010 0x7f507819fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:35.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.619+0000 7f507f200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5078112d90 con 0x7f5078104500 2024-04-24T09:20:35.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.619+0000 7f507f200700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5078112c10 con 0x7f5078103be0 2024-04-24T09:20:35.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.619+0000 7f507f200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5078112f10 con 0x7f507810d010 2024-04-24T09:20:35.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.619+0000 7f507de00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5078103be0 0x7f507819b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:35.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.619+0000 7f507d400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5078104500 0x7f507819b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:35.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.619+0000 7f507de00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5078103be0 0x7f507819b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.90:58500/0 (socket says 172.21.15.90:58500) 2024-04-24T09:20:35.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.619+0000 7f507de00700 1 -- 172.21.15.90:0/2379561428 learned_addr learned my addr 172.21.15.90:0/2379561428 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:20:35.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.619+0000 7f507e800700 1 --2- 172.21.15.90:0/2379561428 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f507810d010 0x7f507819fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:35.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.619+0000 7f507de00700 1 -- 172.21.15.90:0/2379561428 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f507810d010 msgr2=0x7f507819fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:35.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.619+0000 7f507de00700 1 --2- 172.21.15.90:0/2379561428 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f507810d010 0x7f507819fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:35.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.619+0000 7f507de00700 1 -- 172.21.15.90:0/2379561428 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5078104500 msgr2=0x7f507819b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:35.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.619+0000 7f507de00700 1 --2- 172.21.15.90:0/2379561428 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5078104500 0x7f507819b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:35.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.619+0000 7f507de00700 1 -- 172.21.15.90:0/2379561428 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50781a0490 con 0x7f5078103be0 2024-04-24T09:20:35.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.619+0000 7f507e800700 1 --2- 172.21.15.90:0/2379561428 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f507810d010 0x7f507819fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:20:35.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.620+0000 7f507de00700 1 --2- 172.21.15.90:0/2379561428 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5078103be0 0x7f507819b480 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f506c00eb30 tx=0x7f506c00ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:20:35.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.620+0000 7f5072a00700 1 -- 172.21.15.90:0/2379561428 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506c00cc90 con 0x7f5078103be0 2024-04-24T09:20:35.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.620+0000 7f5072a00700 1 -- 172.21.15.90:0/2379561428 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f506c004500 con 0x7f5078103be0 2024-04-24T09:20:35.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.620+0000 7f5072a00700 1 -- 172.21.15.90:0/2379561428 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506c010430 con 0x7f5078103be0 2024-04-24T09:20:35.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.620+0000 7f507f200700 1 -- 172.21.15.90:0/2379561428 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50781a0780 con 0x7f5078103be0 2024-04-24T09:20:35.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.620+0000 7f507f200700 1 -- 172.21.15.90:0/2379561428 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50781a8000 con 0x7f5078103be0 2024-04-24T09:20:35.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.621+0000 7f507f200700 1 -- 172.21.15.90:0/2379561428 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50780697e0 con 0x7f5078103be0 2024-04-24T09:20:35.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.623+0000 7f5072a00700 1 -- 172.21.15.90:0/2379561428 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f506c003f70 con 0x7f5078103be0 2024-04-24T09:20:35.301 INFO:teuthology.orchestra.run.smithi090.stdout:223338299447 2024-04-24T09:20:35.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.624+0000 7f5072a00700 1 --2- 172.21.15.90:0/2379561428 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f50600a3cc0 0x7f50600a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:35.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.625+0000 7f5072a00700 1 -- 172.21.15.90:0/2379561428 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f506c0d06f0 con 0x7f5078103be0 2024-04-24T09:20:35.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.625+0000 7f507d400700 1 --2- 172.21.15.90:0/2379561428 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f50600a3cc0 0x7f50600a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:35.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.625+0000 7f507d400700 1 --2- 172.21.15.90:0/2379561428 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f50600a3cc0 0x7f50600a6180 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f50680098c0 tx=0x7f5068019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:20:35.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.628+0000 7f5072a00700 1 -- 172.21.15.90:0/2379561428 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f506c09d090 con 0x7f5078103be0 2024-04-24T09:20:35.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.901+0000 7f507f200700 1 -- 172.21.15.90:0/2379561428 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f50781a81d0 con 0x7f5078103be0 2024-04-24T09:20:35.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.902+0000 7f5072a00700 1 -- 172.21.15.90:0/2379561428 <== mon.2 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f506c09c990 con 0x7f5078103be0 2024-04-24T09:20:35.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.904+0000 7f507f200700 1 -- 172.21.15.90:0/2379561428 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f50600a3cc0 msgr2=0x7f50600a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:35.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.904+0000 7f507f200700 1 --2- 172.21.15.90:0/2379561428 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f50600a3cc0 0x7f50600a6180 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f50680098c0 tx=0x7f5068019040 comp rx=0 tx=0).stop 2024-04-24T09:20:35.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.904+0000 7f507f200700 1 -- 172.21.15.90:0/2379561428 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5078103be0 msgr2=0x7f507819b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:35.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.904+0000 7f507f200700 1 --2- 172.21.15.90:0/2379561428 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5078103be0 0x7f507819b480 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f506c00eb30 tx=0x7f506c00ee40 comp rx=0 tx=0).stop 2024-04-24T09:20:35.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.904+0000 7f507f200700 1 -- 172.21.15.90:0/2379561428 shutdown_connections 2024-04-24T09:20:35.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.904+0000 7f507f200700 1 --2- 172.21.15.90:0/2379561428 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f507810d010 0x7f507819fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:35.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.904+0000 7f507f200700 1 --2- 172.21.15.90:0/2379561428 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5078103be0 0x7f507819b480 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:35.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.904+0000 7f507f200700 1 --2- 172.21.15.90:0/2379561428 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f50600a3cc0 0x7f50600a6180 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:35.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.904+0000 7f507f200700 1 --2- 172.21.15.90:0/2379561428 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5078104500 0x7f507819b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:35.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.904+0000 7f507f200700 1 -- 172.21.15.90:0/2379561428 >> 172.21.15.90:0/2379561428 conn(0x7f50780fd410 msgr2=0x7f50780ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:20:35.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.905+0000 7f507f200700 1 -- 172.21.15.90:0/2379561428 shutdown_connections 2024-04-24T09:20:35.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:32.905+0000 7f507f200700 1 -- 172.21.15.90:0/2379561428 wait complete. 2024-04-24T09:20:36.019 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612635 got 313532612642 for osd.10 2024-04-24T09:20:36.019 DEBUG:teuthology.parallel:result is None 2024-04-24T09:20:36.098 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:35 smithi090 ceph-mon[27825]: pgmap v367: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-24T09:20:36.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:35 smithi149 ceph-mon[32592]: pgmap v367: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-24T09:20:36.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:35 smithi114 ceph-mon[32501]: pgmap v367: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-24T09:20:38.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.647+0000 7f4ba7600700 1 -- 172.21.15.90:0/4277097512 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4ba0105920 msgr2=0x7f4ba0109970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:38.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.647+0000 7f4ba7600700 1 --2- 172.21.15.90:0/4277097512 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4ba0105920 0x7f4ba0109970 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c0099c0 tx=0x7f4b8c009cd0 comp rx=0 tx=0).stop 2024-04-24T09:20:38.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.648+0000 7f4ba7600700 1 -- 172.21.15.90:0/4277097512 shutdown_connections 2024-04-24T09:20:38.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.648+0000 7f4ba7600700 1 --2- 172.21.15.90:0/4277097512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ba010a050 0x7f4ba0111c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:38.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.648+0000 7f4ba7600700 1 --2- 172.21.15.90:0/4277097512 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ba0104f70 0x7f4ba0105350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:38.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.648+0000 7f4ba7600700 1 --2- 172.21.15.90:0/4277097512 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4ba0105920 0x7f4ba0109970 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:38.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.648+0000 7f4ba7600700 1 -- 172.21.15.90:0/4277097512 >> 172.21.15.90:0/4277097512 conn(0x7f4ba0100800 msgr2=0x7f4ba0102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:20:38.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.648+0000 7f4ba7600700 1 -- 172.21.15.90:0/4277097512 shutdown_connections 2024-04-24T09:20:38.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.648+0000 7f4ba7600700 1 -- 172.21.15.90:0/4277097512 wait complete. 2024-04-24T09:20:38.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.649+0000 7f4ba7600700 1 Processor -- start 2024-04-24T09:20:38.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.649+0000 7f4ba7600700 1 -- start start 2024-04-24T09:20:38.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.649+0000 7f4ba7600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ba0104f70 0x7f4ba019b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:38.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.649+0000 7f4ba7600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4ba0105920 0x7f4ba019b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:38.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.649+0000 7f4ba7600700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ba010a050 0x7f4ba019fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:38.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.649+0000 7f4ba7600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba0115010 con 0x7f4ba0105920 2024-04-24T09:20:38.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.649+0000 7f4ba7600700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba0114e90 con 0x7f4ba010a050 2024-04-24T09:20:38.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.649+0000 7f4ba7600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba0115190 con 0x7f4ba0104f70 2024-04-24T09:20:38.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.650+0000 7f4b9fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4ba0105920 0x7f4ba019b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:38.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.650+0000 7f4b9fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4ba0105920 0x7f4ba019b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55706/0 (socket says 172.21.15.90:55706) 2024-04-24T09:20:38.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.650+0000 7f4b9fe00700 1 -- 172.21.15.90:0/1126676966 learned_addr learned my addr 172.21.15.90:0/1126676966 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:20:38.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.650+0000 7f4ba4c00700 1 --2- 172.21.15.90:0/1126676966 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ba0104f70 0x7f4ba019b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:38.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.650+0000 7f4ba5600700 1 --2- 172.21.15.90:0/1126676966 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ba010a050 0x7f4ba019fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:38.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.650+0000 7f4b9fe00700 1 -- 172.21.15.90:0/1126676966 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ba0104f70 msgr2=0x7f4ba019b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:38.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.650+0000 7f4b9fe00700 1 --2- 172.21.15.90:0/1126676966 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ba0104f70 0x7f4ba019b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:38.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.650+0000 7f4b9fe00700 1 -- 172.21.15.90:0/1126676966 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ba010a050 msgr2=0x7f4ba019fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:38.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.650+0000 7f4b9fe00700 1 --2- 172.21.15.90:0/1126676966 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ba010a050 0x7f4ba019fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:38.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.650+0000 7f4b9fe00700 1 -- 172.21.15.90:0/1126676966 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ba01a0450 con 0x7f4ba0105920 2024-04-24T09:20:38.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.650+0000 7f4b9fe00700 1 --2- 172.21.15.90:0/1126676966 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4ba0105920 0x7f4ba019b980 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c006010 tx=0x7f4b8c003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:20:38.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.651+0000 7f4b9d600700 1 -- 172.21.15.90:0/1126676966 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c009e60 con 0x7f4ba0105920 2024-04-24T09:20:38.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.651+0000 7f4b9d600700 1 -- 172.21.15.90:0/1126676966 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b8c003680 con 0x7f4ba0105920 2024-04-24T09:20:38.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.651+0000 7f4ba7600700 1 -- 172.21.15.90:0/1126676966 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ba01a06e0 con 0x7f4ba0105920 2024-04-24T09:20:38.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.651+0000 7f4b9d600700 1 -- 172.21.15.90:0/1126676966 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c0173f0 con 0x7f4ba0105920 2024-04-24T09:20:38.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.651+0000 7f4ba7600700 1 -- 172.21.15.90:0/1126676966 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ba01a7fc0 con 0x7f4ba0105920 2024-04-24T09:20:38.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.652+0000 7f4ba7600700 1 -- 172.21.15.90:0/1126676966 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ba0106450 con 0x7f4ba0105920 2024-04-24T09:20:38.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.654+0000 7f4b9d600700 1 -- 172.21.15.90:0/1126676966 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4b8c00f5d0 con 0x7f4ba0105920 2024-04-24T09:20:38.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.656+0000 7f4b9d600700 1 --2- 172.21.15.90:0/1126676966 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f4b880a3c40 0x7f4b880a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:38.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.656+0000 7f4b9d600700 1 -- 172.21.15.90:0/1126676966 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f4b8c0d1160 con 0x7f4ba0105920 2024-04-24T09:20:38.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.656+0000 7f4ba4c00700 1 --2- 172.21.15.90:0/1126676966 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f4b880a3c40 0x7f4b880a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:38.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.657+0000 7f4ba4c00700 1 --2- 172.21.15.90:0/1126676966 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f4b880a3c40 0x7f4b880a6100 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4b94009680 tx=0x7f4b94009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:20:38.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.661+0000 7f4b9d600700 1 -- 172.21.15.90:0/1126676966 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4b8c013070 con 0x7f4ba0105920 2024-04-24T09:20:38.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.902+0000 7f4ba7600700 1 -- 172.21.15.90:0/1126676966 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f4ba002dfe0 con 0x7f4ba0105920 2024-04-24T09:20:38.311 INFO:teuthology.orchestra.run.smithi090.stdout:77309411412 2024-04-24T09:20:38.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.903+0000 7f4b9d600700 1 -- 172.21.15.90:0/1126676966 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4b8c09db90 con 0x7f4ba0105920 2024-04-24T09:20:38.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.906+0000 7f4ba7600700 1 -- 172.21.15.90:0/1126676966 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f4b880a3c40 msgr2=0x7f4b880a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:38.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.906+0000 7f4ba7600700 1 --2- 172.21.15.90:0/1126676966 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f4b880a3c40 0x7f4b880a6100 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4b94009680 tx=0x7f4b94009260 comp rx=0 tx=0).stop 2024-04-24T09:20:38.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.906+0000 7f4ba7600700 1 -- 172.21.15.90:0/1126676966 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4ba0105920 msgr2=0x7f4ba019b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:38.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.906+0000 7f4ba7600700 1 --2- 172.21.15.90:0/1126676966 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4ba0105920 0x7f4ba019b980 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c006010 tx=0x7f4b8c003e00 comp rx=0 tx=0).stop 2024-04-24T09:20:38.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.906+0000 7f4ba7600700 1 -- 172.21.15.90:0/1126676966 shutdown_connections 2024-04-24T09:20:38.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.906+0000 7f4ba7600700 1 --2- 172.21.15.90:0/1126676966 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4ba010a050 0x7f4ba019fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:38.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.906+0000 7f4ba7600700 1 --2- 172.21.15.90:0/1126676966 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ba0104f70 0x7f4ba019b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:38.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.906+0000 7f4ba7600700 1 --2- 172.21.15.90:0/1126676966 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f4b880a3c40 0x7f4b880a6100 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:38.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.906+0000 7f4ba7600700 1 --2- 172.21.15.90:0/1126676966 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4ba0105920 0x7f4ba019b980 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:38.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.906+0000 7f4ba7600700 1 -- 172.21.15.90:0/1126676966 >> 172.21.15.90:0/1126676966 conn(0x7f4ba0100800 msgr2=0x7f4ba0101380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:20:38.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.906+0000 7f4ba7600700 1 -- 172.21.15.90:0/1126676966 shutdown_connections 2024-04-24T09:20:38.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:33.906+0000 7f4ba7600700 1 -- 172.21.15.90:0/1126676966 wait complete. 2024-04-24T09:20:38.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:38 smithi114 ceph-mon[32501]: pgmap v368: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-24T09:20:39.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:38 smithi090 ceph-mon[27825]: pgmap v368: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-24T09:20:39.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:38 smithi149 ceph-mon[32592]: pgmap v368: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-24T09:20:39.464 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299441 got 223338299447 for osd.7 2024-04-24T09:20:39.464 DEBUG:teuthology.parallel:result is None 2024-04-24T09:20:40.060 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738455 got 34359738463 for osd.0 2024-04-24T09:20:40.060 DEBUG:teuthology.parallel:result is None 2024-04-24T09:20:40.179 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776165 got 292057776170 for osd.9 2024-04-24T09:20:40.179 DEBUG:teuthology.parallel:result is None 2024-04-24T09:20:40.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:40 smithi090 ceph-mon[27825]: pgmap v369: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-24T09:20:40.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:40 smithi114 ceph-mon[32501]: pgmap v369: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-24T09:20:41.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:40 smithi149 ceph-mon[32592]: pgmap v369: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-24T09:20:41.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:41.806+0000 7f85be800700 1 -- 172.21.15.90:0/2473686178 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85b8105830 msgr2=0x7f85b8109880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:41.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:41.806+0000 7f85be800700 1 --2- 172.21.15.90:0/2473686178 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85b8105830 0x7f85b8109880 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f85a4009990 tx=0x7f85a4009ca0 comp rx=0 tx=0).stop 2024-04-24T09:20:41.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:41.806+0000 7f85be800700 1 -- 172.21.15.90:0/2473686178 shutdown_connections 2024-04-24T09:20:41.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:41.806+0000 7f85be800700 1 --2- 172.21.15.90:0/2473686178 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85b8109f60 0x7f85b8111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:41.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:41.806+0000 7f85be800700 1 --2- 172.21.15.90:0/2473686178 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85b8104e80 0x7f85b8105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:41.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:41.806+0000 7f85be800700 1 --2- 172.21.15.90:0/2473686178 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85b8105830 0x7f85b8109880 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:41.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:41.806+0000 7f85be800700 1 -- 172.21.15.90:0/2473686178 >> 172.21.15.90:0/2473686178 conn(0x7f85b8100710 msgr2=0x7f85b8102b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:20:41.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:41.806+0000 7f85be800700 1 -- 172.21.15.90:0/2473686178 shutdown_connections 2024-04-24T09:20:41.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:41.806+0000 7f85be800700 1 -- 172.21.15.90:0/2473686178 wait complete. 2024-04-24T09:20:41.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:41.807+0000 7f85be800700 1 Processor -- start 2024-04-24T09:20:41.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:41.807+0000 7f85be800700 1 -- start start 2024-04-24T09:20:41.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:41.808+0000 7f85be800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85b8104e80 0x7f85b819b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:41.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:41.808+0000 7f85be800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85b8105830 0x7f85b819b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:41.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:41.808+0000 7f85be800700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85b8109f60 0x7f85b819fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:41.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:41.808+0000 7f85be800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f85b8114f20 con 0x7f85b8105830 2024-04-24T09:20:41.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:41.808+0000 7f85be800700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f85b8114da0 con 0x7f85b8109f60 2024-04-24T09:20:41.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:41.808+0000 7f85be800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f85b81150a0 con 0x7f85b8104e80 2024-04-24T09:20:41.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:41.808+0000 7f85bca00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85b8105830 0x7f85b819b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:41.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:41.808+0000 7f85bca00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85b8105830 0x7f85b819b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44308/0 (socket says 172.21.15.90:44308) 2024-04-24T09:20:41.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:41.808+0000 7f85bde00700 1 --2- 172.21.15.90:0/4206095423 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85b8109f60 0x7f85b819fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:41.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:41.808+0000 7f85bca00700 1 -- 172.21.15.90:0/4206095423 learned_addr learned my addr 172.21.15.90:0/4206095423 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:20:41.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:41.808+0000 7f85bd400700 1 --2- 172.21.15.90:0/4206095423 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85b8104e80 0x7f85b819b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:41.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:41.808+0000 7f85bde00700 1 -- 172.21.15.90:0/4206095423 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85b8104e80 msgr2=0x7f85b819b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:41.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:41.808+0000 7f85bde00700 1 --2- 172.21.15.90:0/4206095423 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85b8104e80 0x7f85b819b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:41.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:41.808+0000 7f85bde00700 1 -- 172.21.15.90:0/4206095423 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85b8105830 msgr2=0x7f85b819b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:41.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:41.808+0000 7f85bde00700 1 --2- 172.21.15.90:0/4206095423 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85b8105830 0x7f85b819b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:41.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:41.808+0000 7f85bde00700 1 -- 172.21.15.90:0/4206095423 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b81a03c0 con 0x7f85b8109f60 2024-04-24T09:20:41.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:41.809+0000 7f85bde00700 1 --2- 172.21.15.90:0/4206095423 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85b8109f60 0x7f85b819fc80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f85b000efc0 tx=0x7f85b000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:20:41.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:41.809+0000 7f85b6000700 1 -- 172.21.15.90:0/4206095423 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b00092e0 con 0x7f85b8109f60 2024-04-24T09:20:41.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:41.809+0000 7f85b6000700 1 -- 172.21.15.90:0/4206095423 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85b0004bb0 con 0x7f85b8109f60 2024-04-24T09:20:41.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:41.809+0000 7f85b6000700 1 -- 172.21.15.90:0/4206095423 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b0004280 con 0x7f85b8109f60 2024-04-24T09:20:41.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:41.809+0000 7f85be800700 1 -- 172.21.15.90:0/4206095423 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85b81a0650 con 0x7f85b8109f60 2024-04-24T09:20:41.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:41.809+0000 7f85be800700 1 -- 172.21.15.90:0/4206095423 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85b81a7f30 con 0x7f85b8109f60 2024-04-24T09:20:41.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:41.810+0000 7f85be800700 1 -- 172.21.15.90:0/4206095423 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85b80697e0 con 0x7f85b8109f60 2024-04-24T09:20:41.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:41.812+0000 7f85b6000700 1 -- 172.21.15.90:0/4206095423 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f85b001b030 con 0x7f85b8109f60 2024-04-24T09:20:41.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:41.814+0000 7f85b6000700 1 --2- 172.21.15.90:0/4206095423 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f85a00a3cc0 0x7f85a00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:41.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:41.814+0000 7f85b6000700 1 -- 172.21.15.90:0/4206095423 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f85b00cfe20 con 0x7f85b8109f60 2024-04-24T09:20:41.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:41.814+0000 7f85bd400700 1 --2- 172.21.15.90:0/4206095423 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f85a00a3cc0 0x7f85a00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:41.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:41.814+0000 7f85bd400700 1 --2- 172.21.15.90:0/4206095423 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f85a00a3cc0 0x7f85a00a6180 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f85ac009c60 tx=0x7f85ac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:20:41.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:41.818+0000 7f85b6000700 1 -- 172.21.15.90:0/4206095423 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f85b0022020 con 0x7f85b8109f60 2024-04-24T09:20:42.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:42.045+0000 7f85be800700 1 -- 172.21.15.90:0/4206095423 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f85b802dfe0 con 0x7f85b8109f60 2024-04-24T09:20:42.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:42.046+0000 7f85b6000700 1 -- 172.21.15.90:0/4206095423 <== mon.2 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f85b0010040 con 0x7f85b8109f60 2024-04-24T09:20:42.049 INFO:teuthology.orchestra.run.smithi090.stdout:347892351010 2024-04-24T09:20:42.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:42.048+0000 7f85be800700 1 -- 172.21.15.90:0/4206095423 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f85a00a3cc0 msgr2=0x7f85a00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:42.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:42.048+0000 7f85be800700 1 --2- 172.21.15.90:0/4206095423 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f85a00a3cc0 0x7f85a00a6180 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f85ac009c60 tx=0x7f85ac009260 comp rx=0 tx=0).stop 2024-04-24T09:20:42.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:42.049+0000 7f85be800700 1 -- 172.21.15.90:0/4206095423 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85b8109f60 msgr2=0x7f85b819fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:42.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:42.049+0000 7f85be800700 1 --2- 172.21.15.90:0/4206095423 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85b8109f60 0x7f85b819fc80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f85b000efc0 tx=0x7f85b000c3a0 comp rx=0 tx=0).stop 2024-04-24T09:20:42.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:42.049+0000 7f85be800700 1 -- 172.21.15.90:0/4206095423 shutdown_connections 2024-04-24T09:20:42.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:42.049+0000 7f85be800700 1 --2- 172.21.15.90:0/4206095423 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f85b8109f60 0x7f85b819fc80 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:42.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:42.049+0000 7f85be800700 1 --2- 172.21.15.90:0/4206095423 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85b8104e80 0x7f85b819b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:42.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:42.049+0000 7f85be800700 1 --2- 172.21.15.90:0/4206095423 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f85a00a3cc0 0x7f85a00a6180 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:42.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:42.049+0000 7f85be800700 1 --2- 172.21.15.90:0/4206095423 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85b8105830 0x7f85b819b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:42.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:42.049+0000 7f85be800700 1 -- 172.21.15.90:0/4206095423 >> 172.21.15.90:0/4206095423 conn(0x7f85b8100710 msgr2=0x7f85b8101260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:20:42.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:42.049+0000 7f85be800700 1 -- 172.21.15.90:0/4206095423 shutdown_connections 2024-04-24T09:20:42.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:42.049+0000 7f85be800700 1 -- 172.21.15.90:0/4206095423 wait complete. 2024-04-24T09:20:42.209 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411404 got 77309411412 for osd.2 2024-04-24T09:20:42.209 DEBUG:teuthology.parallel:result is None 2024-04-24T09:20:42.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:42 smithi090 ceph-mon[27825]: pgmap v370: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-24T09:20:42.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:42 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/4206095423' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-24T09:20:42.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:42 smithi114 ceph-mon[32501]: pgmap v370: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-24T09:20:42.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:42 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/4206095423' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-24T09:20:43.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:42 smithi149 ceph-mon[32592]: pgmap v370: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-24T09:20:43.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:42 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/4206095423' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-24T09:20:43.200 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892351003 got 347892351010 for osd.11 2024-04-24T09:20:43.200 DEBUG:teuthology.parallel:result is None 2024-04-24T09:20:44.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.044+0000 7f46ad800700 1 -- 172.21.15.90:0/3429322327 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46a8108b60 msgr2=0x7f46a8108f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:44.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.044+0000 7f46ad800700 1 --2- 172.21.15.90:0/3429322327 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46a8108b60 0x7f46a8108f40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4698009990 tx=0x7f4698009ca0 comp rx=0 tx=0).stop 2024-04-24T09:20:44.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.044+0000 7f46ad800700 1 -- 172.21.15.90:0/3429322327 shutdown_connections 2024-04-24T09:20:44.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.044+0000 7f46ad800700 1 --2- 172.21.15.90:0/3429322327 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46a806ba30 0x7f46a810f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:44.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.044+0000 7f46ad800700 1 --2- 172.21.15.90:0/3429322327 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46a8108b60 0x7f46a8108f40 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:44.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.044+0000 7f46ad800700 1 --2- 172.21.15.90:0/3429322327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46a806b090 0x7f46a806b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:44.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.044+0000 7f46ad800700 1 -- 172.21.15.90:0/3429322327 >> 172.21.15.90:0/3429322327 conn(0x7f46a8078960 msgr2=0x7f46a8078d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:20:44.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.044+0000 7f46ad800700 1 -- 172.21.15.90:0/3429322327 shutdown_connections 2024-04-24T09:20:44.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.044+0000 7f46ad800700 1 -- 172.21.15.90:0/3429322327 wait complete. 2024-04-24T09:20:44.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.045+0000 7f46ad800700 1 Processor -- start 2024-04-24T09:20:44.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.045+0000 7f46ad800700 1 -- start start 2024-04-24T09:20:44.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.046+0000 7f46ad800700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46a806b090 0x7f46a819b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:44.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.046+0000 7f46ad800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46a806ba30 0x7f46a819bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:44.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.046+0000 7f46ad800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46a8108b60 0x7f46a819fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:44.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.046+0000 7f46ad800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f46a8112df0 con 0x7f46a806ba30 2024-04-24T09:20:44.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.046+0000 7f46ad800700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f46a8112c70 con 0x7f46a806b090 2024-04-24T09:20:44.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.046+0000 7f46ad800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f46a8112f70 con 0x7f46a8108b60 2024-04-24T09:20:44.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.046+0000 7f46a6000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46a806ba30 0x7f46a819bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:44.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.046+0000 7f46a6000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46a806ba30 0x7f46a819bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44328/0 (socket says 172.21.15.90:44328) 2024-04-24T09:20:44.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.046+0000 7f46a6000700 1 -- 172.21.15.90:0/3203610859 learned_addr learned my addr 172.21.15.90:0/3203610859 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:20:44.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.046+0000 7f46a6a00700 1 --2- 172.21.15.90:0/3203610859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46a806b090 0x7f46a819b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:44.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.046+0000 7f46a7400700 1 --2- 172.21.15.90:0/3203610859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46a8108b60 0x7f46a819fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:44.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.046+0000 7f46a6000700 1 -- 172.21.15.90:0/3203610859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46a8108b60 msgr2=0x7f46a819fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:44.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.046+0000 7f46a6000700 1 --2- 172.21.15.90:0/3203610859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46a8108b60 0x7f46a819fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:44.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.046+0000 7f46a6000700 1 -- 172.21.15.90:0/3203610859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46a806b090 msgr2=0x7f46a819b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:44.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.046+0000 7f46a6000700 1 --2- 172.21.15.90:0/3203610859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46a806b090 0x7f46a819b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:44.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.046+0000 7f46a6000700 1 -- 172.21.15.90:0/3203610859 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a81a0540 con 0x7f46a806ba30 2024-04-24T09:20:44.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.046+0000 7f46a6a00700 1 --2- 172.21.15.90:0/3203610859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46a806b090 0x7f46a819b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:20:44.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.047+0000 7f46a6000700 1 --2- 172.21.15.90:0/3203610859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46a806ba30 0x7f46a819bac0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f469400d980 tx=0x7f469400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:20:44.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.047+0000 7f469f400700 1 -- 172.21.15.90:0/3203610859 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4694016070 con 0x7f46a806ba30 2024-04-24T09:20:44.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.047+0000 7f46ad800700 1 -- 172.21.15.90:0/3203610859 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46a81a0830 con 0x7f46a806ba30 2024-04-24T09:20:44.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.047+0000 7f469f400700 1 -- 172.21.15.90:0/3203610859 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4694004d10 con 0x7f46a806ba30 2024-04-24T09:20:44.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.047+0000 7f469f400700 1 -- 172.21.15.90:0/3203610859 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4694005670 con 0x7f46a806ba30 2024-04-24T09:20:44.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.047+0000 7f46ad800700 1 -- 172.21.15.90:0/3203610859 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46a81a80b0 con 0x7f46a806ba30 2024-04-24T09:20:44.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.048+0000 7f46ad800700 1 -- 172.21.15.90:0/3203610859 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46a80697e0 con 0x7f46a806ba30 2024-04-24T09:20:44.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.050+0000 7f469f400700 1 -- 172.21.15.90:0/3203610859 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4694004750 con 0x7f46a806ba30 2024-04-24T09:20:44.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.052+0000 7f469f400700 1 --2- 172.21.15.90:0/3203610859 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f468c0a3c40 0x7f468c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:44.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.052+0000 7f469f400700 1 -- 172.21.15.90:0/3203610859 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f46940d06c0 con 0x7f46a806ba30 2024-04-24T09:20:44.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.052+0000 7f46a6a00700 1 --2- 172.21.15.90:0/3203610859 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f468c0a3c40 0x7f468c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:44.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.053+0000 7f46a6a00700 1 --2- 172.21.15.90:0/3203610859 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f468c0a3c40 0x7f468c0a6100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4698006010 tx=0x7f469800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:20:44.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.054+0000 7f469f400700 1 -- 172.21.15.90:0/3203610859 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f469401f020 con 0x7f46a806ba30 2024-04-24T09:20:44.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.303+0000 7f46ad800700 1 -- 172.21.15.90:0/3203610859 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f46a802dfe0 con 0x7f46a806ba30 2024-04-24T09:20:44.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.304+0000 7f469f400700 1 -- 172.21.15.90:0/3203610859 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f469409d060 con 0x7f46a806ba30 2024-04-24T09:20:44.306 INFO:teuthology.orchestra.run.smithi090.stdout:146028888138 2024-04-24T09:20:44.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.307+0000 7f46ad800700 1 -- 172.21.15.90:0/3203610859 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f468c0a3c40 msgr2=0x7f468c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:44.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.307+0000 7f46ad800700 1 --2- 172.21.15.90:0/3203610859 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f468c0a3c40 0x7f468c0a6100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4698006010 tx=0x7f469800b260 comp rx=0 tx=0).stop 2024-04-24T09:20:44.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.307+0000 7f46ad800700 1 -- 172.21.15.90:0/3203610859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46a806ba30 msgr2=0x7f46a819bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:44.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.307+0000 7f46ad800700 1 --2- 172.21.15.90:0/3203610859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46a806ba30 0x7f46a819bac0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f469400d980 tx=0x7f469400dc90 comp rx=0 tx=0).stop 2024-04-24T09:20:44.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.307+0000 7f46ad800700 1 -- 172.21.15.90:0/3203610859 shutdown_connections 2024-04-24T09:20:44.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.307+0000 7f46ad800700 1 --2- 172.21.15.90:0/3203610859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46a8108b60 0x7f46a819fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:44.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.307+0000 7f46ad800700 1 --2- 172.21.15.90:0/3203610859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f46a806b090 0x7f46a819b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:44.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.307+0000 7f46ad800700 1 --2- 172.21.15.90:0/3203610859 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f468c0a3c40 0x7f468c0a6100 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:44.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.307+0000 7f46ad800700 1 --2- 172.21.15.90:0/3203610859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46a806ba30 0x7f46a819bac0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:44.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.307+0000 7f46ad800700 1 -- 172.21.15.90:0/3203610859 >> 172.21.15.90:0/3203610859 conn(0x7f46a8078960 msgr2=0x7f46a810df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:20:44.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.308+0000 7f46ad800700 1 -- 172.21.15.90:0/3203610859 shutdown_connections 2024-04-24T09:20:44.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:44.308+0000 7f46ad800700 1 -- 172.21.15.90:0/3203610859 wait complete. 2024-04-24T09:20:44.914 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:44 smithi149 ceph-mon[32592]: pgmap v371: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-24T09:20:44.914 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:44 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/3203610859' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-24T09:20:44.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:44 smithi114 ceph-mon[32501]: pgmap v371: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-24T09:20:44.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:44 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/3203610859' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-24T09:20:44.970 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:44 smithi090 ceph-mon[27825]: pgmap v371: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-24T09:20:44.970 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:44 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/3203610859' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-24T09:20:45.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.018+0000 7f3673c00700 1 -- 172.21.15.90:0/3660736764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f366c06a800 msgr2=0x7f366c06ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:45.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.018+0000 7f3673c00700 1 --2- 172.21.15.90:0/3660736764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f366c06a800 0x7f366c06ac60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f36580098f0 tx=0x7f3658009c00 comp rx=0 tx=0).stop 2024-04-24T09:20:45.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.018+0000 7f3673c00700 1 -- 172.21.15.90:0/3660736764 shutdown_connections 2024-04-24T09:20:45.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.018+0000 7f3673c00700 1 --2- 172.21.15.90:0/3660736764 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f366c06b1a0 0x7f366c10f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:45.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.018+0000 7f3673c00700 1 --2- 172.21.15.90:0/3660736764 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f366c108b70 0x7f366c108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:45.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.018+0000 7f3673c00700 1 --2- 172.21.15.90:0/3660736764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f366c06a800 0x7f366c06ac60 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:45.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.018+0000 7f3673c00700 1 -- 172.21.15.90:0/3660736764 >> 172.21.15.90:0/3660736764 conn(0x7f366c0789a0 msgr2=0x7f366c078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:20:45.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.018+0000 7f3673c00700 1 -- 172.21.15.90:0/3660736764 shutdown_connections 2024-04-24T09:20:45.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.018+0000 7f3673c00700 1 -- 172.21.15.90:0/3660736764 wait complete. 2024-04-24T09:20:45.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.019+0000 7f3673c00700 1 Processor -- start 2024-04-24T09:20:45.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.020+0000 7f3673c00700 1 -- start start 2024-04-24T09:20:45.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.020+0000 7f3673c00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f366c06a800 0x7f366c19b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:45.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.020+0000 7f3673c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f366c06b1a0 0x7f366c19ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:45.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.020+0000 7f3673c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f366c108b70 0x7f366c19fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:45.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.020+0000 7f3673c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f366c112dd0 con 0x7f366c06b1a0 2024-04-24T09:20:45.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.020+0000 7f3673c00700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f366c112c50 con 0x7f366c06a800 2024-04-24T09:20:45.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.020+0000 7f3673c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f366c112f50 con 0x7f366c108b70 2024-04-24T09:20:45.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.021+0000 7f366be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f366c06b1a0 0x7f366c19ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:45.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.021+0000 7f366be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f366c06b1a0 0x7f366c19ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44352/0 (socket says 172.21.15.90:44352) 2024-04-24T09:20:45.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.021+0000 7f366be00700 1 -- 172.21.15.90:0/3841857862 learned_addr learned my addr 172.21.15.90:0/3841857862 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:20:45.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.021+0000 7f3671c00700 1 --2- 172.21.15.90:0/3841857862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f366c108b70 0x7f366c19fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:45.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.021+0000 7f3671200700 1 --2- 172.21.15.90:0/3841857862 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f366c06a800 0x7f366c19b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:45.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.021+0000 7f366be00700 1 -- 172.21.15.90:0/3841857862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f366c108b70 msgr2=0x7f366c19fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:45.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.021+0000 7f366be00700 1 --2- 172.21.15.90:0/3841857862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f366c108b70 0x7f366c19fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:45.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.021+0000 7f366be00700 1 -- 172.21.15.90:0/3841857862 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f366c06a800 msgr2=0x7f366c19b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:45.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.021+0000 7f366be00700 1 --2- 172.21.15.90:0/3841857862 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f366c06a800 0x7f366c19b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:45.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.022+0000 7f366be00700 1 -- 172.21.15.90:0/3841857862 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f366c1a04f0 con 0x7f366c06b1a0 2024-04-24T09:20:45.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.022+0000 7f3671c00700 1 --2- 172.21.15.90:0/3841857862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f366c108b70 0x7f366c19fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:20:45.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.022+0000 7f366be00700 1 --2- 172.21.15.90:0/3841857862 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f366c06b1a0 0x7f366c19ba20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3658009230 tx=0x7f365800fb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:20:45.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.022+0000 7f3669600700 1 -- 172.21.15.90:0/3841857862 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36580052b0 con 0x7f366c06b1a0 2024-04-24T09:20:45.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.022+0000 7f3669600700 1 -- 172.21.15.90:0/3841857862 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f365800ba40 con 0x7f366c06b1a0 2024-04-24T09:20:45.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.022+0000 7f3669600700 1 -- 172.21.15.90:0/3841857862 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3658003c30 con 0x7f366c06b1a0 2024-04-24T09:20:45.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.022+0000 7f3673c00700 1 -- 172.21.15.90:0/3841857862 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f366c1a0780 con 0x7f366c06b1a0 2024-04-24T09:20:45.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.023+0000 7f3673c00700 1 -- 172.21.15.90:0/3841857862 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f366c1a0bb0 con 0x7f366c06b1a0 2024-04-24T09:20:45.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.024+0000 7f3673c00700 1 -- 172.21.15.90:0/3841857862 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f366c0697e0 con 0x7f366c06b1a0 2024-04-24T09:20:45.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.026+0000 7f3669600700 1 -- 172.21.15.90:0/3841857862 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3658029060 con 0x7f366c06b1a0 2024-04-24T09:20:45.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.027+0000 7f3669600700 1 --2- 172.21.15.90:0/3841857862 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f36540a3c40 0x7f36540a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:45.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.028+0000 7f3669600700 1 -- 172.21.15.90:0/3841857862 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f36580d0d90 con 0x7f366c06b1a0 2024-04-24T09:20:45.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.028+0000 7f3671200700 1 --2- 172.21.15.90:0/3841857862 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f36540a3c40 0x7f36540a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:45.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.028+0000 7f3671200700 1 --2- 172.21.15.90:0/3841857862 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f36540a3c40 0x7f36540a6100 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3660009bd0 tx=0x7f3660009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:20:45.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.032+0000 7f3669600700 1 -- 172.21.15.90:0/3841857862 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3658013030 con 0x7f366c06b1a0 2024-04-24T09:20:45.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.270+0000 7f3673c00700 1 -- 172.21.15.90:0/3841857862 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f366c02dfe0 con 0x7f366c06b1a0 2024-04-24T09:20:45.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.275+0000 7f3669600700 1 -- 172.21.15.90:0/3841857862 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f365809d7c0 con 0x7f366c06b1a0 2024-04-24T09:20:45.275 INFO:teuthology.orchestra.run.smithi090.stdout:201863462975 2024-04-24T09:20:45.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.278+0000 7f3673c00700 1 -- 172.21.15.90:0/3841857862 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f36540a3c40 msgr2=0x7f36540a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:45.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.278+0000 7f3673c00700 1 --2- 172.21.15.90:0/3841857862 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f36540a3c40 0x7f36540a6100 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3660009bd0 tx=0x7f3660009260 comp rx=0 tx=0).stop 2024-04-24T09:20:45.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.278+0000 7f3673c00700 1 -- 172.21.15.90:0/3841857862 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f366c06b1a0 msgr2=0x7f366c19ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:45.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.278+0000 7f3673c00700 1 --2- 172.21.15.90:0/3841857862 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f366c06b1a0 0x7f366c19ba20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3658009230 tx=0x7f365800fb20 comp rx=0 tx=0).stop 2024-04-24T09:20:45.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.279+0000 7f3673c00700 1 -- 172.21.15.90:0/3841857862 shutdown_connections 2024-04-24T09:20:45.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.279+0000 7f3673c00700 1 --2- 172.21.15.90:0/3841857862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f366c108b70 0x7f366c19fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:45.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.279+0000 7f3673c00700 1 --2- 172.21.15.90:0/3841857862 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f366c06a800 0x7f366c19b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:45.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.279+0000 7f3673c00700 1 --2- 172.21.15.90:0/3841857862 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f36540a3c40 0x7f36540a6100 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:45.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.279+0000 7f3673c00700 1 --2- 172.21.15.90:0/3841857862 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f366c06b1a0 0x7f366c19ba20 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:45.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.279+0000 7f3673c00700 1 -- 172.21.15.90:0/3841857862 >> 172.21.15.90:0/3841857862 conn(0x7f366c0789a0 msgr2=0x7f366c10dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:20:45.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.279+0000 7f3673c00700 1 -- 172.21.15.90:0/3841857862 shutdown_connections 2024-04-24T09:20:45.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:45.279+0000 7f3673c00700 1 -- 172.21.15.90:0/3841857862 wait complete. 2024-04-24T09:20:45.876 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888129 got 146028888138 for osd.4 2024-04-24T09:20:45.876 DEBUG:teuthology.parallel:result is None 2024-04-24T09:20:46.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:46 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:20:46.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:46 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/3841857862' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-24T09:20:46.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:46 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:20:46.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:46 smithi090 ceph-mon[27825]: pgmap v372: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-24T09:20:46.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:46 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:20:46.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:46 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/3841857862' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-24T09:20:46.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:46 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:20:46.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:46 smithi149 ceph-mon[32592]: pgmap v372: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-24T09:20:46.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:46 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:20:46.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:46 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/3841857862' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-24T09:20:46.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:46 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:20:46.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:46 smithi114 ceph-mon[32501]: pgmap v372: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-24T09:20:46.974 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462967 got 201863462975 for osd.6 2024-04-24T09:20:46.974 DEBUG:teuthology.parallel:result is None 2024-04-24T09:20:46.975 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-24T09:20:46.975 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-24T09:20:47.407 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:20:47.552 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:47 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:20:47.552 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:47 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:20:47.552 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:47 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:20:47.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:47 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:20:47.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:47 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:20:47.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:47 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:20:47.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:47 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:20:47.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:47 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:20:47.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:47 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:20:48.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:48 smithi090 ceph-mon[27825]: pgmap v373: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:20:48.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:48 smithi149 ceph-mon[32592]: pgmap v373: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:20:48.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:48 smithi114 ceph-mon[32501]: pgmap v373: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:20:50.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.130+0000 7f68c5200700 1 -- 172.21.15.90:0/1533244722 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68c01099f0 msgr2=0x7f68c0115e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:50.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.130+0000 7f68c5200700 1 --2- 172.21.15.90:0/1533244722 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68c01099f0 0x7f68c0115e90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f68b800b0a0 tx=0x7f68b800b3b0 comp rx=0 tx=0).stop 2024-04-24T09:20:50.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.130+0000 7f68c5200700 1 -- 172.21.15.90:0/1533244722 shutdown_connections 2024-04-24T09:20:50.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.130+0000 7f68c5200700 1 --2- 172.21.15.90:0/1533244722 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68c01099f0 0x7f68c0115e90 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:50.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.130+0000 7f68c5200700 1 --2- 172.21.15.90:0/1533244722 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f68c010f0e0 0x7f68c010f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:50.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.130+0000 7f68c5200700 1 --2- 172.21.15.90:0/1533244722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68c0109030 0x7f68c01094b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:50.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.130+0000 7f68c5200700 1 -- 172.21.15.90:0/1533244722 >> 172.21.15.90:0/1533244722 conn(0x7f68c0078880 msgr2=0x7f68c0107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:20:50.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.130+0000 7f68c5200700 1 -- 172.21.15.90:0/1533244722 shutdown_connections 2024-04-24T09:20:50.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.130+0000 7f68c5200700 1 -- 172.21.15.90:0/1533244722 wait complete. 2024-04-24T09:20:50.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.131+0000 7f68c5200700 1 Processor -- start 2024-04-24T09:20:50.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.132+0000 7f68c5200700 1 -- start start 2024-04-24T09:20:50.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.132+0000 7f68c5200700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68c0109030 0x7f68c019f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:50.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.132+0000 7f68c5200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f68c01099f0 0x7f68c019fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:50.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.132+0000 7f68c5200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68c010f0e0 0x7f68c01a3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:50.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.132+0000 7f68c5200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f68c01192f0 con 0x7f68c01099f0 2024-04-24T09:20:50.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.132+0000 7f68c5200700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f68c0119170 con 0x7f68c0109030 2024-04-24T09:20:50.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.132+0000 7f68c5200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f68c0119470 con 0x7f68c010f0e0 2024-04-24T09:20:50.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.132+0000 7f68bea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f68c01099f0 0x7f68c019fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:50.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.132+0000 7f68bea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f68c01099f0 0x7f68c019fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50110/0 (socket says 172.21.15.90:50110) 2024-04-24T09:20:50.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.132+0000 7f68bea00700 1 -- 172.21.15.90:0/746841058 learned_addr learned my addr 172.21.15.90:0/746841058 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:20:50.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.132+0000 7f68bf400700 1 --2- 172.21.15.90:0/746841058 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68c0109030 0x7f68c019f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:50.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.132+0000 7f68bfe00700 1 --2- 172.21.15.90:0/746841058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68c010f0e0 0x7f68c01a3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:50.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.133+0000 7f68bea00700 1 -- 172.21.15.90:0/746841058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68c010f0e0 msgr2=0x7f68c01a3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:50.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.133+0000 7f68bea00700 1 --2- 172.21.15.90:0/746841058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68c010f0e0 0x7f68c01a3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:50.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.133+0000 7f68bea00700 1 -- 172.21.15.90:0/746841058 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68c0109030 msgr2=0x7f68c019f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:50.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.133+0000 7f68bea00700 1 --2- 172.21.15.90:0/746841058 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68c0109030 0x7f68c019f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:50.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.133+0000 7f68bea00700 1 -- 172.21.15.90:0/746841058 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68c01a46a0 con 0x7f68c01099f0 2024-04-24T09:20:50.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.133+0000 7f68bf400700 1 --2- 172.21.15.90:0/746841058 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68c0109030 0x7f68c019f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:20:50.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.133+0000 7f68bea00700 1 --2- 172.21.15.90:0/746841058 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f68c01099f0 0x7f68c019fc20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f68ac00e9a0 tx=0x7f68ac00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:20:50.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.133+0000 7f68b7e00700 1 -- 172.21.15.90:0/746841058 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac00ca10 con 0x7f68c01099f0 2024-04-24T09:20:50.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.133+0000 7f68b7e00700 1 -- 172.21.15.90:0/746841058 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68ac00cbb0 con 0x7f68c01099f0 2024-04-24T09:20:50.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.133+0000 7f68c5200700 1 -- 172.21.15.90:0/746841058 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68c01a4990 con 0x7f68c01099f0 2024-04-24T09:20:50.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.134+0000 7f68b7e00700 1 -- 172.21.15.90:0/746841058 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac018680 con 0x7f68c01099f0 2024-04-24T09:20:50.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.134+0000 7f68c5200700 1 -- 172.21.15.90:0/746841058 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68c01ac210 con 0x7f68c01099f0 2024-04-24T09:20:50.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.135+0000 7f68c5200700 1 -- 172.21.15.90:0/746841058 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68c00697e0 con 0x7f68c01099f0 2024-04-24T09:20:50.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.137+0000 7f68b7e00700 1 -- 172.21.15.90:0/746841058 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f68ac010430 con 0x7f68c01099f0 2024-04-24T09:20:50.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.138+0000 7f68b7e00700 1 --2- 172.21.15.90:0/746841058 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f68a40a3c40 0x7f68a40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:50.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.138+0000 7f68b7e00700 1 -- 172.21.15.90:0/746841058 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f68ac0d0640 con 0x7f68c01099f0 2024-04-24T09:20:50.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.138+0000 7f68bf400700 1 --2- 172.21.15.90:0/746841058 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f68a40a3c40 0x7f68a40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:50.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.139+0000 7f68bf400700 1 --2- 172.21.15.90:0/746841058 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f68a40a3c40 0x7f68a40a6100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f68b0011490 tx=0x7f68b0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:20:50.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.143+0000 7f68b7e00700 1 -- 172.21.15.90:0/746841058 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f68ac09cfe0 con 0x7f68c01099f0 2024-04-24T09:20:50.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.365+0000 7f68c5200700 1 -- 172.21.15.90:0/746841058 --> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f68c01a0a20 con 0x7f68a40a3c40 2024-04-24T09:20:50.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.377+0000 7f68b7e00700 1 -- 172.21.15.90:0/746841058 <== mgr.14152 v2:172.21.15.90:6800/4261662956 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259163 (secure 0 0 0) 0x7f68c01a0a20 con 0x7f68a40a3c40 2024-04-24T09:20:50.382 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:20:50.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.383+0000 7f68c5200700 1 -- 172.21.15.90:0/746841058 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f68a40a3c40 msgr2=0x7f68a40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:50.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.383+0000 7f68c5200700 1 --2- 172.21.15.90:0/746841058 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f68a40a3c40 0x7f68a40a6100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f68b0011490 tx=0x7f68b0009260 comp rx=0 tx=0).stop 2024-04-24T09:20:50.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.383+0000 7f68c5200700 1 -- 172.21.15.90:0/746841058 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f68c01099f0 msgr2=0x7f68c019fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:50.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.383+0000 7f68c5200700 1 --2- 172.21.15.90:0/746841058 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f68c01099f0 0x7f68c019fc20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f68ac00e9a0 tx=0x7f68ac00ecb0 comp rx=0 tx=0).stop 2024-04-24T09:20:50.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.383+0000 7f68c5200700 1 -- 172.21.15.90:0/746841058 shutdown_connections 2024-04-24T09:20:50.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.383+0000 7f68c5200700 1 --2- 172.21.15.90:0/746841058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68c010f0e0 0x7f68c01a3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:50.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.383+0000 7f68c5200700 1 --2- 172.21.15.90:0/746841058 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68c0109030 0x7f68c019f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:50.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.383+0000 7f68c5200700 1 --2- 172.21.15.90:0/746841058 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f68a40a3c40 0x7f68a40a6100 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:50.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.383+0000 7f68c5200700 1 --2- 172.21.15.90:0/746841058 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f68c01099f0 0x7f68c019fc20 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:50.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.383+0000 7f68c5200700 1 -- 172.21.15.90:0/746841058 >> 172.21.15.90:0/746841058 conn(0x7f68c0078880 msgr2=0x7f68c0079740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:20:50.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.383+0000 7f68c5200700 1 -- 172.21.15.90:0/746841058 shutdown_connections 2024-04-24T09:20:50.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:50.383+0000 7f68c5200700 1 -- 172.21.15.90:0/746841058 wait complete. 2024-04-24T09:20:50.385 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-24T09:20:50.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:50 smithi114 ceph-mon[32501]: pgmap v374: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-24T09:20:51.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:50 smithi090 ceph-mon[27825]: pgmap v374: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-24T09:20:51.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:50 smithi149 ceph-mon[32592]: pgmap v374: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-24T09:20:51.352 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":374,"stamp":"2024-04-24T09:20:49.561827+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658851,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1036,"num_read_kb":663,"num_write":426,"num_write_kb":3077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1182023,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":850,"ondisk_log_size":850,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369268,"kb_used_data":11796,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572556,"statfs":{"total":1151940427776,"available":1151562297344,"internally_reserved":0,"allocated":12079104,"data_stored":7481589,"data_compressed":182403,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":125,"num_read_kb":65,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005849"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.549323+0000","last_change":"2024-04-24T09:19:09.860043+0000","last_active":"2024-04-24T09:20:27.549323+0000","last_peered":"2024-04-24T09:20:27.549323+0000","last_clean":"2024-04-24T09:20:27.549323+0000","last_became_active":"2024-04-24T09:19:09.859766+0000","last_became_peered":"2024-04-24T09:19:09.859766+0000","last_unstale":"2024-04-24T09:20:27.549323+0000","last_undegraded":"2024-04-24T09:20:27.549323+0000","last_fullsized":"2024-04-24T09:20:27.549323+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:40:46.241239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.551329+0000","last_change":"2024-04-24T09:19:09.847293+0000","last_active":"2024-04-24T09:20:27.551329+0000","last_peered":"2024-04-24T09:20:27.551329+0000","last_clean":"2024-04-24T09:20:27.551329+0000","last_became_active":"2024-04-24T09:19:09.846955+0000","last_became_peered":"2024-04-24T09:19:09.846955+0000","last_unstale":"2024-04-24T09:20:27.551329+0000","last_undegraded":"2024-04-24T09:20:27.551329+0000","last_fullsized":"2024-04-24T09:20:27.551329+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:38:10.062654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.555852+0000","last_change":"2024-04-24T09:19:09.858954+0000","last_active":"2024-04-24T09:20:27.555852+0000","last_peered":"2024-04-24T09:20:27.555852+0000","last_clean":"2024-04-24T09:20:27.555852+0000","last_became_active":"2024-04-24T09:19:09.855915+0000","last_became_peered":"2024-04-24T09:19:09.855915+0000","last_unstale":"2024-04-24T09:20:27.555852+0000","last_undegraded":"2024-04-24T09:20:27.555852+0000","last_fullsized":"2024-04-24T09:20:27.555852+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:12:44.214700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.537045+0000","last_change":"2024-04-24T09:19:09.848582+0000","last_active":"2024-04-24T09:20:27.537045+0000","last_peered":"2024-04-24T09:20:27.537045+0000","last_clean":"2024-04-24T09:20:27.537045+0000","last_became_active":"2024-04-24T09:19:09.848163+0000","last_became_peered":"2024-04-24T09:19:09.848163+0000","last_unstale":"2024-04-24T09:20:27.537045+0000","last_undegraded":"2024-04-24T09:20:27.537045+0000","last_fullsized":"2024-04-24T09:20:27.537045+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:33:46.106081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.544380+0000","last_change":"2024-04-24T09:19:09.864608+0000","last_active":"2024-04-24T09:20:27.544380+0000","last_peered":"2024-04-24T09:20:27.544380+0000","last_clean":"2024-04-24T09:20:27.544380+0000","last_became_active":"2024-04-24T09:19:09.864271+0000","last_became_peered":"2024-04-24T09:19:09.864271+0000","last_unstale":"2024-04-24T09:20:27.544380+0000","last_undegraded":"2024-04-24T09:20:27.544380+0000","last_fullsized":"2024-04-24T09:20:27.544380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:08:57.535882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.547057+0000","last_change":"2024-04-24T09:19:09.838589+0000","last_active":"2024-04-24T09:20:27.547057+0000","last_peered":"2024-04-24T09:20:27.547057+0000","last_clean":"2024-04-24T09:20:27.547057+0000","last_became_active":"2024-04-24T09:19:09.837628+0000","last_became_peered":"2024-04-24T09:19:09.837628+0000","last_unstale":"2024-04-24T09:20:27.547057+0000","last_undegraded":"2024-04-24T09:20:27.547057+0000","last_fullsized":"2024-04-24T09:20:27.547057+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:29:56.967777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.551958+0000","last_change":"2024-04-24T09:19:09.846296+0000","last_active":"2024-04-24T09:20:27.551958+0000","last_peered":"2024-04-24T09:20:27.551958+0000","last_clean":"2024-04-24T09:20:27.551958+0000","last_became_active":"2024-04-24T09:19:09.846010+0000","last_became_peered":"2024-04-24T09:19:09.846010+0000","last_unstale":"2024-04-24T09:20:27.551958+0000","last_undegraded":"2024-04-24T09:20:27.551958+0000","last_fullsized":"2024-04-24T09:20:27.551958+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:29:38.507747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.561542+0000","last_change":"2024-04-24T09:19:09.866821+0000","last_active":"2024-04-24T09:20:27.561542+0000","last_peered":"2024-04-24T09:20:27.561542+0000","last_clean":"2024-04-24T09:20:27.561542+0000","last_became_active":"2024-04-24T09:19:09.861793+0000","last_became_peered":"2024-04-24T09:19:09.861793+0000","last_unstale":"2024-04-24T09:20:27.561542+0000","last_undegraded":"2024-04-24T09:20:27.561542+0000","last_fullsized":"2024-04-24T09:20:27.561542+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:25:25.095642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.546659+0000","last_change":"2024-04-24T09:19:09.846433+0000","last_active":"2024-04-24T09:20:27.546659+0000","last_peered":"2024-04-24T09:20:27.546659+0000","last_clean":"2024-04-24T09:20:27.546659+0000","last_became_active":"2024-04-24T09:19:09.846044+0000","last_became_peered":"2024-04-24T09:19:09.846044+0000","last_unstale":"2024-04-24T09:20:27.546659+0000","last_undegraded":"2024-04-24T09:20:27.546659+0000","last_fullsized":"2024-04-24T09:20:27.546659+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:02:41.566626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.547736+0000","last_change":"2024-04-24T09:19:09.836662+0000","last_active":"2024-04-24T09:20:27.547736+0000","last_peered":"2024-04-24T09:20:27.547736+0000","last_clean":"2024-04-24T09:20:27.547736+0000","last_became_active":"2024-04-24T09:19:09.836283+0000","last_became_peered":"2024-04-24T09:19:09.836283+0000","last_unstale":"2024-04-24T09:20:27.547736+0000","last_undegraded":"2024-04-24T09:20:27.547736+0000","last_fullsized":"2024-04-24T09:20:27.547736+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:16:16.503469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.549856+0000","last_change":"2024-04-24T09:19:09.844369+0000","last_active":"2024-04-24T09:20:27.549856+0000","last_peered":"2024-04-24T09:20:27.549856+0000","last_clean":"2024-04-24T09:20:27.549856+0000","last_became_active":"2024-04-24T09:19:09.844044+0000","last_became_peered":"2024-04-24T09:19:09.844044+0000","last_unstale":"2024-04-24T09:20:27.549856+0000","last_undegraded":"2024-04-24T09:20:27.549856+0000","last_fullsized":"2024-04-24T09:20:27.549856+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:51:45.861678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.537668+0000","last_change":"2024-04-24T09:19:58.958306+0000","last_active":"2024-04-24T09:20:27.537668+0000","last_peered":"2024-04-24T09:20:27.537668+0000","last_clean":"2024-04-24T09:20:27.537668+0000","last_became_active":"2024-04-24T09:19:58.957743+0000","last_became_peered":"2024-04-24T09:19:58.957743+0000","last_unstale":"2024-04-24T09:20:27.537668+0000","last_undegraded":"2024-04-24T09:20:27.537668+0000","last_fullsized":"2024-04-24T09:20:27.537668+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:54:07.144064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.562128+0000","last_change":"2024-04-24T09:19:09.847860+0000","last_active":"2024-04-24T09:20:27.562128+0000","last_peered":"2024-04-24T09:20:27.562128+0000","last_clean":"2024-04-24T09:20:27.562128+0000","last_became_active":"2024-04-24T09:19:09.847477+0000","last_became_peered":"2024-04-24T09:19:09.847477+0000","last_unstale":"2024-04-24T09:20:27.562128+0000","last_undegraded":"2024-04-24T09:20:27.562128+0000","last_fullsized":"2024-04-24T09:20:27.562128+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:39:44.655033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.555576+0000","last_change":"2024-04-24T09:19:09.854561+0000","last_active":"2024-04-24T09:20:27.555576+0000","last_peered":"2024-04-24T09:20:27.555576+0000","last_clean":"2024-04-24T09:20:27.555576+0000","last_became_active":"2024-04-24T09:19:09.849681+0000","last_became_peered":"2024-04-24T09:19:09.849681+0000","last_unstale":"2024-04-24T09:20:27.555576+0000","last_undegraded":"2024-04-24T09:20:27.555576+0000","last_fullsized":"2024-04-24T09:20:27.555576+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:46:36.572491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.555184+0000","last_change":"2024-04-24T09:19:58.620539+0000","last_active":"2024-04-24T09:20:27.555184+0000","last_peered":"2024-04-24T09:20:27.555184+0000","last_clean":"2024-04-24T09:20:27.555184+0000","last_became_active":"2024-04-24T09:19:58.620243+0000","last_became_peered":"2024-04-24T09:19:58.620243+0000","last_unstale":"2024-04-24T09:20:27.555184+0000","last_undegraded":"2024-04-24T09:20:27.555184+0000","last_fullsized":"2024-04-24T09:20:27.555184+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:35:34.448731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.558031+0000","last_change":"2024-04-24T09:19:09.837995+0000","last_active":"2024-04-24T09:20:27.558031+0000","last_peered":"2024-04-24T09:20:27.558031+0000","last_clean":"2024-04-24T09:20:27.558031+0000","last_became_active":"2024-04-24T09:19:09.837625+0000","last_became_peered":"2024-04-24T09:19:09.837625+0000","last_unstale":"2024-04-24T09:20:27.558031+0000","last_undegraded":"2024-04-24T09:20:27.558031+0000","last_fullsized":"2024-04-24T09:20:27.558031+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:14:30.602137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.549548+0000","last_change":"2024-04-24T09:19:09.855596+0000","last_active":"2024-04-24T09:20:27.549548+0000","last_peered":"2024-04-24T09:20:27.549548+0000","last_clean":"2024-04-24T09:20:27.549548+0000","last_became_active":"2024-04-24T09:19:09.855250+0000","last_became_peered":"2024-04-24T09:19:09.855250+0000","last_unstale":"2024-04-24T09:20:27.549548+0000","last_undegraded":"2024-04-24T09:20:27.549548+0000","last_fullsized":"2024-04-24T09:20:27.549548+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:48:58.862920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.554044+0000","last_change":"2024-04-24T09:19:09.844898+0000","last_active":"2024-04-24T09:20:27.554044+0000","last_peered":"2024-04-24T09:20:27.554044+0000","last_clean":"2024-04-24T09:20:27.554044+0000","last_became_active":"2024-04-24T09:19:09.844566+0000","last_became_peered":"2024-04-24T09:19:09.844566+0000","last_unstale":"2024-04-24T09:20:27.554044+0000","last_undegraded":"2024-04-24T09:20:27.554044+0000","last_fullsized":"2024-04-24T09:20:27.554044+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:15:34.145221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.555152+0000","last_change":"2024-04-24T09:19:09.856299+0000","last_active":"2024-04-24T09:20:27.555152+0000","last_peered":"2024-04-24T09:20:27.555152+0000","last_clean":"2024-04-24T09:20:27.555152+0000","last_became_active":"2024-04-24T09:19:09.855963+0000","last_became_peered":"2024-04-24T09:19:09.855963+0000","last_unstale":"2024-04-24T09:20:27.555152+0000","last_undegraded":"2024-04-24T09:20:27.555152+0000","last_fullsized":"2024-04-24T09:20:27.555152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:10:27.372894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":123,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:45.422198+0000","last_change":"2024-04-24T09:19:08.973018+0000","last_active":"2024-04-24T09:20:45.422198+0000","last_peered":"2024-04-24T09:20:45.422198+0000","last_clean":"2024-04-24T09:20:45.422198+0000","last_became_active":"2024-04-24T09:19:08.972637+0000","last_became_peered":"2024-04-24T09:19:08.972637+0000","last_unstale":"2024-04-24T09:20:45.422198+0000","last_undegraded":"2024-04-24T09:20:45.422198+0000","last_fullsized":"2024-04-24T09:20:45.422198+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:11:10.890872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":91,"num_read_kb":91,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.550818+0000","last_change":"2024-04-24T09:19:09.862806+0000","last_active":"2024-04-24T09:20:27.550818+0000","last_peered":"2024-04-24T09:20:27.550818+0000","last_clean":"2024-04-24T09:20:27.550818+0000","last_became_active":"2024-04-24T09:19:09.862499+0000","last_became_peered":"2024-04-24T09:19:09.862499+0000","last_unstale":"2024-04-24T09:20:27.550818+0000","last_undegraded":"2024-04-24T09:20:27.550818+0000","last_fullsized":"2024-04-24T09:20:27.550818+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:13:19.494260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.624945+0000","last_change":"2024-04-24T09:19:08.973130+0000","last_active":"2024-04-24T09:19:58.624945+0000","last_peered":"2024-04-24T09:19:58.624945+0000","last_clean":"2024-04-24T09:19:58.624945+0000","last_became_active":"2024-04-24T09:19:08.972797+0000","last_became_peered":"2024-04-24T09:19:08.972797+0000","last_unstale":"2024-04-24T09:19:58.624945+0000","last_undegraded":"2024-04-24T09:19:58.624945+0000","last_fullsized":"2024-04-24T09:19:58.624945+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:21:49.236525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.557800+0000","last_change":"2024-04-24T09:19:09.860809+0000","last_active":"2024-04-24T09:20:27.557800+0000","last_peered":"2024-04-24T09:20:27.557800+0000","last_clean":"2024-04-24T09:20:27.557800+0000","last_became_active":"2024-04-24T09:19:09.860480+0000","last_became_peered":"2024-04-24T09:19:09.860480+0000","last_unstale":"2024-04-24T09:20:27.557800+0000","last_undegraded":"2024-04-24T09:20:27.557800+0000","last_fullsized":"2024-04-24T09:20:27.557800+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:39:21.114116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.617760+0000","last_change":"2024-04-24T09:19:08.950145+0000","last_active":"2024-04-24T09:19:58.617760+0000","last_peered":"2024-04-24T09:19:58.617760+0000","last_clean":"2024-04-24T09:19:58.617760+0000","last_became_active":"2024-04-24T09:19:08.949808+0000","last_became_peered":"2024-04-24T09:19:08.949808+0000","last_unstale":"2024-04-24T09:19:58.617760+0000","last_undegraded":"2024-04-24T09:19:58.617760+0000","last_fullsized":"2024-04-24T09:19:58.617760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:15:36.962348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.560944+0000","last_change":"2024-04-24T09:19:09.850123+0000","last_active":"2024-04-24T09:20:27.560944+0000","last_peered":"2024-04-24T09:20:27.560944+0000","last_clean":"2024-04-24T09:20:27.560944+0000","last_became_active":"2024-04-24T09:19:09.849770+0000","last_became_peered":"2024-04-24T09:19:09.849770+0000","last_unstale":"2024-04-24T09:20:27.560944+0000","last_undegraded":"2024-04-24T09:20:27.560944+0000","last_fullsized":"2024-04-24T09:20:27.560944+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:10:00.374950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.618394+0000","last_change":"2024-04-24T09:19:08.952190+0000","last_active":"2024-04-24T09:19:58.618394+0000","last_peered":"2024-04-24T09:19:58.618394+0000","last_clean":"2024-04-24T09:19:58.618394+0000","last_became_active":"2024-04-24T09:19:08.951848+0000","last_became_peered":"2024-04-24T09:19:08.951848+0000","last_unstale":"2024-04-24T09:19:58.618394+0000","last_undegraded":"2024-04-24T09:19:58.618394+0000","last_fullsized":"2024-04-24T09:19:58.618394+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:02:54.892658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.542753+0000","last_change":"2024-04-24T09:19:09.845083+0000","last_active":"2024-04-24T09:20:27.542753+0000","last_peered":"2024-04-24T09:20:27.542753+0000","last_clean":"2024-04-24T09:20:27.542753+0000","last_became_active":"2024-04-24T09:19:09.844516+0000","last_became_peered":"2024-04-24T09:19:09.844516+0000","last_unstale":"2024-04-24T09:20:27.542753+0000","last_undegraded":"2024-04-24T09:20:27.542753+0000","last_fullsized":"2024-04-24T09:20:27.542753+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:10:41.995167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":116,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:44.383665+0000","last_change":"2024-04-24T09:19:08.950498+0000","last_active":"2024-04-24T09:20:44.383665+0000","last_peered":"2024-04-24T09:20:44.383665+0000","last_clean":"2024-04-24T09:20:44.383665+0000","last_became_active":"2024-04-24T09:19:08.950189+0000","last_became_peered":"2024-04-24T09:19:08.950189+0000","last_unstale":"2024-04-24T09:20:44.383665+0000","last_undegraded":"2024-04-24T09:20:44.383665+0000","last_fullsized":"2024-04-24T09:20:44.383665+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:40:41.342349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":92,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.531098+0000","last_change":"2024-04-24T09:19:09.847388+0000","last_active":"2024-04-24T09:20:27.531098+0000","last_peered":"2024-04-24T09:20:27.531098+0000","last_clean":"2024-04-24T09:20:27.531098+0000","last_became_active":"2024-04-24T09:19:09.846999+0000","last_became_peered":"2024-04-24T09:19:09.846999+0000","last_unstale":"2024-04-24T09:20:27.531098+0000","last_undegraded":"2024-04-24T09:20:27.531098+0000","last_fullsized":"2024-04-24T09:20:27.531098+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:49:09.814212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.956429+0000","last_change":"2024-04-24T09:19:08.965244+0000","last_active":"2024-04-24T09:19:58.956429+0000","last_peered":"2024-04-24T09:19:58.956429+0000","last_clean":"2024-04-24T09:19:58.956429+0000","last_became_active":"2024-04-24T09:19:08.964899+0000","last_became_peered":"2024-04-24T09:19:08.964899+0000","last_unstale":"2024-04-24T09:19:58.956429+0000","last_undegraded":"2024-04-24T09:19:58.956429+0000","last_fullsized":"2024-04-24T09:19:58.956429+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:53:28.035567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.544506+0000","last_change":"2024-04-24T09:19:09.864902+0000","last_active":"2024-04-24T09:20:27.544506+0000","last_peered":"2024-04-24T09:20:27.544506+0000","last_clean":"2024-04-24T09:20:27.544506+0000","last_became_active":"2024-04-24T09:19:09.864500+0000","last_became_peered":"2024-04-24T09:19:09.864500+0000","last_unstale":"2024-04-24T09:20:27.544506+0000","last_undegraded":"2024-04-24T09:20:27.544506+0000","last_fullsized":"2024-04-24T09:20:27.544506+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:37:31.810050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.952883+0000","last_change":"2024-04-24T09:19:08.977987+0000","last_active":"2024-04-24T09:19:58.952883+0000","last_peered":"2024-04-24T09:19:58.952883+0000","last_clean":"2024-04-24T09:19:58.952883+0000","last_became_active":"2024-04-24T09:19:08.977666+0000","last_became_peered":"2024-04-24T09:19:08.977666+0000","last_unstale":"2024-04-24T09:19:58.952883+0000","last_undegraded":"2024-04-24T09:19:58.952883+0000","last_fullsized":"2024-04-24T09:19:58.952883+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:21:21.328463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.559798+0000","last_change":"2024-04-24T09:19:09.863990+0000","last_active":"2024-04-24T09:20:27.559798+0000","last_peered":"2024-04-24T09:20:27.559798+0000","last_clean":"2024-04-24T09:20:27.559798+0000","last_became_active":"2024-04-24T09:19:09.854938+0000","last_became_peered":"2024-04-24T09:19:09.854938+0000","last_unstale":"2024-04-24T09:20:27.559798+0000","last_undegraded":"2024-04-24T09:20:27.559798+0000","last_fullsized":"2024-04-24T09:20:27.559798+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:41:55.054373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.620691+0000","last_change":"2024-04-24T09:19:08.949938+0000","last_active":"2024-04-24T09:19:58.620691+0000","last_peered":"2024-04-24T09:19:58.620691+0000","last_clean":"2024-04-24T09:19:58.620691+0000","last_became_active":"2024-04-24T09:19:08.949561+0000","last_became_peered":"2024-04-24T09:19:08.949561+0000","last_unstale":"2024-04-24T09:19:58.620691+0000","last_undegraded":"2024-04-24T09:19:58.620691+0000","last_fullsized":"2024-04-24T09:19:58.620691+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:55:50.488129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.553406+0000","last_change":"2024-04-24T09:19:09.841782+0000","last_active":"2024-04-24T09:20:27.553406+0000","last_peered":"2024-04-24T09:20:27.553406+0000","last_clean":"2024-04-24T09:20:27.553406+0000","last_became_active":"2024-04-24T09:19:09.841452+0000","last_became_peered":"2024-04-24T09:19:09.841452+0000","last_unstale":"2024-04-24T09:20:27.553406+0000","last_undegraded":"2024-04-24T09:20:27.553406+0000","last_fullsized":"2024-04-24T09:20:27.553406+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:58:19.618999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":209,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:48.389395+0000","last_change":"2024-04-24T09:19:08.973169+0000","last_active":"2024-04-24T09:20:48.389395+0000","last_peered":"2024-04-24T09:20:48.389395+0000","last_clean":"2024-04-24T09:20:48.389395+0000","last_became_active":"2024-04-24T09:19:08.972835+0000","last_became_peered":"2024-04-24T09:19:08.972835+0000","last_unstale":"2024-04-24T09:20:48.389395+0000","last_undegraded":"2024-04-24T09:20:48.389395+0000","last_fullsized":"2024-04-24T09:20:48.389395+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:34:04.156280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":186,"num_read_kb":186,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.556131+0000","last_change":"2024-04-24T09:19:09.854957+0000","last_active":"2024-04-24T09:20:27.556131+0000","last_peered":"2024-04-24T09:20:27.556131+0000","last_clean":"2024-04-24T09:20:27.556131+0000","last_became_active":"2024-04-24T09:19:09.854574+0000","last_became_peered":"2024-04-24T09:19:09.854574+0000","last_unstale":"2024-04-24T09:20:27.556131+0000","last_undegraded":"2024-04-24T09:20:27.556131+0000","last_fullsized":"2024-04-24T09:20:27.556131+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:07:28.296335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.618727+0000","last_change":"2024-04-24T09:19:08.953179+0000","last_active":"2024-04-24T09:19:58.618727+0000","last_peered":"2024-04-24T09:19:58.618727+0000","last_clean":"2024-04-24T09:19:58.618727+0000","last_became_active":"2024-04-24T09:19:08.952864+0000","last_became_peered":"2024-04-24T09:19:08.952864+0000","last_unstale":"2024-04-24T09:19:58.618727+0000","last_undegraded":"2024-04-24T09:19:58.618727+0000","last_fullsized":"2024-04-24T09:19:58.618727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:22:31.470593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.556402+0000","last_change":"2024-04-24T09:19:09.850855+0000","last_active":"2024-04-24T09:20:27.556402+0000","last_peered":"2024-04-24T09:20:27.556402+0000","last_clean":"2024-04-24T09:20:27.556402+0000","last_became_active":"2024-04-24T09:19:09.850535+0000","last_became_peered":"2024-04-24T09:19:09.850535+0000","last_unstale":"2024-04-24T09:20:27.556402+0000","last_undegraded":"2024-04-24T09:20:27.556402+0000","last_fullsized":"2024-04-24T09:20:27.556402+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:27:23.294935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.620766+0000","last_change":"2024-04-24T09:19:08.967882+0000","last_active":"2024-04-24T09:19:58.620766+0000","last_peered":"2024-04-24T09:19:58.620766+0000","last_clean":"2024-04-24T09:19:58.620766+0000","last_became_active":"2024-04-24T09:19:08.967625+0000","last_became_peered":"2024-04-24T09:19:08.967625+0000","last_unstale":"2024-04-24T09:19:58.620766+0000","last_undegraded":"2024-04-24T09:19:58.620766+0000","last_fullsized":"2024-04-24T09:19:58.620766+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:39:57.802717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.548220+0000","last_change":"2024-04-24T09:19:09.844721+0000","last_active":"2024-04-24T09:20:27.548220+0000","last_peered":"2024-04-24T09:20:27.548220+0000","last_clean":"2024-04-24T09:20:27.548220+0000","last_became_active":"2024-04-24T09:19:09.844365+0000","last_became_peered":"2024-04-24T09:19:09.844365+0000","last_unstale":"2024-04-24T09:20:27.548220+0000","last_undegraded":"2024-04-24T09:20:27.548220+0000","last_fullsized":"2024-04-24T09:20:27.548220+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:11:11.215015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'33","reported_seq":247,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:48.406099+0000","last_change":"2024-04-24T09:19:08.955409+0000","last_active":"2024-04-24T09:20:48.406099+0000","last_peered":"2024-04-24T09:20:48.406099+0000","last_clean":"2024-04-24T09:20:48.406099+0000","last_became_active":"2024-04-24T09:19:08.955065+0000","last_became_peered":"2024-04-24T09:19:08.955065+0000","last_unstale":"2024-04-24T09:20:48.406099+0000","last_undegraded":"2024-04-24T09:20:48.406099+0000","last_fullsized":"2024-04-24T09:20:48.406099+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:28:34.411386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61295,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":195,"num_read_kb":158,"num_write":35,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.546315+0000","last_change":"2024-04-24T09:19:09.846521+0000","last_active":"2024-04-24T09:20:27.546315+0000","last_peered":"2024-04-24T09:20:27.546315+0000","last_clean":"2024-04-24T09:20:27.546315+0000","last_became_active":"2024-04-24T09:19:09.846162+0000","last_became_peered":"2024-04-24T09:19:09.846162+0000","last_unstale":"2024-04-24T09:20:27.546315+0000","last_undegraded":"2024-04-24T09:20:27.546315+0000","last_fullsized":"2024-04-24T09:20:27.546315+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:05:50.383432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.616812+0000","last_change":"2024-04-24T09:19:08.963754+0000","last_active":"2024-04-24T09:19:58.616812+0000","last_peered":"2024-04-24T09:19:58.616812+0000","last_clean":"2024-04-24T09:19:58.616812+0000","last_became_active":"2024-04-24T09:19:08.963379+0000","last_became_peered":"2024-04-24T09:19:08.963379+0000","last_unstale":"2024-04-24T09:19:58.616812+0000","last_undegraded":"2024-04-24T09:19:58.616812+0000","last_fullsized":"2024-04-24T09:19:58.616812+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:25:01.538860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.548852+0000","last_change":"2024-04-24T09:19:09.863649+0000","last_active":"2024-04-24T09:20:27.548852+0000","last_peered":"2024-04-24T09:20:27.548852+0000","last_clean":"2024-04-24T09:20:27.548852+0000","last_became_active":"2024-04-24T09:19:09.863295+0000","last_became_peered":"2024-04-24T09:19:09.863295+0000","last_unstale":"2024-04-24T09:20:27.548852+0000","last_undegraded":"2024-04-24T09:20:27.548852+0000","last_fullsized":"2024-04-24T09:20:27.548852+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:06:56.404909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.620355+0000","last_change":"2024-04-24T09:19:08.961409+0000","last_active":"2024-04-24T09:19:58.620355+0000","last_peered":"2024-04-24T09:19:58.620355+0000","last_clean":"2024-04-24T09:19:58.620355+0000","last_became_active":"2024-04-24T09:19:08.961085+0000","last_became_peered":"2024-04-24T09:19:08.961085+0000","last_unstale":"2024-04-24T09:19:58.620355+0000","last_undegraded":"2024-04-24T09:19:58.620355+0000","last_fullsized":"2024-04-24T09:19:58.620355+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:10:32.816204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.561538+0000","last_change":"2024-04-24T09:19:09.845992+0000","last_active":"2024-04-24T09:20:27.561538+0000","last_peered":"2024-04-24T09:20:27.561538+0000","last_clean":"2024-04-24T09:20:27.561538+0000","last_became_active":"2024-04-24T09:19:09.845673+0000","last_became_peered":"2024-04-24T09:19:09.845673+0000","last_unstale":"2024-04-24T09:20:27.561538+0000","last_undegraded":"2024-04-24T09:20:27.561538+0000","last_fullsized":"2024-04-24T09:20:27.561538+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:42:53.136476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'7","reported_seq":119,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:44.423156+0000","last_change":"2024-04-24T09:19:08.954872+0000","last_active":"2024-04-24T09:20:44.423156+0000","last_peered":"2024-04-24T09:20:44.423156+0000","last_clean":"2024-04-24T09:20:44.423156+0000","last_became_active":"2024-04-24T09:19:08.954540+0000","last_became_peered":"2024-04-24T09:19:08.954540+0000","last_unstale":"2024-04-24T09:20:44.423156+0000","last_undegraded":"2024-04-24T09:20:44.423156+0000","last_fullsized":"2024-04-24T09:20:44.423156+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:32:15.637337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":94,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.547856+0000","last_change":"2024-04-24T09:19:09.841735+0000","last_active":"2024-04-24T09:20:27.547856+0000","last_peered":"2024-04-24T09:20:27.547856+0000","last_clean":"2024-04-24T09:20:27.547856+0000","last_became_active":"2024-04-24T09:19:09.841422+0000","last_became_peered":"2024-04-24T09:19:09.841422+0000","last_unstale":"2024-04-24T09:20:27.547856+0000","last_undegraded":"2024-04-24T09:20:27.547856+0000","last_fullsized":"2024-04-24T09:20:27.547856+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:37:46.128344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.616590+0000","last_change":"2024-04-24T09:19:08.971108+0000","last_active":"2024-04-24T09:19:58.616590+0000","last_peered":"2024-04-24T09:19:58.616590+0000","last_clean":"2024-04-24T09:19:58.616590+0000","last_became_active":"2024-04-24T09:19:08.970808+0000","last_became_peered":"2024-04-24T09:19:08.970808+0000","last_unstale":"2024-04-24T09:19:58.616590+0000","last_undegraded":"2024-04-24T09:19:58.616590+0000","last_fullsized":"2024-04-24T09:19:58.616590+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:37:26.311351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.533903+0000","last_change":"2024-04-24T09:19:09.848041+0000","last_active":"2024-04-24T09:20:27.533903+0000","last_peered":"2024-04-24T09:20:27.533903+0000","last_clean":"2024-04-24T09:20:27.533903+0000","last_became_active":"2024-04-24T09:19:09.847090+0000","last_became_peered":"2024-04-24T09:19:09.847090+0000","last_unstale":"2024-04-24T09:20:27.533903+0000","last_undegraded":"2024-04-24T09:20:27.533903+0000","last_fullsized":"2024-04-24T09:20:27.533903+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:45:13.652402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.617244+0000","last_change":"2024-04-24T09:19:08.952061+0000","last_active":"2024-04-24T09:19:58.617244+0000","last_peered":"2024-04-24T09:19:58.617244+0000","last_clean":"2024-04-24T09:19:58.617244+0000","last_became_active":"2024-04-24T09:19:08.950549+0000","last_became_peered":"2024-04-24T09:19:08.950549+0000","last_unstale":"2024-04-24T09:19:58.617244+0000","last_undegraded":"2024-04-24T09:19:58.617244+0000","last_fullsized":"2024-04-24T09:19:58.617244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:41:40.949650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.535934+0000","last_change":"2024-04-24T09:19:09.848619+0000","last_active":"2024-04-24T09:20:27.535934+0000","last_peered":"2024-04-24T09:20:27.535934+0000","last_clean":"2024-04-24T09:20:27.535934+0000","last_became_active":"2024-04-24T09:19:09.848246+0000","last_became_peered":"2024-04-24T09:19:09.848246+0000","last_unstale":"2024-04-24T09:20:27.535934+0000","last_undegraded":"2024-04-24T09:20:27.535934+0000","last_fullsized":"2024-04-24T09:20:27.535934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:30:22.925368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.614283+0000","last_change":"2024-04-24T09:19:08.977448+0000","last_active":"2024-04-24T09:19:58.614283+0000","last_peered":"2024-04-24T09:19:58.614283+0000","last_clean":"2024-04-24T09:19:58.614283+0000","last_became_active":"2024-04-24T09:19:08.976525+0000","last_became_peered":"2024-04-24T09:19:08.976525+0000","last_unstale":"2024-04-24T09:19:58.614283+0000","last_undegraded":"2024-04-24T09:19:58.614283+0000","last_fullsized":"2024-04-24T09:19:58.614283+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:06:59.151486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.552995+0000","last_change":"2024-04-24T09:19:09.856260+0000","last_active":"2024-04-24T09:20:27.552995+0000","last_peered":"2024-04-24T09:20:27.552995+0000","last_clean":"2024-04-24T09:20:27.552995+0000","last_became_active":"2024-04-24T09:19:09.855895+0000","last_became_peered":"2024-04-24T09:19:09.855895+0000","last_unstale":"2024-04-24T09:20:27.552995+0000","last_undegraded":"2024-04-24T09:20:27.552995+0000","last_fullsized":"2024-04-24T09:20:27.552995+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:44:22.804210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":117,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:47.426582+0000","last_change":"2024-04-24T09:19:08.962268+0000","last_active":"2024-04-24T09:20:47.426582+0000","last_peered":"2024-04-24T09:20:47.426582+0000","last_clean":"2024-04-24T09:20:47.426582+0000","last_became_active":"2024-04-24T09:19:08.961930+0000","last_became_peered":"2024-04-24T09:19:08.961930+0000","last_unstale":"2024-04-24T09:20:47.426582+0000","last_undegraded":"2024-04-24T09:20:47.426582+0000","last_fullsized":"2024-04-24T09:20:47.426582+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:51:28.357674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":93,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.546550+0000","last_change":"2024-04-24T09:19:09.850782+0000","last_active":"2024-04-24T09:20:27.546550+0000","last_peered":"2024-04-24T09:20:27.546550+0000","last_clean":"2024-04-24T09:20:27.546550+0000","last_became_active":"2024-04-24T09:19:09.850458+0000","last_became_peered":"2024-04-24T09:19:09.850458+0000","last_unstale":"2024-04-24T09:20:27.546550+0000","last_undegraded":"2024-04-24T09:20:27.546550+0000","last_fullsized":"2024-04-24T09:20:27.546550+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:02:19.464011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.550926+0000","last_change":"2024-04-24T09:19:58.619597+0000","last_active":"2024-04-24T09:20:27.550926+0000","last_peered":"2024-04-24T09:20:27.550926+0000","last_clean":"2024-04-24T09:20:27.550926+0000","last_became_active":"2024-04-24T09:19:58.619160+0000","last_became_peered":"2024-04-24T09:19:58.619160+0000","last_unstale":"2024-04-24T09:20:27.550926+0000","last_undegraded":"2024-04-24T09:20:27.550926+0000","last_fullsized":"2024-04-24T09:20:27.550926+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:34:05.633062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"88'6","reported_seq":117,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:45.424035+0000","last_change":"2024-04-24T09:19:08.972070+0000","last_active":"2024-04-24T09:20:45.424035+0000","last_peered":"2024-04-24T09:20:45.424035+0000","last_clean":"2024-04-24T09:20:45.424035+0000","last_became_active":"2024-04-24T09:19:08.971699+0000","last_became_peered":"2024-04-24T09:19:08.971699+0000","last_unstale":"2024-04-24T09:20:45.424035+0000","last_undegraded":"2024-04-24T09:20:45.424035+0000","last_fullsized":"2024-04-24T09:20:45.424035+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:38:13.312940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":92,"num_read_kb":92,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.559887+0000","last_change":"2024-04-24T09:19:09.855902+0000","last_active":"2024-04-24T09:20:27.559887+0000","last_peered":"2024-04-24T09:20:27.559887+0000","last_clean":"2024-04-24T09:20:27.559887+0000","last_became_active":"2024-04-24T09:19:09.855503+0000","last_became_peered":"2024-04-24T09:19:09.855503+0000","last_unstale":"2024-04-24T09:20:27.559887+0000","last_undegraded":"2024-04-24T09:20:27.559887+0000","last_fullsized":"2024-04-24T09:20:27.559887+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:29:41.621269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.614543+0000","last_change":"2024-04-24T09:19:08.973774+0000","last_active":"2024-04-24T09:19:58.614543+0000","last_peered":"2024-04-24T09:19:58.614543+0000","last_clean":"2024-04-24T09:19:58.614543+0000","last_became_active":"2024-04-24T09:19:08.973372+0000","last_became_peered":"2024-04-24T09:19:08.973372+0000","last_unstale":"2024-04-24T09:19:58.614543+0000","last_undegraded":"2024-04-24T09:19:58.614543+0000","last_fullsized":"2024-04-24T09:19:58.614543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:25:12.910023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":343,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.618651+0000","last_change":"2024-04-24T09:17:48.282773+0000","last_active":"2024-04-24T09:19:58.618651+0000","last_peered":"2024-04-24T09:19:58.618651+0000","last_clean":"2024-04-24T09:19:58.618651+0000","last_became_active":"2024-04-24T09:17:48.282128+0000","last_became_peered":"2024-04-24T09:17:48.282128+0000","last_unstale":"2024-04-24T09:19:58.618651+0000","last_undegraded":"2024-04-24T09:19:58.618651+0000","last_fullsized":"2024-04-24T09:19:58.618651+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:13:45.562295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:13:45.562295+0000","last_clean_scrub_stamp":"2024-04-24T09:13:45.562295+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:54:34.557051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.538367+0000","last_change":"2024-04-24T09:19:09.847771+0000","last_active":"2024-04-24T09:20:27.538367+0000","last_peered":"2024-04-24T09:20:27.538367+0000","last_clean":"2024-04-24T09:20:27.538367+0000","last_became_active":"2024-04-24T09:19:09.845873+0000","last_became_peered":"2024-04-24T09:19:09.845873+0000","last_unstale":"2024-04-24T09:20:27.538367+0000","last_undegraded":"2024-04-24T09:20:27.538367+0000","last_fullsized":"2024-04-24T09:20:27.538367+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:31:46.124082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.624968+0000","last_change":"2024-04-24T09:19:58.624968+0000","last_active":"2024-04-24T09:19:58.624968+0000","last_peered":"2024-04-24T09:19:58.624968+0000","last_clean":"2024-04-24T09:19:58.624968+0000","last_became_active":"2024-04-24T09:19:58.621737+0000","last_became_peered":"2024-04-24T09:19:58.621737+0000","last_unstale":"2024-04-24T09:19:58.624968+0000","last_undegraded":"2024-04-24T09:19:58.624968+0000","last_fullsized":"2024-04-24T09:19:58.624968+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:55:20.244149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.617560+0000","last_change":"2024-04-24T09:19:08.971475+0000","last_active":"2024-04-24T09:19:58.617560+0000","last_peered":"2024-04-24T09:19:58.617560+0000","last_clean":"2024-04-24T09:19:58.617560+0000","last_became_active":"2024-04-24T09:19:08.971188+0000","last_became_peered":"2024-04-24T09:19:08.971188+0000","last_unstale":"2024-04-24T09:19:58.617560+0000","last_undegraded":"2024-04-24T09:19:58.617560+0000","last_fullsized":"2024-04-24T09:19:58.617560+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:50:06.630182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.556407+0000","last_change":"2024-04-24T09:19:09.857102+0000","last_active":"2024-04-24T09:20:27.556407+0000","last_peered":"2024-04-24T09:20:27.556407+0000","last_clean":"2024-04-24T09:20:27.556407+0000","last_became_active":"2024-04-24T09:19:09.856632+0000","last_became_peered":"2024-04-24T09:19:09.856632+0000","last_unstale":"2024-04-24T09:20:27.556407+0000","last_undegraded":"2024-04-24T09:20:27.556407+0000","last_fullsized":"2024-04-24T09:20:27.556407+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:04:03.300741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'13","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.636195+0000","last_change":"2024-04-24T09:19:58.635644+0000","last_active":"2024-04-24T09:19:58.636195+0000","last_peered":"2024-04-24T09:19:58.636195+0000","last_clean":"2024-04-24T09:19:58.636195+0000","last_became_active":"2024-04-24T09:19:58.625594+0000","last_became_peered":"2024-04-24T09:19:58.625594+0000","last_unstale":"2024-04-24T09:19:58.636195+0000","last_undegraded":"2024-04-24T09:19:58.636195+0000","last_fullsized":"2024-04-24T09:19:58.636195+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:29:38.682899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.561122+0000","last_change":"2024-04-24T09:19:09.863386+0000","last_active":"2024-04-24T09:20:27.561122+0000","last_peered":"2024-04-24T09:20:27.561122+0000","last_clean":"2024-04-24T09:20:27.561122+0000","last_became_active":"2024-04-24T09:19:09.856300+0000","last_became_peered":"2024-04-24T09:19:09.856300+0000","last_unstale":"2024-04-24T09:20:27.561122+0000","last_undegraded":"2024-04-24T09:20:27.561122+0000","last_fullsized":"2024-04-24T09:20:27.561122+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:26:37.603307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.614840+0000","last_change":"2024-04-24T09:19:08.961751+0000","last_active":"2024-04-24T09:19:58.614840+0000","last_peered":"2024-04-24T09:19:58.614840+0000","last_clean":"2024-04-24T09:19:58.614840+0000","last_became_active":"2024-04-24T09:19:08.961389+0000","last_became_peered":"2024-04-24T09:19:08.961389+0000","last_unstale":"2024-04-24T09:19:58.614840+0000","last_undegraded":"2024-04-24T09:19:58.614840+0000","last_fullsized":"2024-04-24T09:19:58.614840+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:25:17.913335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.561229+0000","last_change":"2024-04-24T09:19:09.862140+0000","last_active":"2024-04-24T09:20:27.561229+0000","last_peered":"2024-04-24T09:20:27.561229+0000","last_clean":"2024-04-24T09:20:27.561229+0000","last_became_active":"2024-04-24T09:19:09.860776+0000","last_became_peered":"2024-04-24T09:19:09.860776+0000","last_unstale":"2024-04-24T09:20:27.561229+0000","last_undegraded":"2024-04-24T09:20:27.561229+0000","last_fullsized":"2024-04-24T09:20:27.561229+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:08:29.437422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.996452+0000","last_change":"2024-04-24T09:19:08.973503+0000","last_active":"2024-04-24T09:19:58.996452+0000","last_peered":"2024-04-24T09:19:58.996452+0000","last_clean":"2024-04-24T09:19:58.996452+0000","last_became_active":"2024-04-24T09:19:08.973153+0000","last_became_peered":"2024-04-24T09:19:08.973153+0000","last_unstale":"2024-04-24T09:19:58.996452+0000","last_undegraded":"2024-04-24T09:19:58.996452+0000","last_fullsized":"2024-04-24T09:19:58.996452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:57:21.489696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.507289+0000","last_change":"2024-04-24T09:19:09.838175+0000","last_active":"2024-04-24T09:20:27.507289+0000","last_peered":"2024-04-24T09:20:27.507289+0000","last_clean":"2024-04-24T09:20:27.507289+0000","last_became_active":"2024-04-24T09:19:09.837781+0000","last_became_peered":"2024-04-24T09:19:09.837781+0000","last_unstale":"2024-04-24T09:20:27.507289+0000","last_undegraded":"2024-04-24T09:20:27.507289+0000","last_fullsized":"2024-04-24T09:20:27.507289+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:48:03.967587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.622324+0000","last_change":"2024-04-24T09:19:08.976790+0000","last_active":"2024-04-24T09:19:58.622324+0000","last_peered":"2024-04-24T09:19:58.622324+0000","last_clean":"2024-04-24T09:19:58.622324+0000","last_became_active":"2024-04-24T09:19:08.975196+0000","last_became_peered":"2024-04-24T09:19:08.975196+0000","last_unstale":"2024-04-24T09:19:58.622324+0000","last_undegraded":"2024-04-24T09:19:58.622324+0000","last_fullsized":"2024-04-24T09:19:58.622324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:54:59.415169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.541194+0000","last_change":"2024-04-24T09:19:09.850958+0000","last_active":"2024-04-24T09:20:27.541194+0000","last_peered":"2024-04-24T09:20:27.541194+0000","last_clean":"2024-04-24T09:20:27.541194+0000","last_became_active":"2024-04-24T09:19:09.850660+0000","last_became_peered":"2024-04-24T09:19:09.850660+0000","last_unstale":"2024-04-24T09:20:27.541194+0000","last_undegraded":"2024-04-24T09:20:27.541194+0000","last_fullsized":"2024-04-24T09:20:27.541194+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:41:27.496169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.630129+0000","last_change":"2024-04-24T09:19:58.629650+0000","last_active":"2024-04-24T09:19:58.630129+0000","last_peered":"2024-04-24T09:19:58.630129+0000","last_clean":"2024-04-24T09:19:58.630129+0000","last_became_active":"2024-04-24T09:19:58.624974+0000","last_became_peered":"2024-04-24T09:19:58.624974+0000","last_unstale":"2024-04-24T09:19:58.630129+0000","last_undegraded":"2024-04-24T09:19:58.630129+0000","last_fullsized":"2024-04-24T09:19:58.630129+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:12:42.319723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.561625+0000","last_change":"2024-04-24T09:19:09.863144+0000","last_active":"2024-04-24T09:20:27.561625+0000","last_peered":"2024-04-24T09:20:27.561625+0000","last_clean":"2024-04-24T09:20:27.561625+0000","last_became_active":"2024-04-24T09:19:09.862772+0000","last_became_peered":"2024-04-24T09:19:09.862772+0000","last_unstale":"2024-04-24T09:20:27.561625+0000","last_undegraded":"2024-04-24T09:20:27.561625+0000","last_fullsized":"2024-04-24T09:20:27.561625+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:05:42.926470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.616697+0000","last_change":"2024-04-24T09:19:08.953467+0000","last_active":"2024-04-24T09:19:58.616697+0000","last_peered":"2024-04-24T09:19:58.616697+0000","last_clean":"2024-04-24T09:19:58.616697+0000","last_became_active":"2024-04-24T09:19:08.953116+0000","last_became_peered":"2024-04-24T09:19:08.953116+0000","last_unstale":"2024-04-24T09:19:58.616697+0000","last_undegraded":"2024-04-24T09:19:58.616697+0000","last_fullsized":"2024-04-24T09:19:58.616697+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:07:35.834466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.562016+0000","last_change":"2024-04-24T09:19:09.852329+0000","last_active":"2024-04-24T09:20:27.562016+0000","last_peered":"2024-04-24T09:20:27.562016+0000","last_clean":"2024-04-24T09:20:27.562016+0000","last_became_active":"2024-04-24T09:19:09.851762+0000","last_became_peered":"2024-04-24T09:19:09.851762+0000","last_unstale":"2024-04-24T09:20:27.562016+0000","last_undegraded":"2024-04-24T09:20:27.562016+0000","last_fullsized":"2024-04-24T09:20:27.562016+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:56:59.234680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'5","reported_seq":66,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:43.700228+0000","last_change":"2024-04-24T09:19:58.626186+0000","last_active":"2024-04-24T09:20:43.700228+0000","last_peered":"2024-04-24T09:20:43.700228+0000","last_clean":"2024-04-24T09:20:43.700228+0000","last_became_active":"2024-04-24T09:19:58.619730+0000","last_became_peered":"2024-04-24T09:19:58.619730+0000","last_unstale":"2024-04-24T09:20:43.700228+0000","last_undegraded":"2024-04-24T09:20:43.700228+0000","last_fullsized":"2024-04-24T09:20:43.700228+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:18:56.284816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":47,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.550088+0000","last_change":"2024-04-24T09:19:09.840340+0000","last_active":"2024-04-24T09:20:27.550088+0000","last_peered":"2024-04-24T09:20:27.550088+0000","last_clean":"2024-04-24T09:20:27.550088+0000","last_became_active":"2024-04-24T09:19:09.839689+0000","last_became_peered":"2024-04-24T09:19:09.839689+0000","last_unstale":"2024-04-24T09:20:27.550088+0000","last_undegraded":"2024-04-24T09:20:27.550088+0000","last_fullsized":"2024-04-24T09:20:27.550088+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:36:06.174519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.620640+0000","last_change":"2024-04-24T09:19:08.959025+0000","last_active":"2024-04-24T09:19:58.620640+0000","last_peered":"2024-04-24T09:19:58.620640+0000","last_clean":"2024-04-24T09:19:58.620640+0000","last_became_active":"2024-04-24T09:19:08.958749+0000","last_became_peered":"2024-04-24T09:19:08.958749+0000","last_unstale":"2024-04-24T09:19:58.620640+0000","last_undegraded":"2024-04-24T09:19:58.620640+0000","last_fullsized":"2024-04-24T09:19:58.620640+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:59:41.334448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.552451+0000","last_change":"2024-04-24T09:19:58.629338+0000","last_active":"2024-04-24T09:20:27.552451+0000","last_peered":"2024-04-24T09:20:27.552451+0000","last_clean":"2024-04-24T09:20:27.552451+0000","last_became_active":"2024-04-24T09:19:58.625976+0000","last_became_peered":"2024-04-24T09:19:58.625976+0000","last_unstale":"2024-04-24T09:20:27.552451+0000","last_undegraded":"2024-04-24T09:20:27.552451+0000","last_fullsized":"2024-04-24T09:20:27.552451+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:17:37.470806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.953266+0000","last_change":"2024-04-24T09:19:08.965880+0000","last_active":"2024-04-24T09:19:58.953266+0000","last_peered":"2024-04-24T09:19:58.953266+0000","last_clean":"2024-04-24T09:19:58.953266+0000","last_became_active":"2024-04-24T09:19:08.965378+0000","last_became_peered":"2024-04-24T09:19:08.965378+0000","last_unstale":"2024-04-24T09:19:58.953266+0000","last_undegraded":"2024-04-24T09:19:58.953266+0000","last_fullsized":"2024-04-24T09:19:58.953266+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:08:22.628556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.547817+0000","last_change":"2024-04-24T09:19:09.845151+0000","last_active":"2024-04-24T09:20:27.547817+0000","last_peered":"2024-04-24T09:20:27.547817+0000","last_clean":"2024-04-24T09:20:27.547817+0000","last_became_active":"2024-04-24T09:19:09.844791+0000","last_became_peered":"2024-04-24T09:19:09.844791+0000","last_unstale":"2024-04-24T09:20:27.547817+0000","last_undegraded":"2024-04-24T09:20:27.547817+0000","last_fullsized":"2024-04-24T09:20:27.547817+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:38:00.782591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.538975+0000","last_change":"2024-04-24T09:19:09.844449+0000","last_active":"2024-04-24T09:20:27.538975+0000","last_peered":"2024-04-24T09:20:27.538975+0000","last_clean":"2024-04-24T09:20:27.538975+0000","last_became_active":"2024-04-24T09:19:09.844122+0000","last_became_peered":"2024-04-24T09:19:09.844122+0000","last_unstale":"2024-04-24T09:20:27.538975+0000","last_undegraded":"2024-04-24T09:20:27.538975+0000","last_fullsized":"2024-04-24T09:20:27.538975+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:36:01.857309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.545661+0000","last_change":"2024-04-24T09:19:09.843151+0000","last_active":"2024-04-24T09:20:27.545661+0000","last_peered":"2024-04-24T09:20:27.545661+0000","last_clean":"2024-04-24T09:20:27.545661+0000","last_became_active":"2024-04-24T09:19:09.842519+0000","last_became_peered":"2024-04-24T09:19:09.842519+0000","last_unstale":"2024-04-24T09:20:27.545661+0000","last_undegraded":"2024-04-24T09:20:27.545661+0000","last_fullsized":"2024-04-24T09:20:27.545661+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:26:00.791618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.558108+0000","last_change":"2024-04-24T09:19:09.856291+0000","last_active":"2024-04-24T09:20:27.558108+0000","last_peered":"2024-04-24T09:20:27.558108+0000","last_clean":"2024-04-24T09:20:27.558108+0000","last_became_active":"2024-04-24T09:19:09.855932+0000","last_became_peered":"2024-04-24T09:19:09.855932+0000","last_unstale":"2024-04-24T09:20:27.558108+0000","last_undegraded":"2024-04-24T09:20:27.558108+0000","last_fullsized":"2024-04-24T09:20:27.558108+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:11:14.113321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.562476+0000","last_change":"2024-04-24T09:19:58.955125+0000","last_active":"2024-04-24T09:20:27.562476+0000","last_peered":"2024-04-24T09:20:27.562476+0000","last_clean":"2024-04-24T09:20:27.562476+0000","last_became_active":"2024-04-24T09:19:58.954545+0000","last_became_peered":"2024-04-24T09:19:58.954545+0000","last_unstale":"2024-04-24T09:20:27.562476+0000","last_undegraded":"2024-04-24T09:20:27.562476+0000","last_fullsized":"2024-04-24T09:20:27.562476+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:18:14.132248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.560486+0000","last_change":"2024-04-24T09:19:09.852140+0000","last_active":"2024-04-24T09:20:27.560486+0000","last_peered":"2024-04-24T09:20:27.560486+0000","last_clean":"2024-04-24T09:20:27.560486+0000","last_became_active":"2024-04-24T09:19:09.851766+0000","last_became_peered":"2024-04-24T09:19:09.851766+0000","last_unstale":"2024-04-24T09:20:27.560486+0000","last_undegraded":"2024-04-24T09:20:27.560486+0000","last_fullsized":"2024-04-24T09:20:27.560486+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:14:51.794470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.533826+0000","last_change":"2024-04-24T09:19:09.839389+0000","last_active":"2024-04-24T09:20:27.533826+0000","last_peered":"2024-04-24T09:20:27.533826+0000","last_clean":"2024-04-24T09:20:27.533826+0000","last_became_active":"2024-04-24T09:19:09.838989+0000","last_became_peered":"2024-04-24T09:19:09.838989+0000","last_unstale":"2024-04-24T09:20:27.533826+0000","last_undegraded":"2024-04-24T09:20:27.533826+0000","last_fullsized":"2024-04-24T09:20:27.533826+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:00:28.081837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.546358+0000","last_change":"2024-04-24T09:19:09.845304+0000","last_active":"2024-04-24T09:20:27.546358+0000","last_peered":"2024-04-24T09:20:27.546358+0000","last_clean":"2024-04-24T09:20:27.546358+0000","last_became_active":"2024-04-24T09:19:09.844979+0000","last_became_peered":"2024-04-24T09:19:09.844979+0000","last_unstale":"2024-04-24T09:20:27.546358+0000","last_undegraded":"2024-04-24T09:20:27.546358+0000","last_fullsized":"2024-04-24T09:20:27.546358+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:36:58.294217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.555155+0000","last_change":"2024-04-24T09:19:09.865120+0000","last_active":"2024-04-24T09:20:27.555155+0000","last_peered":"2024-04-24T09:20:27.555155+0000","last_clean":"2024-04-24T09:20:27.555155+0000","last_became_active":"2024-04-24T09:19:09.856837+0000","last_became_peered":"2024-04-24T09:19:09.856837+0000","last_unstale":"2024-04-24T09:20:27.555155+0000","last_undegraded":"2024-04-24T09:20:27.555155+0000","last_fullsized":"2024-04-24T09:20:27.555155+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:30:51.888927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.550991+0000","last_change":"2024-04-24T09:19:09.853625+0000","last_active":"2024-04-24T09:20:27.550991+0000","last_peered":"2024-04-24T09:20:27.550991+0000","last_clean":"2024-04-24T09:20:27.550991+0000","last_became_active":"2024-04-24T09:19:09.853390+0000","last_became_peered":"2024-04-24T09:19:09.853390+0000","last_unstale":"2024-04-24T09:20:27.550991+0000","last_undegraded":"2024-04-24T09:20:27.550991+0000","last_fullsized":"2024-04-24T09:20:27.550991+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:14:48.571283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.555050+0000","last_change":"2024-04-24T09:19:09.851380+0000","last_active":"2024-04-24T09:20:27.555050+0000","last_peered":"2024-04-24T09:20:27.555050+0000","last_clean":"2024-04-24T09:20:27.555050+0000","last_became_active":"2024-04-24T09:19:09.843723+0000","last_became_peered":"2024-04-24T09:19:09.843723+0000","last_unstale":"2024-04-24T09:20:27.555050+0000","last_undegraded":"2024-04-24T09:20:27.555050+0000","last_fullsized":"2024-04-24T09:20:27.555050+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:55:50.103891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.561198+0000","last_change":"2024-04-24T09:19:09.856620+0000","last_active":"2024-04-24T09:20:27.561198+0000","last_peered":"2024-04-24T09:20:27.561198+0000","last_clean":"2024-04-24T09:20:27.561198+0000","last_became_active":"2024-04-24T09:19:09.856101+0000","last_became_peered":"2024-04-24T09:19:09.856101+0000","last_unstale":"2024-04-24T09:20:27.561198+0000","last_undegraded":"2024-04-24T09:20:27.561198+0000","last_fullsized":"2024-04-24T09:20:27.561198+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:32:26.035336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.562463+0000","last_change":"2024-04-24T09:19:09.853959+0000","last_active":"2024-04-24T09:20:27.562463+0000","last_peered":"2024-04-24T09:20:27.562463+0000","last_clean":"2024-04-24T09:20:27.562463+0000","last_became_active":"2024-04-24T09:19:09.853588+0000","last_became_peered":"2024-04-24T09:19:09.853588+0000","last_unstale":"2024-04-24T09:20:27.562463+0000","last_undegraded":"2024-04-24T09:20:27.562463+0000","last_fullsized":"2024-04-24T09:20:27.562463+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:43:09.128758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.536846+0000","last_change":"2024-04-24T09:19:09.845317+0000","last_active":"2024-04-24T09:20:27.536846+0000","last_peered":"2024-04-24T09:20:27.536846+0000","last_clean":"2024-04-24T09:20:27.536846+0000","last_became_active":"2024-04-24T09:19:09.844975+0000","last_became_peered":"2024-04-24T09:19:09.844975+0000","last_unstale":"2024-04-24T09:20:27.536846+0000","last_undegraded":"2024-04-24T09:20:27.536846+0000","last_fullsized":"2024-04-24T09:20:27.536846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:36:28.053045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":246913,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68249,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":910,"num_read_kb":554,"num_write":243,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1170,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":204747,"data_compressed":5610,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":211,"ondisk_log_size":211,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351011,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":459945,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612647,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1110016,"data_stored":1112801,"data_compressed":43410,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776174,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":459992,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037813,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":978944,"data_stored":460253,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299451,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":1052912,"data_compressed":41540,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462976,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":459592,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1024000,"data_stored":521800,"data_compressed":7667,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":460931,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411415,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":520850,"data_compressed":7667,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149777,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":459300,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888139,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":461155,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724614,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":1052058,"data_compressed":41540,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34099,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34099,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35743,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35743,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35743,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35743,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35743,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":63047,"data_compressed":1870,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":62097,"data_compressed":1870,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":63563,"data_compressed":1870,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T09:20:51.356 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-24T09:20:51.837 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:20:51.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:51 smithi114 ceph-mon[32501]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T09:20:51.986 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:51 smithi090 ceph-mon[27825]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T09:20:52.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:51 smithi149 ceph-mon[32592]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T09:20:52.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:52 smithi114 ceph-mon[32501]: pgmap v375: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T09:20:53.012 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:52 smithi090 ceph-mon[27825]: pgmap v375: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T09:20:53.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:52 smithi149 ceph-mon[32592]: pgmap v375: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T09:20:54.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.014+0000 7f81af000700 1 -- 172.21.15.90:0/26285352 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81a8075640 msgr2=0x7f81a8113780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:54.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.014+0000 7f81af000700 1 --2- 172.21.15.90:0/26285352 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81a8075640 0x7f81a8113780 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8194009990 tx=0x7f8194009ca0 comp rx=0 tx=0).stop 2024-04-24T09:20:54.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.015+0000 7f81af000700 1 -- 172.21.15.90:0/26285352 shutdown_connections 2024-04-24T09:20:54.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.015+0000 7f81af000700 1 --2- 172.21.15.90:0/26285352 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f81a8113cc0 0x7f81a81160b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:54.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.015+0000 7f81af000700 1 --2- 172.21.15.90:0/26285352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81a8074d20 0x7f81a8075100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:54.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.015+0000 7f81af000700 1 --2- 172.21.15.90:0/26285352 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81a8075640 0x7f81a8113780 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:54.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.015+0000 7f81af000700 1 -- 172.21.15.90:0/26285352 >> 172.21.15.90:0/26285352 conn(0x7f81a8079ca0 msgr2=0x7f81a807a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:20:54.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.015+0000 7f81af000700 1 -- 172.21.15.90:0/26285352 shutdown_connections 2024-04-24T09:20:54.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.015+0000 7f81af000700 1 -- 172.21.15.90:0/26285352 wait complete. 2024-04-24T09:20:54.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.016+0000 7f81af000700 1 Processor -- start 2024-04-24T09:20:54.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.016+0000 7f81af000700 1 -- start start 2024-04-24T09:20:54.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.016+0000 7f81af000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81a8074d20 0x7f81a81a5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:54.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.016+0000 7f81af000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81a8075640 0x7f81a81a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:54.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.016+0000 7f81af000700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f81a8113cc0 0x7f81a819f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:54.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.016+0000 7f81af000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f81a8119450 con 0x7f81a8075640 2024-04-24T09:20:54.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.016+0000 7f81af000700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f81a81192d0 con 0x7f81a8113cc0 2024-04-24T09:20:54.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.016+0000 7f81af000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f81a81195d0 con 0x7f81a8074d20 2024-04-24T09:20:54.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.017+0000 7f81a7400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81a8075640 0x7f81a81a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:54.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.017+0000 7f81ad000700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f81a8113cc0 0x7f81a819f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:54.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.017+0000 7f81a7e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81a8074d20 0x7f81a81a5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:54.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.017+0000 7f81ad000700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f81a8113cc0 0x7f81a819f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.90:56914/0 (socket says 172.21.15.90:56914) 2024-04-24T09:20:54.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.017+0000 7f81a7e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81a8074d20 0x7f81a81a5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:56072/0 (socket says 172.21.15.90:56072) 2024-04-24T09:20:54.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.017+0000 7f81ad000700 1 -- 172.21.15.90:0/371177557 learned_addr learned my addr 172.21.15.90:0/371177557 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:20:54.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.017+0000 7f81ad000700 1 -- 172.21.15.90:0/371177557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81a8074d20 msgr2=0x7f81a81a5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:54.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.017+0000 7f81ad000700 1 --2- 172.21.15.90:0/371177557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81a8074d20 0x7f81a81a5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:54.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.017+0000 7f81ad000700 1 -- 172.21.15.90:0/371177557 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81a8075640 msgr2=0x7f81a81a5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:54.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.017+0000 7f81ad000700 1 --2- 172.21.15.90:0/371177557 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81a8075640 0x7f81a81a5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:54.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.017+0000 7f81ad000700 1 -- 172.21.15.90:0/371177557 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81a819ffd0 con 0x7f81a8113cc0 2024-04-24T09:20:54.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.017+0000 7f81a7e00700 1 --2- 172.21.15.90:0/371177557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81a8074d20 0x7f81a81a5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:20:54.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.017+0000 7f81ad000700 1 --2- 172.21.15.90:0/371177557 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f81a8113cc0 0x7f81a819f6f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f81a000efc0 tx=0x7f81a000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:20:54.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.018+0000 7f81a4c00700 1 -- 172.21.15.90:0/371177557 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a00092e0 con 0x7f81a8113cc0 2024-04-24T09:20:54.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.018+0000 7f81a4c00700 1 -- 172.21.15.90:0/371177557 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81a0004bb0 con 0x7f81a8113cc0 2024-04-24T09:20:54.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.018+0000 7f81af000700 1 -- 172.21.15.90:0/371177557 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81a81a0260 con 0x7f81a8113cc0 2024-04-24T09:20:54.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.018+0000 7f81a4c00700 1 -- 172.21.15.90:0/371177557 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a0004020 con 0x7f81a8113cc0 2024-04-24T09:20:54.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.018+0000 7f81af000700 1 -- 172.21.15.90:0/371177557 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81a81ac5a0 con 0x7f81a8113cc0 2024-04-24T09:20:54.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.019+0000 7f81af000700 1 -- 172.21.15.90:0/371177557 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81a81a05d0 con 0x7f81a8113cc0 2024-04-24T09:20:54.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.021+0000 7f81a4c00700 1 -- 172.21.15.90:0/371177557 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f81a001b030 con 0x7f81a8113cc0 2024-04-24T09:20:54.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.022+0000 7f81a4c00700 1 --2- 172.21.15.90:0/371177557 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f818c0a3c40 0x7f818c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:54.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.022+0000 7f81a4c00700 1 -- 172.21.15.90:0/371177557 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f81a00cfcb0 con 0x7f81a8113cc0 2024-04-24T09:20:54.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.022+0000 7f81a7e00700 1 --2- 172.21.15.90:0/371177557 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f818c0a3c40 0x7f818c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:54.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.023+0000 7f81a7e00700 1 --2- 172.21.15.90:0/371177557 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f818c0a3c40 0x7f818c0a6100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8198011490 tx=0x7f8198009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:20:54.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.027+0000 7f81a4c00700 1 -- 172.21.15.90:0/371177557 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f81a81a05d0 con 0x7f81a8113cc0 2024-04-24T09:20:54.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.249+0000 7f81af000700 1 -- 172.21.15.90:0/371177557 --> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f81a8113120 con 0x7f818c0a3c40 2024-04-24T09:20:54.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.265+0000 7f81a4c00700 1 -- 172.21.15.90:0/371177557 <== mgr.14152 v2:172.21.15.90:6800/4261662956 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259163 (secure 0 0 0) 0x7f81a8113120 con 0x7f818c0a3c40 2024-04-24T09:20:54.271 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:20:54.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.272+0000 7f81af000700 1 -- 172.21.15.90:0/371177557 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f818c0a3c40 msgr2=0x7f818c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:54.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.272+0000 7f81af000700 1 --2- 172.21.15.90:0/371177557 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f818c0a3c40 0x7f818c0a6100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8198011490 tx=0x7f8198009260 comp rx=0 tx=0).stop 2024-04-24T09:20:54.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.272+0000 7f81af000700 1 -- 172.21.15.90:0/371177557 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f81a8113cc0 msgr2=0x7f81a819f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:54.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.272+0000 7f81af000700 1 --2- 172.21.15.90:0/371177557 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f81a8113cc0 0x7f81a819f6f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f81a000efc0 tx=0x7f81a000c3a0 comp rx=0 tx=0).stop 2024-04-24T09:20:54.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.272+0000 7f81af000700 1 -- 172.21.15.90:0/371177557 shutdown_connections 2024-04-24T09:20:54.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.273+0000 7f81af000700 1 --2- 172.21.15.90:0/371177557 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f81a8113cc0 0x7f81a819f6f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:54.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.273+0000 7f81af000700 1 --2- 172.21.15.90:0/371177557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81a8074d20 0x7f81a81a5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:54.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.273+0000 7f81af000700 1 --2- 172.21.15.90:0/371177557 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f818c0a3c40 0x7f818c0a6100 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:54.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.273+0000 7f81af000700 1 --2- 172.21.15.90:0/371177557 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81a8075640 0x7f81a81a5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:54.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.273+0000 7f81af000700 1 -- 172.21.15.90:0/371177557 >> 172.21.15.90:0/371177557 conn(0x7f81a8079ca0 msgr2=0x7f81a8112a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:20:54.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.273+0000 7f81af000700 1 -- 172.21.15.90:0/371177557 shutdown_connections 2024-04-24T09:20:54.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:54.273+0000 7f81af000700 1 -- 172.21.15.90:0/371177557 wait complete. 2024-04-24T09:20:54.274 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-24T09:20:54.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:54 smithi114 ceph-mon[32501]: pgmap v376: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:20:55.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:54 smithi090 ceph-mon[27825]: pgmap v376: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:20:55.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:54 smithi149 ceph-mon[32592]: pgmap v376: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:20:55.207 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":376,"stamp":"2024-04-24T09:20:53.563323+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658851,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1061,"num_read_kb":673,"num_write":426,"num_write_kb":3077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1182023,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":850,"ondisk_log_size":850,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369268,"kb_used_data":11796,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572556,"statfs":{"total":1151940427776,"available":1151562297344,"internally_reserved":0,"allocated":12079104,"data_stored":7481589,"data_compressed":182403,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":60,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005502"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.549323+0000","last_change":"2024-04-24T09:19:09.860043+0000","last_active":"2024-04-24T09:20:27.549323+0000","last_peered":"2024-04-24T09:20:27.549323+0000","last_clean":"2024-04-24T09:20:27.549323+0000","last_became_active":"2024-04-24T09:19:09.859766+0000","last_became_peered":"2024-04-24T09:19:09.859766+0000","last_unstale":"2024-04-24T09:20:27.549323+0000","last_undegraded":"2024-04-24T09:20:27.549323+0000","last_fullsized":"2024-04-24T09:20:27.549323+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:40:46.241239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.551329+0000","last_change":"2024-04-24T09:19:09.847293+0000","last_active":"2024-04-24T09:20:27.551329+0000","last_peered":"2024-04-24T09:20:27.551329+0000","last_clean":"2024-04-24T09:20:27.551329+0000","last_became_active":"2024-04-24T09:19:09.846955+0000","last_became_peered":"2024-04-24T09:19:09.846955+0000","last_unstale":"2024-04-24T09:20:27.551329+0000","last_undegraded":"2024-04-24T09:20:27.551329+0000","last_fullsized":"2024-04-24T09:20:27.551329+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:38:10.062654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.555852+0000","last_change":"2024-04-24T09:19:09.858954+0000","last_active":"2024-04-24T09:20:27.555852+0000","last_peered":"2024-04-24T09:20:27.555852+0000","last_clean":"2024-04-24T09:20:27.555852+0000","last_became_active":"2024-04-24T09:19:09.855915+0000","last_became_peered":"2024-04-24T09:19:09.855915+0000","last_unstale":"2024-04-24T09:20:27.555852+0000","last_undegraded":"2024-04-24T09:20:27.555852+0000","last_fullsized":"2024-04-24T09:20:27.555852+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:12:44.214700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.537045+0000","last_change":"2024-04-24T09:19:09.848582+0000","last_active":"2024-04-24T09:20:27.537045+0000","last_peered":"2024-04-24T09:20:27.537045+0000","last_clean":"2024-04-24T09:20:27.537045+0000","last_became_active":"2024-04-24T09:19:09.848163+0000","last_became_peered":"2024-04-24T09:19:09.848163+0000","last_unstale":"2024-04-24T09:20:27.537045+0000","last_undegraded":"2024-04-24T09:20:27.537045+0000","last_fullsized":"2024-04-24T09:20:27.537045+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:33:46.106081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.544380+0000","last_change":"2024-04-24T09:19:09.864608+0000","last_active":"2024-04-24T09:20:27.544380+0000","last_peered":"2024-04-24T09:20:27.544380+0000","last_clean":"2024-04-24T09:20:27.544380+0000","last_became_active":"2024-04-24T09:19:09.864271+0000","last_became_peered":"2024-04-24T09:19:09.864271+0000","last_unstale":"2024-04-24T09:20:27.544380+0000","last_undegraded":"2024-04-24T09:20:27.544380+0000","last_fullsized":"2024-04-24T09:20:27.544380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:08:57.535882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.547057+0000","last_change":"2024-04-24T09:19:09.838589+0000","last_active":"2024-04-24T09:20:27.547057+0000","last_peered":"2024-04-24T09:20:27.547057+0000","last_clean":"2024-04-24T09:20:27.547057+0000","last_became_active":"2024-04-24T09:19:09.837628+0000","last_became_peered":"2024-04-24T09:19:09.837628+0000","last_unstale":"2024-04-24T09:20:27.547057+0000","last_undegraded":"2024-04-24T09:20:27.547057+0000","last_fullsized":"2024-04-24T09:20:27.547057+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:29:56.967777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.551958+0000","last_change":"2024-04-24T09:19:09.846296+0000","last_active":"2024-04-24T09:20:27.551958+0000","last_peered":"2024-04-24T09:20:27.551958+0000","last_clean":"2024-04-24T09:20:27.551958+0000","last_became_active":"2024-04-24T09:19:09.846010+0000","last_became_peered":"2024-04-24T09:19:09.846010+0000","last_unstale":"2024-04-24T09:20:27.551958+0000","last_undegraded":"2024-04-24T09:20:27.551958+0000","last_fullsized":"2024-04-24T09:20:27.551958+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:29:38.507747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.561542+0000","last_change":"2024-04-24T09:19:09.866821+0000","last_active":"2024-04-24T09:20:27.561542+0000","last_peered":"2024-04-24T09:20:27.561542+0000","last_clean":"2024-04-24T09:20:27.561542+0000","last_became_active":"2024-04-24T09:19:09.861793+0000","last_became_peered":"2024-04-24T09:19:09.861793+0000","last_unstale":"2024-04-24T09:20:27.561542+0000","last_undegraded":"2024-04-24T09:20:27.561542+0000","last_fullsized":"2024-04-24T09:20:27.561542+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:25:25.095642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.546659+0000","last_change":"2024-04-24T09:19:09.846433+0000","last_active":"2024-04-24T09:20:27.546659+0000","last_peered":"2024-04-24T09:20:27.546659+0000","last_clean":"2024-04-24T09:20:27.546659+0000","last_became_active":"2024-04-24T09:19:09.846044+0000","last_became_peered":"2024-04-24T09:19:09.846044+0000","last_unstale":"2024-04-24T09:20:27.546659+0000","last_undegraded":"2024-04-24T09:20:27.546659+0000","last_fullsized":"2024-04-24T09:20:27.546659+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:02:41.566626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.547736+0000","last_change":"2024-04-24T09:19:09.836662+0000","last_active":"2024-04-24T09:20:27.547736+0000","last_peered":"2024-04-24T09:20:27.547736+0000","last_clean":"2024-04-24T09:20:27.547736+0000","last_became_active":"2024-04-24T09:19:09.836283+0000","last_became_peered":"2024-04-24T09:19:09.836283+0000","last_unstale":"2024-04-24T09:20:27.547736+0000","last_undegraded":"2024-04-24T09:20:27.547736+0000","last_fullsized":"2024-04-24T09:20:27.547736+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:16:16.503469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.549856+0000","last_change":"2024-04-24T09:19:09.844369+0000","last_active":"2024-04-24T09:20:27.549856+0000","last_peered":"2024-04-24T09:20:27.549856+0000","last_clean":"2024-04-24T09:20:27.549856+0000","last_became_active":"2024-04-24T09:19:09.844044+0000","last_became_peered":"2024-04-24T09:19:09.844044+0000","last_unstale":"2024-04-24T09:20:27.549856+0000","last_undegraded":"2024-04-24T09:20:27.549856+0000","last_fullsized":"2024-04-24T09:20:27.549856+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:51:45.861678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.537668+0000","last_change":"2024-04-24T09:19:58.958306+0000","last_active":"2024-04-24T09:20:27.537668+0000","last_peered":"2024-04-24T09:20:27.537668+0000","last_clean":"2024-04-24T09:20:27.537668+0000","last_became_active":"2024-04-24T09:19:58.957743+0000","last_became_peered":"2024-04-24T09:19:58.957743+0000","last_unstale":"2024-04-24T09:20:27.537668+0000","last_undegraded":"2024-04-24T09:20:27.537668+0000","last_fullsized":"2024-04-24T09:20:27.537668+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:54:07.144064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.562128+0000","last_change":"2024-04-24T09:19:09.847860+0000","last_active":"2024-04-24T09:20:27.562128+0000","last_peered":"2024-04-24T09:20:27.562128+0000","last_clean":"2024-04-24T09:20:27.562128+0000","last_became_active":"2024-04-24T09:19:09.847477+0000","last_became_peered":"2024-04-24T09:19:09.847477+0000","last_unstale":"2024-04-24T09:20:27.562128+0000","last_undegraded":"2024-04-24T09:20:27.562128+0000","last_fullsized":"2024-04-24T09:20:27.562128+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:39:44.655033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.555576+0000","last_change":"2024-04-24T09:19:09.854561+0000","last_active":"2024-04-24T09:20:27.555576+0000","last_peered":"2024-04-24T09:20:27.555576+0000","last_clean":"2024-04-24T09:20:27.555576+0000","last_became_active":"2024-04-24T09:19:09.849681+0000","last_became_peered":"2024-04-24T09:19:09.849681+0000","last_unstale":"2024-04-24T09:20:27.555576+0000","last_undegraded":"2024-04-24T09:20:27.555576+0000","last_fullsized":"2024-04-24T09:20:27.555576+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:46:36.572491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.555184+0000","last_change":"2024-04-24T09:19:58.620539+0000","last_active":"2024-04-24T09:20:27.555184+0000","last_peered":"2024-04-24T09:20:27.555184+0000","last_clean":"2024-04-24T09:20:27.555184+0000","last_became_active":"2024-04-24T09:19:58.620243+0000","last_became_peered":"2024-04-24T09:19:58.620243+0000","last_unstale":"2024-04-24T09:20:27.555184+0000","last_undegraded":"2024-04-24T09:20:27.555184+0000","last_fullsized":"2024-04-24T09:20:27.555184+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:35:34.448731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.558031+0000","last_change":"2024-04-24T09:19:09.837995+0000","last_active":"2024-04-24T09:20:27.558031+0000","last_peered":"2024-04-24T09:20:27.558031+0000","last_clean":"2024-04-24T09:20:27.558031+0000","last_became_active":"2024-04-24T09:19:09.837625+0000","last_became_peered":"2024-04-24T09:19:09.837625+0000","last_unstale":"2024-04-24T09:20:27.558031+0000","last_undegraded":"2024-04-24T09:20:27.558031+0000","last_fullsized":"2024-04-24T09:20:27.558031+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:14:30.602137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.549548+0000","last_change":"2024-04-24T09:19:09.855596+0000","last_active":"2024-04-24T09:20:27.549548+0000","last_peered":"2024-04-24T09:20:27.549548+0000","last_clean":"2024-04-24T09:20:27.549548+0000","last_became_active":"2024-04-24T09:19:09.855250+0000","last_became_peered":"2024-04-24T09:19:09.855250+0000","last_unstale":"2024-04-24T09:20:27.549548+0000","last_undegraded":"2024-04-24T09:20:27.549548+0000","last_fullsized":"2024-04-24T09:20:27.549548+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:48:58.862920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.554044+0000","last_change":"2024-04-24T09:19:09.844898+0000","last_active":"2024-04-24T09:20:27.554044+0000","last_peered":"2024-04-24T09:20:27.554044+0000","last_clean":"2024-04-24T09:20:27.554044+0000","last_became_active":"2024-04-24T09:19:09.844566+0000","last_became_peered":"2024-04-24T09:19:09.844566+0000","last_unstale":"2024-04-24T09:20:27.554044+0000","last_undegraded":"2024-04-24T09:20:27.554044+0000","last_fullsized":"2024-04-24T09:20:27.554044+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:15:34.145221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.555152+0000","last_change":"2024-04-24T09:19:09.856299+0000","last_active":"2024-04-24T09:20:27.555152+0000","last_peered":"2024-04-24T09:20:27.555152+0000","last_clean":"2024-04-24T09:20:27.555152+0000","last_became_active":"2024-04-24T09:19:09.855963+0000","last_became_peered":"2024-04-24T09:19:09.855963+0000","last_unstale":"2024-04-24T09:20:27.555152+0000","last_undegraded":"2024-04-24T09:20:27.555152+0000","last_fullsized":"2024-04-24T09:20:27.555152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:10:27.372894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":128,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:50.431195+0000","last_change":"2024-04-24T09:19:08.973018+0000","last_active":"2024-04-24T09:20:50.431195+0000","last_peered":"2024-04-24T09:20:50.431195+0000","last_clean":"2024-04-24T09:20:50.431195+0000","last_became_active":"2024-04-24T09:19:08.972637+0000","last_became_peered":"2024-04-24T09:19:08.972637+0000","last_unstale":"2024-04-24T09:20:50.431195+0000","last_undegraded":"2024-04-24T09:20:50.431195+0000","last_fullsized":"2024-04-24T09:20:50.431195+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:11:10.890872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":96,"num_read_kb":96,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.550818+0000","last_change":"2024-04-24T09:19:09.862806+0000","last_active":"2024-04-24T09:20:27.550818+0000","last_peered":"2024-04-24T09:20:27.550818+0000","last_clean":"2024-04-24T09:20:27.550818+0000","last_became_active":"2024-04-24T09:19:09.862499+0000","last_became_peered":"2024-04-24T09:19:09.862499+0000","last_unstale":"2024-04-24T09:20:27.550818+0000","last_undegraded":"2024-04-24T09:20:27.550818+0000","last_fullsized":"2024-04-24T09:20:27.550818+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:13:19.494260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.624945+0000","last_change":"2024-04-24T09:19:08.973130+0000","last_active":"2024-04-24T09:19:58.624945+0000","last_peered":"2024-04-24T09:19:58.624945+0000","last_clean":"2024-04-24T09:19:58.624945+0000","last_became_active":"2024-04-24T09:19:08.972797+0000","last_became_peered":"2024-04-24T09:19:08.972797+0000","last_unstale":"2024-04-24T09:19:58.624945+0000","last_undegraded":"2024-04-24T09:19:58.624945+0000","last_fullsized":"2024-04-24T09:19:58.624945+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:21:49.236525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.557800+0000","last_change":"2024-04-24T09:19:09.860809+0000","last_active":"2024-04-24T09:20:27.557800+0000","last_peered":"2024-04-24T09:20:27.557800+0000","last_clean":"2024-04-24T09:20:27.557800+0000","last_became_active":"2024-04-24T09:19:09.860480+0000","last_became_peered":"2024-04-24T09:19:09.860480+0000","last_unstale":"2024-04-24T09:20:27.557800+0000","last_undegraded":"2024-04-24T09:20:27.557800+0000","last_fullsized":"2024-04-24T09:20:27.557800+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:39:21.114116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.617760+0000","last_change":"2024-04-24T09:19:08.950145+0000","last_active":"2024-04-24T09:19:58.617760+0000","last_peered":"2024-04-24T09:19:58.617760+0000","last_clean":"2024-04-24T09:19:58.617760+0000","last_became_active":"2024-04-24T09:19:08.949808+0000","last_became_peered":"2024-04-24T09:19:08.949808+0000","last_unstale":"2024-04-24T09:19:58.617760+0000","last_undegraded":"2024-04-24T09:19:58.617760+0000","last_fullsized":"2024-04-24T09:19:58.617760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:15:36.962348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.560944+0000","last_change":"2024-04-24T09:19:09.850123+0000","last_active":"2024-04-24T09:20:27.560944+0000","last_peered":"2024-04-24T09:20:27.560944+0000","last_clean":"2024-04-24T09:20:27.560944+0000","last_became_active":"2024-04-24T09:19:09.849770+0000","last_became_peered":"2024-04-24T09:19:09.849770+0000","last_unstale":"2024-04-24T09:20:27.560944+0000","last_undegraded":"2024-04-24T09:20:27.560944+0000","last_fullsized":"2024-04-24T09:20:27.560944+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:10:00.374950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.618394+0000","last_change":"2024-04-24T09:19:08.952190+0000","last_active":"2024-04-24T09:19:58.618394+0000","last_peered":"2024-04-24T09:19:58.618394+0000","last_clean":"2024-04-24T09:19:58.618394+0000","last_became_active":"2024-04-24T09:19:08.951848+0000","last_became_peered":"2024-04-24T09:19:08.951848+0000","last_unstale":"2024-04-24T09:19:58.618394+0000","last_undegraded":"2024-04-24T09:19:58.618394+0000","last_fullsized":"2024-04-24T09:19:58.618394+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:02:54.892658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.542753+0000","last_change":"2024-04-24T09:19:09.845083+0000","last_active":"2024-04-24T09:20:27.542753+0000","last_peered":"2024-04-24T09:20:27.542753+0000","last_clean":"2024-04-24T09:20:27.542753+0000","last_became_active":"2024-04-24T09:19:09.844516+0000","last_became_peered":"2024-04-24T09:19:09.844516+0000","last_unstale":"2024-04-24T09:20:27.542753+0000","last_undegraded":"2024-04-24T09:20:27.542753+0000","last_fullsized":"2024-04-24T09:20:27.542753+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:10:41.995167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":121,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:49.391588+0000","last_change":"2024-04-24T09:19:08.950498+0000","last_active":"2024-04-24T09:20:49.391588+0000","last_peered":"2024-04-24T09:20:49.391588+0000","last_clean":"2024-04-24T09:20:49.391588+0000","last_became_active":"2024-04-24T09:19:08.950189+0000","last_became_peered":"2024-04-24T09:19:08.950189+0000","last_unstale":"2024-04-24T09:20:49.391588+0000","last_undegraded":"2024-04-24T09:20:49.391588+0000","last_fullsized":"2024-04-24T09:20:49.391588+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:40:41.342349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.531098+0000","last_change":"2024-04-24T09:19:09.847388+0000","last_active":"2024-04-24T09:20:27.531098+0000","last_peered":"2024-04-24T09:20:27.531098+0000","last_clean":"2024-04-24T09:20:27.531098+0000","last_became_active":"2024-04-24T09:19:09.846999+0000","last_became_peered":"2024-04-24T09:19:09.846999+0000","last_unstale":"2024-04-24T09:20:27.531098+0000","last_undegraded":"2024-04-24T09:20:27.531098+0000","last_fullsized":"2024-04-24T09:20:27.531098+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:49:09.814212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.956429+0000","last_change":"2024-04-24T09:19:08.965244+0000","last_active":"2024-04-24T09:19:58.956429+0000","last_peered":"2024-04-24T09:19:58.956429+0000","last_clean":"2024-04-24T09:19:58.956429+0000","last_became_active":"2024-04-24T09:19:08.964899+0000","last_became_peered":"2024-04-24T09:19:08.964899+0000","last_unstale":"2024-04-24T09:19:58.956429+0000","last_undegraded":"2024-04-24T09:19:58.956429+0000","last_fullsized":"2024-04-24T09:19:58.956429+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:53:28.035567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.544506+0000","last_change":"2024-04-24T09:19:09.864902+0000","last_active":"2024-04-24T09:20:27.544506+0000","last_peered":"2024-04-24T09:20:27.544506+0000","last_clean":"2024-04-24T09:20:27.544506+0000","last_became_active":"2024-04-24T09:19:09.864500+0000","last_became_peered":"2024-04-24T09:19:09.864500+0000","last_unstale":"2024-04-24T09:20:27.544506+0000","last_undegraded":"2024-04-24T09:20:27.544506+0000","last_fullsized":"2024-04-24T09:20:27.544506+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:37:31.810050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.952883+0000","last_change":"2024-04-24T09:19:08.977987+0000","last_active":"2024-04-24T09:19:58.952883+0000","last_peered":"2024-04-24T09:19:58.952883+0000","last_clean":"2024-04-24T09:19:58.952883+0000","last_became_active":"2024-04-24T09:19:08.977666+0000","last_became_peered":"2024-04-24T09:19:08.977666+0000","last_unstale":"2024-04-24T09:19:58.952883+0000","last_undegraded":"2024-04-24T09:19:58.952883+0000","last_fullsized":"2024-04-24T09:19:58.952883+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:21:21.328463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.559798+0000","last_change":"2024-04-24T09:19:09.863990+0000","last_active":"2024-04-24T09:20:27.559798+0000","last_peered":"2024-04-24T09:20:27.559798+0000","last_clean":"2024-04-24T09:20:27.559798+0000","last_became_active":"2024-04-24T09:19:09.854938+0000","last_became_peered":"2024-04-24T09:19:09.854938+0000","last_unstale":"2024-04-24T09:20:27.559798+0000","last_undegraded":"2024-04-24T09:20:27.559798+0000","last_fullsized":"2024-04-24T09:20:27.559798+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:41:55.054373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.620691+0000","last_change":"2024-04-24T09:19:08.949938+0000","last_active":"2024-04-24T09:19:58.620691+0000","last_peered":"2024-04-24T09:19:58.620691+0000","last_clean":"2024-04-24T09:19:58.620691+0000","last_became_active":"2024-04-24T09:19:08.949561+0000","last_became_peered":"2024-04-24T09:19:08.949561+0000","last_unstale":"2024-04-24T09:19:58.620691+0000","last_undegraded":"2024-04-24T09:19:58.620691+0000","last_fullsized":"2024-04-24T09:19:58.620691+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:55:50.488129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.553406+0000","last_change":"2024-04-24T09:19:09.841782+0000","last_active":"2024-04-24T09:20:27.553406+0000","last_peered":"2024-04-24T09:20:27.553406+0000","last_clean":"2024-04-24T09:20:27.553406+0000","last_became_active":"2024-04-24T09:19:09.841452+0000","last_became_peered":"2024-04-24T09:19:09.841452+0000","last_unstale":"2024-04-24T09:20:27.553406+0000","last_undegraded":"2024-04-24T09:20:27.553406+0000","last_fullsized":"2024-04-24T09:20:27.553406+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:58:19.618999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":209,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:48.389395+0000","last_change":"2024-04-24T09:19:08.973169+0000","last_active":"2024-04-24T09:20:48.389395+0000","last_peered":"2024-04-24T09:20:48.389395+0000","last_clean":"2024-04-24T09:20:48.389395+0000","last_became_active":"2024-04-24T09:19:08.972835+0000","last_became_peered":"2024-04-24T09:19:08.972835+0000","last_unstale":"2024-04-24T09:20:48.389395+0000","last_undegraded":"2024-04-24T09:20:48.389395+0000","last_fullsized":"2024-04-24T09:20:48.389395+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:34:04.156280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":186,"num_read_kb":186,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.556131+0000","last_change":"2024-04-24T09:19:09.854957+0000","last_active":"2024-04-24T09:20:27.556131+0000","last_peered":"2024-04-24T09:20:27.556131+0000","last_clean":"2024-04-24T09:20:27.556131+0000","last_became_active":"2024-04-24T09:19:09.854574+0000","last_became_peered":"2024-04-24T09:19:09.854574+0000","last_unstale":"2024-04-24T09:20:27.556131+0000","last_undegraded":"2024-04-24T09:20:27.556131+0000","last_fullsized":"2024-04-24T09:20:27.556131+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:07:28.296335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.618727+0000","last_change":"2024-04-24T09:19:08.953179+0000","last_active":"2024-04-24T09:19:58.618727+0000","last_peered":"2024-04-24T09:19:58.618727+0000","last_clean":"2024-04-24T09:19:58.618727+0000","last_became_active":"2024-04-24T09:19:08.952864+0000","last_became_peered":"2024-04-24T09:19:08.952864+0000","last_unstale":"2024-04-24T09:19:58.618727+0000","last_undegraded":"2024-04-24T09:19:58.618727+0000","last_fullsized":"2024-04-24T09:19:58.618727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:22:31.470593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.556402+0000","last_change":"2024-04-24T09:19:09.850855+0000","last_active":"2024-04-24T09:20:27.556402+0000","last_peered":"2024-04-24T09:20:27.556402+0000","last_clean":"2024-04-24T09:20:27.556402+0000","last_became_active":"2024-04-24T09:19:09.850535+0000","last_became_peered":"2024-04-24T09:19:09.850535+0000","last_unstale":"2024-04-24T09:20:27.556402+0000","last_undegraded":"2024-04-24T09:20:27.556402+0000","last_fullsized":"2024-04-24T09:20:27.556402+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:27:23.294935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.620766+0000","last_change":"2024-04-24T09:19:08.967882+0000","last_active":"2024-04-24T09:19:58.620766+0000","last_peered":"2024-04-24T09:19:58.620766+0000","last_clean":"2024-04-24T09:19:58.620766+0000","last_became_active":"2024-04-24T09:19:08.967625+0000","last_became_peered":"2024-04-24T09:19:08.967625+0000","last_unstale":"2024-04-24T09:19:58.620766+0000","last_undegraded":"2024-04-24T09:19:58.620766+0000","last_fullsized":"2024-04-24T09:19:58.620766+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:39:57.802717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.548220+0000","last_change":"2024-04-24T09:19:09.844721+0000","last_active":"2024-04-24T09:20:27.548220+0000","last_peered":"2024-04-24T09:20:27.548220+0000","last_clean":"2024-04-24T09:20:27.548220+0000","last_became_active":"2024-04-24T09:19:09.844365+0000","last_became_peered":"2024-04-24T09:19:09.844365+0000","last_unstale":"2024-04-24T09:20:27.548220+0000","last_undegraded":"2024-04-24T09:20:27.548220+0000","last_fullsized":"2024-04-24T09:20:27.548220+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:11:11.215015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'33","reported_seq":247,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:48.406099+0000","last_change":"2024-04-24T09:19:08.955409+0000","last_active":"2024-04-24T09:20:48.406099+0000","last_peered":"2024-04-24T09:20:48.406099+0000","last_clean":"2024-04-24T09:20:48.406099+0000","last_became_active":"2024-04-24T09:19:08.955065+0000","last_became_peered":"2024-04-24T09:19:08.955065+0000","last_unstale":"2024-04-24T09:20:48.406099+0000","last_undegraded":"2024-04-24T09:20:48.406099+0000","last_fullsized":"2024-04-24T09:20:48.406099+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:28:34.411386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61295,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":195,"num_read_kb":158,"num_write":35,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.546315+0000","last_change":"2024-04-24T09:19:09.846521+0000","last_active":"2024-04-24T09:20:27.546315+0000","last_peered":"2024-04-24T09:20:27.546315+0000","last_clean":"2024-04-24T09:20:27.546315+0000","last_became_active":"2024-04-24T09:19:09.846162+0000","last_became_peered":"2024-04-24T09:19:09.846162+0000","last_unstale":"2024-04-24T09:20:27.546315+0000","last_undegraded":"2024-04-24T09:20:27.546315+0000","last_fullsized":"2024-04-24T09:20:27.546315+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:05:50.383432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.616812+0000","last_change":"2024-04-24T09:19:08.963754+0000","last_active":"2024-04-24T09:19:58.616812+0000","last_peered":"2024-04-24T09:19:58.616812+0000","last_clean":"2024-04-24T09:19:58.616812+0000","last_became_active":"2024-04-24T09:19:08.963379+0000","last_became_peered":"2024-04-24T09:19:08.963379+0000","last_unstale":"2024-04-24T09:19:58.616812+0000","last_undegraded":"2024-04-24T09:19:58.616812+0000","last_fullsized":"2024-04-24T09:19:58.616812+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:25:01.538860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.548852+0000","last_change":"2024-04-24T09:19:09.863649+0000","last_active":"2024-04-24T09:20:27.548852+0000","last_peered":"2024-04-24T09:20:27.548852+0000","last_clean":"2024-04-24T09:20:27.548852+0000","last_became_active":"2024-04-24T09:19:09.863295+0000","last_became_peered":"2024-04-24T09:19:09.863295+0000","last_unstale":"2024-04-24T09:20:27.548852+0000","last_undegraded":"2024-04-24T09:20:27.548852+0000","last_fullsized":"2024-04-24T09:20:27.548852+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:06:56.404909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.620355+0000","last_change":"2024-04-24T09:19:08.961409+0000","last_active":"2024-04-24T09:19:58.620355+0000","last_peered":"2024-04-24T09:19:58.620355+0000","last_clean":"2024-04-24T09:19:58.620355+0000","last_became_active":"2024-04-24T09:19:08.961085+0000","last_became_peered":"2024-04-24T09:19:08.961085+0000","last_unstale":"2024-04-24T09:19:58.620355+0000","last_undegraded":"2024-04-24T09:19:58.620355+0000","last_fullsized":"2024-04-24T09:19:58.620355+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:10:32.816204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.561538+0000","last_change":"2024-04-24T09:19:09.845992+0000","last_active":"2024-04-24T09:20:27.561538+0000","last_peered":"2024-04-24T09:20:27.561538+0000","last_clean":"2024-04-24T09:20:27.561538+0000","last_became_active":"2024-04-24T09:19:09.845673+0000","last_became_peered":"2024-04-24T09:19:09.845673+0000","last_unstale":"2024-04-24T09:20:27.561538+0000","last_undegraded":"2024-04-24T09:20:27.561538+0000","last_fullsized":"2024-04-24T09:20:27.561538+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:42:53.136476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'7","reported_seq":124,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:49.432135+0000","last_change":"2024-04-24T09:19:08.954872+0000","last_active":"2024-04-24T09:20:49.432135+0000","last_peered":"2024-04-24T09:20:49.432135+0000","last_clean":"2024-04-24T09:20:49.432135+0000","last_became_active":"2024-04-24T09:19:08.954540+0000","last_became_peered":"2024-04-24T09:19:08.954540+0000","last_unstale":"2024-04-24T09:20:49.432135+0000","last_undegraded":"2024-04-24T09:20:49.432135+0000","last_fullsized":"2024-04-24T09:20:49.432135+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:32:15.637337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":99,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.547856+0000","last_change":"2024-04-24T09:19:09.841735+0000","last_active":"2024-04-24T09:20:27.547856+0000","last_peered":"2024-04-24T09:20:27.547856+0000","last_clean":"2024-04-24T09:20:27.547856+0000","last_became_active":"2024-04-24T09:19:09.841422+0000","last_became_peered":"2024-04-24T09:19:09.841422+0000","last_unstale":"2024-04-24T09:20:27.547856+0000","last_undegraded":"2024-04-24T09:20:27.547856+0000","last_fullsized":"2024-04-24T09:20:27.547856+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:37:46.128344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.616590+0000","last_change":"2024-04-24T09:19:08.971108+0000","last_active":"2024-04-24T09:19:58.616590+0000","last_peered":"2024-04-24T09:19:58.616590+0000","last_clean":"2024-04-24T09:19:58.616590+0000","last_became_active":"2024-04-24T09:19:08.970808+0000","last_became_peered":"2024-04-24T09:19:08.970808+0000","last_unstale":"2024-04-24T09:19:58.616590+0000","last_undegraded":"2024-04-24T09:19:58.616590+0000","last_fullsized":"2024-04-24T09:19:58.616590+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:37:26.311351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.533903+0000","last_change":"2024-04-24T09:19:09.848041+0000","last_active":"2024-04-24T09:20:27.533903+0000","last_peered":"2024-04-24T09:20:27.533903+0000","last_clean":"2024-04-24T09:20:27.533903+0000","last_became_active":"2024-04-24T09:19:09.847090+0000","last_became_peered":"2024-04-24T09:19:09.847090+0000","last_unstale":"2024-04-24T09:20:27.533903+0000","last_undegraded":"2024-04-24T09:20:27.533903+0000","last_fullsized":"2024-04-24T09:20:27.533903+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:45:13.652402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.617244+0000","last_change":"2024-04-24T09:19:08.952061+0000","last_active":"2024-04-24T09:19:58.617244+0000","last_peered":"2024-04-24T09:19:58.617244+0000","last_clean":"2024-04-24T09:19:58.617244+0000","last_became_active":"2024-04-24T09:19:08.950549+0000","last_became_peered":"2024-04-24T09:19:08.950549+0000","last_unstale":"2024-04-24T09:19:58.617244+0000","last_undegraded":"2024-04-24T09:19:58.617244+0000","last_fullsized":"2024-04-24T09:19:58.617244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:41:40.949650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.535934+0000","last_change":"2024-04-24T09:19:09.848619+0000","last_active":"2024-04-24T09:20:27.535934+0000","last_peered":"2024-04-24T09:20:27.535934+0000","last_clean":"2024-04-24T09:20:27.535934+0000","last_became_active":"2024-04-24T09:19:09.848246+0000","last_became_peered":"2024-04-24T09:19:09.848246+0000","last_unstale":"2024-04-24T09:20:27.535934+0000","last_undegraded":"2024-04-24T09:20:27.535934+0000","last_fullsized":"2024-04-24T09:20:27.535934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:30:22.925368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.614283+0000","last_change":"2024-04-24T09:19:08.977448+0000","last_active":"2024-04-24T09:19:58.614283+0000","last_peered":"2024-04-24T09:19:58.614283+0000","last_clean":"2024-04-24T09:19:58.614283+0000","last_became_active":"2024-04-24T09:19:08.976525+0000","last_became_peered":"2024-04-24T09:19:08.976525+0000","last_unstale":"2024-04-24T09:19:58.614283+0000","last_undegraded":"2024-04-24T09:19:58.614283+0000","last_fullsized":"2024-04-24T09:19:58.614283+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:06:59.151486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.552995+0000","last_change":"2024-04-24T09:19:09.856260+0000","last_active":"2024-04-24T09:20:27.552995+0000","last_peered":"2024-04-24T09:20:27.552995+0000","last_clean":"2024-04-24T09:20:27.552995+0000","last_became_active":"2024-04-24T09:19:09.855895+0000","last_became_peered":"2024-04-24T09:19:09.855895+0000","last_unstale":"2024-04-24T09:20:27.552995+0000","last_undegraded":"2024-04-24T09:20:27.552995+0000","last_fullsized":"2024-04-24T09:20:27.552995+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:44:22.804210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":117,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:47.426582+0000","last_change":"2024-04-24T09:19:08.962268+0000","last_active":"2024-04-24T09:20:47.426582+0000","last_peered":"2024-04-24T09:20:47.426582+0000","last_clean":"2024-04-24T09:20:47.426582+0000","last_became_active":"2024-04-24T09:19:08.961930+0000","last_became_peered":"2024-04-24T09:19:08.961930+0000","last_unstale":"2024-04-24T09:20:47.426582+0000","last_undegraded":"2024-04-24T09:20:47.426582+0000","last_fullsized":"2024-04-24T09:20:47.426582+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:51:28.357674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":93,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.546550+0000","last_change":"2024-04-24T09:19:09.850782+0000","last_active":"2024-04-24T09:20:27.546550+0000","last_peered":"2024-04-24T09:20:27.546550+0000","last_clean":"2024-04-24T09:20:27.546550+0000","last_became_active":"2024-04-24T09:19:09.850458+0000","last_became_peered":"2024-04-24T09:19:09.850458+0000","last_unstale":"2024-04-24T09:20:27.546550+0000","last_undegraded":"2024-04-24T09:20:27.546550+0000","last_fullsized":"2024-04-24T09:20:27.546550+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:02:19.464011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.550926+0000","last_change":"2024-04-24T09:19:58.619597+0000","last_active":"2024-04-24T09:20:27.550926+0000","last_peered":"2024-04-24T09:20:27.550926+0000","last_clean":"2024-04-24T09:20:27.550926+0000","last_became_active":"2024-04-24T09:19:58.619160+0000","last_became_peered":"2024-04-24T09:19:58.619160+0000","last_unstale":"2024-04-24T09:20:27.550926+0000","last_undegraded":"2024-04-24T09:20:27.550926+0000","last_fullsized":"2024-04-24T09:20:27.550926+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:34:05.633062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"88'6","reported_seq":122,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:50.433293+0000","last_change":"2024-04-24T09:19:08.972070+0000","last_active":"2024-04-24T09:20:50.433293+0000","last_peered":"2024-04-24T09:20:50.433293+0000","last_clean":"2024-04-24T09:20:50.433293+0000","last_became_active":"2024-04-24T09:19:08.971699+0000","last_became_peered":"2024-04-24T09:19:08.971699+0000","last_unstale":"2024-04-24T09:20:50.433293+0000","last_undegraded":"2024-04-24T09:20:50.433293+0000","last_fullsized":"2024-04-24T09:20:50.433293+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:38:13.312940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.559887+0000","last_change":"2024-04-24T09:19:09.855902+0000","last_active":"2024-04-24T09:20:27.559887+0000","last_peered":"2024-04-24T09:20:27.559887+0000","last_clean":"2024-04-24T09:20:27.559887+0000","last_became_active":"2024-04-24T09:19:09.855503+0000","last_became_peered":"2024-04-24T09:19:09.855503+0000","last_unstale":"2024-04-24T09:20:27.559887+0000","last_undegraded":"2024-04-24T09:20:27.559887+0000","last_fullsized":"2024-04-24T09:20:27.559887+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:29:41.621269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.614543+0000","last_change":"2024-04-24T09:19:08.973774+0000","last_active":"2024-04-24T09:19:58.614543+0000","last_peered":"2024-04-24T09:19:58.614543+0000","last_clean":"2024-04-24T09:19:58.614543+0000","last_became_active":"2024-04-24T09:19:08.973372+0000","last_became_peered":"2024-04-24T09:19:08.973372+0000","last_unstale":"2024-04-24T09:19:58.614543+0000","last_undegraded":"2024-04-24T09:19:58.614543+0000","last_fullsized":"2024-04-24T09:19:58.614543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:25:12.910023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":343,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.618651+0000","last_change":"2024-04-24T09:17:48.282773+0000","last_active":"2024-04-24T09:19:58.618651+0000","last_peered":"2024-04-24T09:19:58.618651+0000","last_clean":"2024-04-24T09:19:58.618651+0000","last_became_active":"2024-04-24T09:17:48.282128+0000","last_became_peered":"2024-04-24T09:17:48.282128+0000","last_unstale":"2024-04-24T09:19:58.618651+0000","last_undegraded":"2024-04-24T09:19:58.618651+0000","last_fullsized":"2024-04-24T09:19:58.618651+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:13:45.562295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:13:45.562295+0000","last_clean_scrub_stamp":"2024-04-24T09:13:45.562295+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:54:34.557051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.538367+0000","last_change":"2024-04-24T09:19:09.847771+0000","last_active":"2024-04-24T09:20:27.538367+0000","last_peered":"2024-04-24T09:20:27.538367+0000","last_clean":"2024-04-24T09:20:27.538367+0000","last_became_active":"2024-04-24T09:19:09.845873+0000","last_became_peered":"2024-04-24T09:19:09.845873+0000","last_unstale":"2024-04-24T09:20:27.538367+0000","last_undegraded":"2024-04-24T09:20:27.538367+0000","last_fullsized":"2024-04-24T09:20:27.538367+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:31:46.124082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.624968+0000","last_change":"2024-04-24T09:19:58.624968+0000","last_active":"2024-04-24T09:19:58.624968+0000","last_peered":"2024-04-24T09:19:58.624968+0000","last_clean":"2024-04-24T09:19:58.624968+0000","last_became_active":"2024-04-24T09:19:58.621737+0000","last_became_peered":"2024-04-24T09:19:58.621737+0000","last_unstale":"2024-04-24T09:19:58.624968+0000","last_undegraded":"2024-04-24T09:19:58.624968+0000","last_fullsized":"2024-04-24T09:19:58.624968+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:55:20.244149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.617560+0000","last_change":"2024-04-24T09:19:08.971475+0000","last_active":"2024-04-24T09:19:58.617560+0000","last_peered":"2024-04-24T09:19:58.617560+0000","last_clean":"2024-04-24T09:19:58.617560+0000","last_became_active":"2024-04-24T09:19:08.971188+0000","last_became_peered":"2024-04-24T09:19:08.971188+0000","last_unstale":"2024-04-24T09:19:58.617560+0000","last_undegraded":"2024-04-24T09:19:58.617560+0000","last_fullsized":"2024-04-24T09:19:58.617560+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:50:06.630182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.556407+0000","last_change":"2024-04-24T09:19:09.857102+0000","last_active":"2024-04-24T09:20:27.556407+0000","last_peered":"2024-04-24T09:20:27.556407+0000","last_clean":"2024-04-24T09:20:27.556407+0000","last_became_active":"2024-04-24T09:19:09.856632+0000","last_became_peered":"2024-04-24T09:19:09.856632+0000","last_unstale":"2024-04-24T09:20:27.556407+0000","last_undegraded":"2024-04-24T09:20:27.556407+0000","last_fullsized":"2024-04-24T09:20:27.556407+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:04:03.300741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'13","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.636195+0000","last_change":"2024-04-24T09:19:58.635644+0000","last_active":"2024-04-24T09:19:58.636195+0000","last_peered":"2024-04-24T09:19:58.636195+0000","last_clean":"2024-04-24T09:19:58.636195+0000","last_became_active":"2024-04-24T09:19:58.625594+0000","last_became_peered":"2024-04-24T09:19:58.625594+0000","last_unstale":"2024-04-24T09:19:58.636195+0000","last_undegraded":"2024-04-24T09:19:58.636195+0000","last_fullsized":"2024-04-24T09:19:58.636195+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:29:38.682899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.561122+0000","last_change":"2024-04-24T09:19:09.863386+0000","last_active":"2024-04-24T09:20:27.561122+0000","last_peered":"2024-04-24T09:20:27.561122+0000","last_clean":"2024-04-24T09:20:27.561122+0000","last_became_active":"2024-04-24T09:19:09.856300+0000","last_became_peered":"2024-04-24T09:19:09.856300+0000","last_unstale":"2024-04-24T09:20:27.561122+0000","last_undegraded":"2024-04-24T09:20:27.561122+0000","last_fullsized":"2024-04-24T09:20:27.561122+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:26:37.603307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.614840+0000","last_change":"2024-04-24T09:19:08.961751+0000","last_active":"2024-04-24T09:19:58.614840+0000","last_peered":"2024-04-24T09:19:58.614840+0000","last_clean":"2024-04-24T09:19:58.614840+0000","last_became_active":"2024-04-24T09:19:08.961389+0000","last_became_peered":"2024-04-24T09:19:08.961389+0000","last_unstale":"2024-04-24T09:19:58.614840+0000","last_undegraded":"2024-04-24T09:19:58.614840+0000","last_fullsized":"2024-04-24T09:19:58.614840+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:25:17.913335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.561229+0000","last_change":"2024-04-24T09:19:09.862140+0000","last_active":"2024-04-24T09:20:27.561229+0000","last_peered":"2024-04-24T09:20:27.561229+0000","last_clean":"2024-04-24T09:20:27.561229+0000","last_became_active":"2024-04-24T09:19:09.860776+0000","last_became_peered":"2024-04-24T09:19:09.860776+0000","last_unstale":"2024-04-24T09:20:27.561229+0000","last_undegraded":"2024-04-24T09:20:27.561229+0000","last_fullsized":"2024-04-24T09:20:27.561229+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:08:29.437422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.996452+0000","last_change":"2024-04-24T09:19:08.973503+0000","last_active":"2024-04-24T09:19:58.996452+0000","last_peered":"2024-04-24T09:19:58.996452+0000","last_clean":"2024-04-24T09:19:58.996452+0000","last_became_active":"2024-04-24T09:19:08.973153+0000","last_became_peered":"2024-04-24T09:19:08.973153+0000","last_unstale":"2024-04-24T09:19:58.996452+0000","last_undegraded":"2024-04-24T09:19:58.996452+0000","last_fullsized":"2024-04-24T09:19:58.996452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:57:21.489696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.507289+0000","last_change":"2024-04-24T09:19:09.838175+0000","last_active":"2024-04-24T09:20:27.507289+0000","last_peered":"2024-04-24T09:20:27.507289+0000","last_clean":"2024-04-24T09:20:27.507289+0000","last_became_active":"2024-04-24T09:19:09.837781+0000","last_became_peered":"2024-04-24T09:19:09.837781+0000","last_unstale":"2024-04-24T09:20:27.507289+0000","last_undegraded":"2024-04-24T09:20:27.507289+0000","last_fullsized":"2024-04-24T09:20:27.507289+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:48:03.967587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.622324+0000","last_change":"2024-04-24T09:19:08.976790+0000","last_active":"2024-04-24T09:19:58.622324+0000","last_peered":"2024-04-24T09:19:58.622324+0000","last_clean":"2024-04-24T09:19:58.622324+0000","last_became_active":"2024-04-24T09:19:08.975196+0000","last_became_peered":"2024-04-24T09:19:08.975196+0000","last_unstale":"2024-04-24T09:19:58.622324+0000","last_undegraded":"2024-04-24T09:19:58.622324+0000","last_fullsized":"2024-04-24T09:19:58.622324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:54:59.415169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.541194+0000","last_change":"2024-04-24T09:19:09.850958+0000","last_active":"2024-04-24T09:20:27.541194+0000","last_peered":"2024-04-24T09:20:27.541194+0000","last_clean":"2024-04-24T09:20:27.541194+0000","last_became_active":"2024-04-24T09:19:09.850660+0000","last_became_peered":"2024-04-24T09:19:09.850660+0000","last_unstale":"2024-04-24T09:20:27.541194+0000","last_undegraded":"2024-04-24T09:20:27.541194+0000","last_fullsized":"2024-04-24T09:20:27.541194+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:41:27.496169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.630129+0000","last_change":"2024-04-24T09:19:58.629650+0000","last_active":"2024-04-24T09:19:58.630129+0000","last_peered":"2024-04-24T09:19:58.630129+0000","last_clean":"2024-04-24T09:19:58.630129+0000","last_became_active":"2024-04-24T09:19:58.624974+0000","last_became_peered":"2024-04-24T09:19:58.624974+0000","last_unstale":"2024-04-24T09:19:58.630129+0000","last_undegraded":"2024-04-24T09:19:58.630129+0000","last_fullsized":"2024-04-24T09:19:58.630129+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:12:42.319723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.561625+0000","last_change":"2024-04-24T09:19:09.863144+0000","last_active":"2024-04-24T09:20:27.561625+0000","last_peered":"2024-04-24T09:20:27.561625+0000","last_clean":"2024-04-24T09:20:27.561625+0000","last_became_active":"2024-04-24T09:19:09.862772+0000","last_became_peered":"2024-04-24T09:19:09.862772+0000","last_unstale":"2024-04-24T09:20:27.561625+0000","last_undegraded":"2024-04-24T09:20:27.561625+0000","last_fullsized":"2024-04-24T09:20:27.561625+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:05:42.926470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.616697+0000","last_change":"2024-04-24T09:19:08.953467+0000","last_active":"2024-04-24T09:19:58.616697+0000","last_peered":"2024-04-24T09:19:58.616697+0000","last_clean":"2024-04-24T09:19:58.616697+0000","last_became_active":"2024-04-24T09:19:08.953116+0000","last_became_peered":"2024-04-24T09:19:08.953116+0000","last_unstale":"2024-04-24T09:19:58.616697+0000","last_undegraded":"2024-04-24T09:19:58.616697+0000","last_fullsized":"2024-04-24T09:19:58.616697+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:07:35.834466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.562016+0000","last_change":"2024-04-24T09:19:09.852329+0000","last_active":"2024-04-24T09:20:27.562016+0000","last_peered":"2024-04-24T09:20:27.562016+0000","last_clean":"2024-04-24T09:20:27.562016+0000","last_became_active":"2024-04-24T09:19:09.851762+0000","last_became_peered":"2024-04-24T09:19:09.851762+0000","last_unstale":"2024-04-24T09:20:27.562016+0000","last_undegraded":"2024-04-24T09:20:27.562016+0000","last_fullsized":"2024-04-24T09:20:27.562016+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:56:59.234680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'5","reported_seq":71,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:48.708027+0000","last_change":"2024-04-24T09:19:58.626186+0000","last_active":"2024-04-24T09:20:48.708027+0000","last_peered":"2024-04-24T09:20:48.708027+0000","last_clean":"2024-04-24T09:20:48.708027+0000","last_became_active":"2024-04-24T09:19:58.619730+0000","last_became_peered":"2024-04-24T09:19:58.619730+0000","last_unstale":"2024-04-24T09:20:48.708027+0000","last_undegraded":"2024-04-24T09:20:48.708027+0000","last_fullsized":"2024-04-24T09:20:48.708027+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:18:56.284816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":52,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.550088+0000","last_change":"2024-04-24T09:19:09.840340+0000","last_active":"2024-04-24T09:20:27.550088+0000","last_peered":"2024-04-24T09:20:27.550088+0000","last_clean":"2024-04-24T09:20:27.550088+0000","last_became_active":"2024-04-24T09:19:09.839689+0000","last_became_peered":"2024-04-24T09:19:09.839689+0000","last_unstale":"2024-04-24T09:20:27.550088+0000","last_undegraded":"2024-04-24T09:20:27.550088+0000","last_fullsized":"2024-04-24T09:20:27.550088+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:36:06.174519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.620640+0000","last_change":"2024-04-24T09:19:08.959025+0000","last_active":"2024-04-24T09:19:58.620640+0000","last_peered":"2024-04-24T09:19:58.620640+0000","last_clean":"2024-04-24T09:19:58.620640+0000","last_became_active":"2024-04-24T09:19:08.958749+0000","last_became_peered":"2024-04-24T09:19:08.958749+0000","last_unstale":"2024-04-24T09:19:58.620640+0000","last_undegraded":"2024-04-24T09:19:58.620640+0000","last_fullsized":"2024-04-24T09:19:58.620640+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:59:41.334448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.552451+0000","last_change":"2024-04-24T09:19:58.629338+0000","last_active":"2024-04-24T09:20:27.552451+0000","last_peered":"2024-04-24T09:20:27.552451+0000","last_clean":"2024-04-24T09:20:27.552451+0000","last_became_active":"2024-04-24T09:19:58.625976+0000","last_became_peered":"2024-04-24T09:19:58.625976+0000","last_unstale":"2024-04-24T09:20:27.552451+0000","last_undegraded":"2024-04-24T09:20:27.552451+0000","last_fullsized":"2024-04-24T09:20:27.552451+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:17:37.470806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:19:58.953266+0000","last_change":"2024-04-24T09:19:08.965880+0000","last_active":"2024-04-24T09:19:58.953266+0000","last_peered":"2024-04-24T09:19:58.953266+0000","last_clean":"2024-04-24T09:19:58.953266+0000","last_became_active":"2024-04-24T09:19:08.965378+0000","last_became_peered":"2024-04-24T09:19:08.965378+0000","last_unstale":"2024-04-24T09:19:58.953266+0000","last_undegraded":"2024-04-24T09:19:58.953266+0000","last_fullsized":"2024-04-24T09:19:58.953266+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:07.928625+0000","last_clean_scrub_stamp":"2024-04-24T09:19:07.928625+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:08:22.628556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.547817+0000","last_change":"2024-04-24T09:19:09.845151+0000","last_active":"2024-04-24T09:20:27.547817+0000","last_peered":"2024-04-24T09:20:27.547817+0000","last_clean":"2024-04-24T09:20:27.547817+0000","last_became_active":"2024-04-24T09:19:09.844791+0000","last_became_peered":"2024-04-24T09:19:09.844791+0000","last_unstale":"2024-04-24T09:20:27.547817+0000","last_undegraded":"2024-04-24T09:20:27.547817+0000","last_fullsized":"2024-04-24T09:20:27.547817+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:38:00.782591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.538975+0000","last_change":"2024-04-24T09:19:09.844449+0000","last_active":"2024-04-24T09:20:27.538975+0000","last_peered":"2024-04-24T09:20:27.538975+0000","last_clean":"2024-04-24T09:20:27.538975+0000","last_became_active":"2024-04-24T09:19:09.844122+0000","last_became_peered":"2024-04-24T09:19:09.844122+0000","last_unstale":"2024-04-24T09:20:27.538975+0000","last_undegraded":"2024-04-24T09:20:27.538975+0000","last_fullsized":"2024-04-24T09:20:27.538975+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:36:01.857309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.545661+0000","last_change":"2024-04-24T09:19:09.843151+0000","last_active":"2024-04-24T09:20:27.545661+0000","last_peered":"2024-04-24T09:20:27.545661+0000","last_clean":"2024-04-24T09:20:27.545661+0000","last_became_active":"2024-04-24T09:19:09.842519+0000","last_became_peered":"2024-04-24T09:19:09.842519+0000","last_unstale":"2024-04-24T09:20:27.545661+0000","last_undegraded":"2024-04-24T09:20:27.545661+0000","last_fullsized":"2024-04-24T09:20:27.545661+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:26:00.791618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.558108+0000","last_change":"2024-04-24T09:19:09.856291+0000","last_active":"2024-04-24T09:20:27.558108+0000","last_peered":"2024-04-24T09:20:27.558108+0000","last_clean":"2024-04-24T09:20:27.558108+0000","last_became_active":"2024-04-24T09:19:09.855932+0000","last_became_peered":"2024-04-24T09:19:09.855932+0000","last_unstale":"2024-04-24T09:20:27.558108+0000","last_undegraded":"2024-04-24T09:20:27.558108+0000","last_fullsized":"2024-04-24T09:20:27.558108+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:11:14.113321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.562476+0000","last_change":"2024-04-24T09:19:58.955125+0000","last_active":"2024-04-24T09:20:27.562476+0000","last_peered":"2024-04-24T09:20:27.562476+0000","last_clean":"2024-04-24T09:20:27.562476+0000","last_became_active":"2024-04-24T09:19:58.954545+0000","last_became_peered":"2024-04-24T09:19:58.954545+0000","last_unstale":"2024-04-24T09:20:27.562476+0000","last_undegraded":"2024-04-24T09:20:27.562476+0000","last_fullsized":"2024-04-24T09:20:27.562476+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:18:14.132248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.560486+0000","last_change":"2024-04-24T09:19:09.852140+0000","last_active":"2024-04-24T09:20:27.560486+0000","last_peered":"2024-04-24T09:20:27.560486+0000","last_clean":"2024-04-24T09:20:27.560486+0000","last_became_active":"2024-04-24T09:19:09.851766+0000","last_became_peered":"2024-04-24T09:19:09.851766+0000","last_unstale":"2024-04-24T09:20:27.560486+0000","last_undegraded":"2024-04-24T09:20:27.560486+0000","last_fullsized":"2024-04-24T09:20:27.560486+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:14:51.794470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.533826+0000","last_change":"2024-04-24T09:19:09.839389+0000","last_active":"2024-04-24T09:20:27.533826+0000","last_peered":"2024-04-24T09:20:27.533826+0000","last_clean":"2024-04-24T09:20:27.533826+0000","last_became_active":"2024-04-24T09:19:09.838989+0000","last_became_peered":"2024-04-24T09:19:09.838989+0000","last_unstale":"2024-04-24T09:20:27.533826+0000","last_undegraded":"2024-04-24T09:20:27.533826+0000","last_fullsized":"2024-04-24T09:20:27.533826+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:00:28.081837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.546358+0000","last_change":"2024-04-24T09:19:09.845304+0000","last_active":"2024-04-24T09:20:27.546358+0000","last_peered":"2024-04-24T09:20:27.546358+0000","last_clean":"2024-04-24T09:20:27.546358+0000","last_became_active":"2024-04-24T09:19:09.844979+0000","last_became_peered":"2024-04-24T09:19:09.844979+0000","last_unstale":"2024-04-24T09:20:27.546358+0000","last_undegraded":"2024-04-24T09:20:27.546358+0000","last_fullsized":"2024-04-24T09:20:27.546358+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:36:58.294217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.555155+0000","last_change":"2024-04-24T09:19:09.865120+0000","last_active":"2024-04-24T09:20:27.555155+0000","last_peered":"2024-04-24T09:20:27.555155+0000","last_clean":"2024-04-24T09:20:27.555155+0000","last_became_active":"2024-04-24T09:19:09.856837+0000","last_became_peered":"2024-04-24T09:19:09.856837+0000","last_unstale":"2024-04-24T09:20:27.555155+0000","last_undegraded":"2024-04-24T09:20:27.555155+0000","last_fullsized":"2024-04-24T09:20:27.555155+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:30:51.888927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.550991+0000","last_change":"2024-04-24T09:19:09.853625+0000","last_active":"2024-04-24T09:20:27.550991+0000","last_peered":"2024-04-24T09:20:27.550991+0000","last_clean":"2024-04-24T09:20:27.550991+0000","last_became_active":"2024-04-24T09:19:09.853390+0000","last_became_peered":"2024-04-24T09:19:09.853390+0000","last_unstale":"2024-04-24T09:20:27.550991+0000","last_undegraded":"2024-04-24T09:20:27.550991+0000","last_fullsized":"2024-04-24T09:20:27.550991+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:14:48.571283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.555050+0000","last_change":"2024-04-24T09:19:09.851380+0000","last_active":"2024-04-24T09:20:27.555050+0000","last_peered":"2024-04-24T09:20:27.555050+0000","last_clean":"2024-04-24T09:20:27.555050+0000","last_became_active":"2024-04-24T09:19:09.843723+0000","last_became_peered":"2024-04-24T09:19:09.843723+0000","last_unstale":"2024-04-24T09:20:27.555050+0000","last_undegraded":"2024-04-24T09:20:27.555050+0000","last_fullsized":"2024-04-24T09:20:27.555050+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:55:50.103891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.561198+0000","last_change":"2024-04-24T09:19:09.856620+0000","last_active":"2024-04-24T09:20:27.561198+0000","last_peered":"2024-04-24T09:20:27.561198+0000","last_clean":"2024-04-24T09:20:27.561198+0000","last_became_active":"2024-04-24T09:19:09.856101+0000","last_became_peered":"2024-04-24T09:19:09.856101+0000","last_unstale":"2024-04-24T09:20:27.561198+0000","last_undegraded":"2024-04-24T09:20:27.561198+0000","last_fullsized":"2024-04-24T09:20:27.561198+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:32:26.035336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.562463+0000","last_change":"2024-04-24T09:19:09.853959+0000","last_active":"2024-04-24T09:20:27.562463+0000","last_peered":"2024-04-24T09:20:27.562463+0000","last_clean":"2024-04-24T09:20:27.562463+0000","last_became_active":"2024-04-24T09:19:09.853588+0000","last_became_peered":"2024-04-24T09:19:09.853588+0000","last_unstale":"2024-04-24T09:20:27.562463+0000","last_undegraded":"2024-04-24T09:20:27.562463+0000","last_fullsized":"2024-04-24T09:20:27.562463+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:43:09.128758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-24T09:20:27.536846+0000","last_change":"2024-04-24T09:19:09.845317+0000","last_active":"2024-04-24T09:20:27.536846+0000","last_peered":"2024-04-24T09:20:27.536846+0000","last_clean":"2024-04-24T09:20:27.536846+0000","last_became_active":"2024-04-24T09:19:09.844975+0000","last_became_peered":"2024-04-24T09:19:09.844975+0000","last_unstale":"2024-04-24T09:20:27.536846+0000","last_undegraded":"2024-04-24T09:20:27.536846+0000","last_fullsized":"2024-04-24T09:20:27.536846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T09:19:08.934459+0000","last_clean_scrub_stamp":"2024-04-24T09:19:08.934459+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:36:28.053045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":246913,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68249,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":935,"num_read_kb":564,"num_write":243,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1170,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":204747,"data_compressed":5610,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":211,"ondisk_log_size":211,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351012,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":459945,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612647,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1110016,"data_stored":1112801,"data_compressed":43410,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776175,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":459992,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037813,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":978944,"data_stored":460253,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299451,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":1052912,"data_compressed":41540,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462977,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":459592,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1024000,"data_stored":521800,"data_compressed":7667,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":460931,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":520850,"data_compressed":7667,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149778,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":459300,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888140,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":461155,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724615,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":1052058,"data_compressed":41540,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34099,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34099,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35743,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35743,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35743,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35743,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35743,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":63047,"data_compressed":1870,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":62097,"data_compressed":1870,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":63563,"data_compressed":1870,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T09:20:55.211 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-24T09:20:55.211 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-24T09:20:55.211 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-24T09:20:55.211 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-24T09:20:55.696 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:20:55.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:55 smithi114 ceph-mon[32501]: from='client.24875 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T09:20:56.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:55 smithi090 ceph-mon[27825]: from='client.24875 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T09:20:56.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:55 smithi149 ceph-mon[32592]: from='client.24875 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T09:20:56.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:56 smithi114 ceph-mon[32501]: pgmap v377: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-24T09:20:57.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:56 smithi090 ceph-mon[27825]: pgmap v377: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-24T09:20:57.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:56 smithi149 ceph-mon[32592]: pgmap v377: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-24T09:20:58.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.073+0000 7f6fddc00700 1 -- 172.21.15.90:0/2501947716 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fd8102b40 msgr2=0x7f6fd8102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:58.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.073+0000 7f6fddc00700 1 --2- 172.21.15.90:0/2501947716 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fd8102b40 0x7f6fd8102fc0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0009990 tx=0x7f6fc0009ca0 comp rx=0 tx=0).stop 2024-04-24T09:20:58.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.074+0000 7f6fddc00700 1 -- 172.21.15.90:0/2501947716 shutdown_connections 2024-04-24T09:20:58.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.074+0000 7f6fddc00700 1 --2- 172.21.15.90:0/2501947716 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fd8103500 0x7f6fd810f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:58.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.074+0000 7f6fddc00700 1 --2- 172.21.15.90:0/2501947716 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6fd8108b90 0x7f6fd8108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:58.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.074+0000 7f6fddc00700 1 --2- 172.21.15.90:0/2501947716 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fd8102b40 0x7f6fd8102fc0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:58.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.074+0000 7f6fddc00700 1 -- 172.21.15.90:0/2501947716 >> 172.21.15.90:0/2501947716 conn(0x7f6fd8079c50 msgr2=0x7f6fd807a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:20:58.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.074+0000 7f6fddc00700 1 -- 172.21.15.90:0/2501947716 shutdown_connections 2024-04-24T09:20:58.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.074+0000 7f6fddc00700 1 -- 172.21.15.90:0/2501947716 wait complete. 2024-04-24T09:20:58.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.075+0000 7f6fddc00700 1 Processor -- start 2024-04-24T09:20:58.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.075+0000 7f6fddc00700 1 -- start start 2024-04-24T09:20:58.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.075+0000 7f6fddc00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6fd8102b40 0x7f6fd81992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:58.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.075+0000 7f6fddc00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fd8103500 0x7f6fd8199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:58.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.075+0000 7f6fddc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fd8108b90 0x7f6fd819db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:58.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.075+0000 7f6fddc00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd8112e50 con 0x7f6fd8108b90 2024-04-24T09:20:58.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.075+0000 7f6fddc00700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd8112cd0 con 0x7f6fd8102b40 2024-04-24T09:20:58.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.075+0000 7f6fddc00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd8112fd0 con 0x7f6fd8103500 2024-04-24T09:20:58.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.076+0000 7f6fd7400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fd8108b90 0x7f6fd819db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:58.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.076+0000 7f6fd6a00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6fd8102b40 0x7f6fd81992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:58.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.076+0000 7f6fce000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fd8103500 0x7f6fd8199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:58.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.076+0000 7f6fd6a00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6fd8102b40 0x7f6fd81992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.90:38528/0 (socket says 172.21.15.90:38528) 2024-04-24T09:20:58.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.076+0000 7f6fd6a00700 1 -- 172.21.15.90:0/3857936170 learned_addr learned my addr 172.21.15.90:0/3857936170 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:20:58.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.076+0000 7f6fce000700 1 -- 172.21.15.90:0/3857936170 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6fd8102b40 msgr2=0x7f6fd81992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:58.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.076+0000 7f6fce000700 1 --2- 172.21.15.90:0/3857936170 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6fd8102b40 0x7f6fd81992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:58.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.076+0000 7f6fce000700 1 -- 172.21.15.90:0/3857936170 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fd8108b90 msgr2=0x7f6fd819db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:58.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.076+0000 7f6fce000700 1 --2- 172.21.15.90:0/3857936170 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fd8108b90 0x7f6fd819db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:58.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.076+0000 7f6fce000700 1 -- 172.21.15.90:0/3857936170 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fd819e2d0 con 0x7f6fd8103500 2024-04-24T09:20:58.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.076+0000 7f6fd6a00700 1 --2- 172.21.15.90:0/3857936170 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6fd8102b40 0x7f6fd81992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:20:58.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.077+0000 7f6fce000700 1 --2- 172.21.15.90:0/3857936170 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fd8103500 0x7f6fd8199800 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6fc00098e0 tx=0x7f6fc000b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:20:58.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.077+0000 7f6fcfe00700 1 -- 172.21.15.90:0/3857936170 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc0027070 con 0x7f6fd8103500 2024-04-24T09:20:58.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.077+0000 7f6fcfe00700 1 -- 172.21.15.90:0/3857936170 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fc0003b60 con 0x7f6fd8103500 2024-04-24T09:20:58.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.077+0000 7f6fddc00700 1 -- 172.21.15.90:0/3857936170 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fd819e560 con 0x7f6fd8103500 2024-04-24T09:20:58.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.077+0000 7f6fcfe00700 1 -- 172.21.15.90:0/3857936170 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc0022d50 con 0x7f6fd8103500 2024-04-24T09:20:58.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.077+0000 7f6fddc00700 1 -- 172.21.15.90:0/3857936170 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fd81a5e40 con 0x7f6fd8103500 2024-04-24T09:20:58.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.078+0000 7f6fddc00700 1 -- 172.21.15.90:0/3857936170 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fd81041f0 con 0x7f6fd8103500 2024-04-24T09:20:58.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.080+0000 7f6fcfe00700 1 -- 172.21.15.90:0/3857936170 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6fc002a070 con 0x7f6fd8103500 2024-04-24T09:20:58.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.081+0000 7f6fcfe00700 1 --2- 172.21.15.90:0/3857936170 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f6fbc0a3cc0 0x7f6fbc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:58.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.082+0000 7f6fcfe00700 1 -- 172.21.15.90:0/3857936170 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f6fc00d12b0 con 0x7f6fd8103500 2024-04-24T09:20:58.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.082+0000 7f6fd6a00700 1 --2- 172.21.15.90:0/3857936170 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f6fbc0a3cc0 0x7f6fbc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:58.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.082+0000 7f6fd6a00700 1 --2- 172.21.15.90:0/3857936170 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f6fbc0a3cc0 0x7f6fbc0a6180 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8009680 tx=0x7f6fc8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:20:58.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.086+0000 7f6fcfe00700 1 -- 172.21.15.90:0/3857936170 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6fc005eff0 con 0x7f6fd8103500 2024-04-24T09:20:58.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.358+0000 7f6fddc00700 1 -- 172.21.15.90:0/3857936170 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f6fd819a620 con 0x7f6fd8103500 2024-04-24T09:20:58.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.359+0000 7f6fcfe00700 1 -- 172.21.15.90:0/3857936170 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f6fc0013070 con 0x7f6fd8103500 2024-04-24T09:20:58.359 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:20:58.359 INFO:teuthology.orchestra.run.smithi090.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-24T09:20:58.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.362+0000 7f6fddc00700 1 -- 172.21.15.90:0/3857936170 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f6fbc0a3cc0 msgr2=0x7f6fbc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:58.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.362+0000 7f6fddc00700 1 --2- 172.21.15.90:0/3857936170 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f6fbc0a3cc0 0x7f6fbc0a6180 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8009680 tx=0x7f6fc8009260 comp rx=0 tx=0).stop 2024-04-24T09:20:58.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.362+0000 7f6fddc00700 1 -- 172.21.15.90:0/3857936170 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fd8103500 msgr2=0x7f6fd8199800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:58.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.362+0000 7f6fddc00700 1 --2- 172.21.15.90:0/3857936170 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fd8103500 0x7f6fd8199800 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6fc00098e0 tx=0x7f6fc000b8a0 comp rx=0 tx=0).stop 2024-04-24T09:20:58.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.362+0000 7f6fddc00700 1 -- 172.21.15.90:0/3857936170 shutdown_connections 2024-04-24T09:20:58.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.362+0000 7f6fddc00700 1 --2- 172.21.15.90:0/3857936170 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f6fbc0a3cc0 0x7f6fbc0a6180 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:58.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.362+0000 7f6fddc00700 1 --2- 172.21.15.90:0/3857936170 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6fd8108b90 0x7f6fd819db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:58.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.363+0000 7f6fddc00700 1 --2- 172.21.15.90:0/3857936170 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6fd8102b40 0x7f6fd81992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:58.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.363+0000 7f6fddc00700 1 --2- 172.21.15.90:0/3857936170 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fd8103500 0x7f6fd8199800 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:58.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.363+0000 7f6fddc00700 1 -- 172.21.15.90:0/3857936170 >> 172.21.15.90:0/3857936170 conn(0x7f6fd8079c50 msgr2=0x7f6fd81017f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:20:58.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.363+0000 7f6fddc00700 1 -- 172.21.15.90:0/3857936170 shutdown_connections 2024-04-24T09:20:58.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:58.363+0000 7f6fddc00700 1 -- 172.21.15.90:0/3857936170 wait complete. 2024-04-24T09:20:58.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:58 smithi114 ceph-mon[32501]: pgmap v378: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:20:58.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:20:58 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/3857936170' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-24T09:20:59.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:58 smithi090 ceph-mon[27825]: pgmap v378: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:20:59.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:20:58 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/3857936170' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-24T09:20:59.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:58 smithi149 ceph-mon[32592]: pgmap v378: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:20:59.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:20:58 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/3857936170' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-24T09:20:59.269 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-24T09:20:59.269 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T09:20:59.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.478+0000 7f866b000640 1 -- 172.21.15.90:0/1364292025 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f866410a030 msgr2=0x7f86641108c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:59.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.478+0000 7f866b000640 1 --2- 172.21.15.90:0/1364292025 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f866410a030 0x7f86641108c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f865c009ca0 tx=0x7f865c02a8b0 comp rx=0 tx=0).stop 2024-04-24T09:20:59.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.478+0000 7f866b000640 1 -- 172.21.15.90:0/1364292025 shutdown_connections 2024-04-24T09:20:59.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.478+0000 7f866b000640 1 --2- 172.21.15.90:0/1364292025 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f866410a030 0x7f86641108c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:59.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.478+0000 7f866b000640 1 --2- 172.21.15.90:0/1364292025 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8664109670 0x7f8664109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:59.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.478+0000 7f866b000640 1 --2- 172.21.15.90:0/1364292025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8664108480 0x7f86641088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:59.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.478+0000 7f866b000640 1 -- 172.21.15.90:0/1364292025 >> 172.21.15.90:0/1364292025 conn(0x7f8664077440 msgr2=0x7f8664079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:20:59.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.478+0000 7f866b000640 1 -- 172.21.15.90:0/1364292025 shutdown_connections 2024-04-24T09:20:59.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.478+0000 7f866b000640 1 -- 172.21.15.90:0/1364292025 wait complete. 2024-04-24T09:20:59.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.479+0000 7f866b000640 1 Processor -- start 2024-04-24T09:20:59.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.479+0000 7f866b000640 1 -- start start 2024-04-24T09:20:59.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.479+0000 7f866b000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8664108480 0x7f866419e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:59.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.479+0000 7f866b000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8664109670 0x7f866419ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:59.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.479+0000 7f866b000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f866410a030 0x7f86641a5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:59.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.479+0000 7f866b000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f86641141c0 con 0x7f866410a030 2024-04-24T09:20:59.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.479+0000 7f866b000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8664114040 con 0x7f8664109670 2024-04-24T09:20:59.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.479+0000 7f866b000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8664114340 con 0x7f8664108480 2024-04-24T09:20:59.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.480+0000 7f866a600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f866410a030 0x7f86641a5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:59.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.480+0000 7f866a600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f866410a030 0x7f86641a5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45254/0 (socket says 172.21.15.90:45254) 2024-04-24T09:20:59.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.480+0000 7f866a600640 1 -- 172.21.15.90:0/3927752049 learned_addr learned my addr 172.21.15.90:0/3927752049 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:20:59.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.480+0000 7f8669c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8664108480 0x7f866419e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:59.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.480+0000 7f8669200640 1 --2- 172.21.15.90:0/3927752049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8664109670 0x7f866419ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:59.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.480+0000 7f866a600640 1 -- 172.21.15.90:0/3927752049 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8664108480 msgr2=0x7f866419e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:59.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.480+0000 7f866a600640 1 --2- 172.21.15.90:0/3927752049 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8664108480 0x7f866419e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:59.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.480+0000 7f866a600640 1 -- 172.21.15.90:0/3927752049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8664109670 msgr2=0x7f866419ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:59.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.480+0000 7f866a600640 1 --2- 172.21.15.90:0/3927752049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8664109670 0x7f866419ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:59.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.480+0000 7f866a600640 1 -- 172.21.15.90:0/3927752049 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86641a63d0 con 0x7f866410a030 2024-04-24T09:20:59.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.480+0000 7f866a600640 1 --2- 172.21.15.90:0/3927752049 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f866410a030 0x7f86641a5d60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f865c009430 tx=0x7f865c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:20:59.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.481+0000 7f8662000640 1 -- 172.21.15.90:0/3927752049 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865c004040 con 0x7f866410a030 2024-04-24T09:20:59.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.481+0000 7f8662000640 1 -- 172.21.15.90:0/3927752049 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f865c040070 con 0x7f866410a030 2024-04-24T09:20:59.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.481+0000 7f866b000640 1 -- 172.21.15.90:0/3927752049 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86641a6660 con 0x7f866410a030 2024-04-24T09:20:59.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.481+0000 7f8662000640 1 -- 172.21.15.90:0/3927752049 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865c045c90 con 0x7f866410a030 2024-04-24T09:20:59.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.481+0000 7f866b000640 1 -- 172.21.15.90:0/3927752049 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86641a6c30 con 0x7f866410a030 2024-04-24T09:20:59.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.482+0000 7f866b000640 1 -- 172.21.15.90:0/3927752049 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f866410f7a0 con 0x7f866410a030 2024-04-24T09:20:59.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.484+0000 7f8662000640 1 -- 172.21.15.90:0/3927752049 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f865c0041e0 con 0x7f866410a030 2024-04-24T09:20:59.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.485+0000 7f8662000640 1 --2- 172.21.15.90:0/3927752049 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f864c0a3bb0 0x7f864c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:59.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.486+0000 7f8662000640 1 -- 172.21.15.90:0/3927752049 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f865c0f1620 con 0x7f866410a030 2024-04-24T09:20:59.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.486+0000 7f8669c00640 1 --2- 172.21.15.90:0/3927752049 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f864c0a3bb0 0x7f864c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:59.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.486+0000 7f8669c00640 1 --2- 172.21.15.90:0/3927752049 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f864c0a3bb0 0x7f864c0a6070 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f86641094d0 tx=0x7f8658009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:20:59.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.490+0000 7f8662000640 1 -- 172.21.15.90:0/3927752049 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f865c0bdfc0 con 0x7f866410a030 2024-04-24T09:20:59.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.693+0000 7f866b000640 1 -- 172.21.15.90:0/3927752049 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8664062c00 con 0x7f866410a030 2024-04-24T09:20:59.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.696+0000 7f8662000640 1 -- 172.21.15.90:0/3927752049 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22652 (secure 0 0 0) 0x7f865c0bd8c0 con 0x7f866410a030 2024-04-24T09:20:59.696 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:20:59.696 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":88,"fsid":"30ee9be2-021a-11ef-bc93-c7b262605968","created":"2024-04-24T09:09:49.496019+0000","modified":"2024-04-24T09:19:58.609640+0000","last_up_change":"2024-04-24T09:18:10.455632+0000","last_in_change":"2024-04-24T09:17:50.600366+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T09:13:45.548774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T09:19:07.185580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T09:19:08.374818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13a59b9f-b46e-4ec4-90af-bb7a3d5c753b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6803","nonce":2864288066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6805","nonce":2864288066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6809","nonce":2864288066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6807","nonce":2864288066}]},"public_addr":"172.21.15.90:6803/2864288066","cluster_addr":"172.21.15.90:6805/2864288066","heartbeat_back_addr":"172.21.15.90:6809/2864288066","heartbeat_front_addr":"172.21.15.90:6807/2864288066","state":["exists","up"]},{"osd":1,"uuid":"1567ce8f-24e6-42cf-bf64-f6f0c534a9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6801","nonce":344222414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6803","nonce":344222414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6807","nonce":344222414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6805","nonce":344222414}]},"public_addr":"172.21.15.114:6801/344222414","cluster_addr":"172.21.15.114:6803/344222414","heartbeat_back_addr":"172.21.15.114:6807/344222414","heartbeat_front_addr":"172.21.15.114:6805/344222414","state":["exists","up"]},{"osd":2,"uuid":"f861b304-9c7d-4629-8522-37f33a747945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6801","nonce":3257067419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6803","nonce":3257067419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6807","nonce":3257067419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6805","nonce":3257067419}]},"public_addr":"172.21.15.149:6801/3257067419","cluster_addr":"172.21.15.149:6803/3257067419","heartbeat_back_addr":"172.21.15.149:6807/3257067419","heartbeat_front_addr":"172.21.15.149:6805/3257067419","state":["exists","up"]},{"osd":3,"uuid":"9377ab6e-5996-45fb-93a9-475a2e5f17a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6811","nonce":3068524970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6813","nonce":3068524970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6817","nonce":3068524970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6815","nonce":3068524970}]},"public_addr":"172.21.15.90:6811/3068524970","cluster_addr":"172.21.15.90:6813/3068524970","heartbeat_back_addr":"172.21.15.90:6817/3068524970","heartbeat_front_addr":"172.21.15.90:6815/3068524970","state":["exists","up"]},{"osd":4,"uuid":"e2ddd19e-e641-4d61-b342-02b5f12b1c4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6809","nonce":2782015522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6811","nonce":2782015522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6815","nonce":2782015522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6813","nonce":2782015522}]},"public_addr":"172.21.15.114:6809/2782015522","cluster_addr":"172.21.15.114:6811/2782015522","heartbeat_back_addr":"172.21.15.114:6815/2782015522","heartbeat_front_addr":"172.21.15.114:6813/2782015522","state":["exists","up"]},{"osd":5,"uuid":"b6ff9d10-5210-4b46-a192-6dcbf9d1f156","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6809","nonce":4266460479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6811","nonce":4266460479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6815","nonce":4266460479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6813","nonce":4266460479}]},"public_addr":"172.21.15.149:6809/4266460479","cluster_addr":"172.21.15.149:6811/4266460479","heartbeat_back_addr":"172.21.15.149:6815/4266460479","heartbeat_front_addr":"172.21.15.149:6813/4266460479","state":["exists","up"]},{"osd":6,"uuid":"4287b5a1-f953-421c-bf62-5e390bee834d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6819","nonce":1121461276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6821","nonce":1121461276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6825","nonce":1121461276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6823","nonce":1121461276}]},"public_addr":"172.21.15.90:6819/1121461276","cluster_addr":"172.21.15.90:6821/1121461276","heartbeat_back_addr":"172.21.15.90:6825/1121461276","heartbeat_front_addr":"172.21.15.90:6823/1121461276","state":["exists","up"]},{"osd":7,"uuid":"a54e507d-d6ba-4bf2-9a7f-a7b95c3859bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6817","nonce":3222546280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6819","nonce":3222546280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6823","nonce":3222546280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6821","nonce":3222546280}]},"public_addr":"172.21.15.114:6817/3222546280","cluster_addr":"172.21.15.114:6819/3222546280","heartbeat_back_addr":"172.21.15.114:6823/3222546280","heartbeat_front_addr":"172.21.15.114:6821/3222546280","state":["exists","up"]},{"osd":8,"uuid":"9950b010-5865-4109-ae37-582b1dbd2f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6817","nonce":157615149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6819","nonce":157615149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6823","nonce":157615149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6821","nonce":157615149}]},"public_addr":"172.21.15.149:6817/157615149","cluster_addr":"172.21.15.149:6819/157615149","heartbeat_back_addr":"172.21.15.149:6823/157615149","heartbeat_front_addr":"172.21.15.149:6821/157615149","state":["exists","up"]},{"osd":9,"uuid":"31e2635c-e234-421f-9757-592fd673c936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6827","nonce":1307318076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6829","nonce":1307318076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6833","nonce":1307318076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6831","nonce":1307318076}]},"public_addr":"172.21.15.90:6827/1307318076","cluster_addr":"172.21.15.90:6829/1307318076","heartbeat_back_addr":"172.21.15.90:6833/1307318076","heartbeat_front_addr":"172.21.15.90:6831/1307318076","state":["exists","up"]},{"osd":10,"uuid":"46a3b2a6-e6e5-4578-b4bb-f3e998b9bc95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6825","nonce":1719802140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6827","nonce":1719802140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6831","nonce":1719802140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6829","nonce":1719802140}]},"public_addr":"172.21.15.114:6825/1719802140","cluster_addr":"172.21.15.114:6827/1719802140","heartbeat_back_addr":"172.21.15.114:6831/1719802140","heartbeat_front_addr":"172.21.15.114:6829/1719802140","state":["exists","up"]},{"osd":11,"uuid":"ccd4456b-e947-46cf-a779-148ffa289b58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6825","nonce":1737275166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6827","nonce":1737275166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6831","nonce":1737275166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6829","nonce":1737275166}]},"public_addr":"172.21.15.149:6825/1737275166","cluster_addr":"172.21.15.149:6827/1737275166","heartbeat_back_addr":"172.21.15.149:6831/1737275166","heartbeat_front_addr":"172.21.15.149:6829/1737275166","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:12:53.015078+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:13:18.047053+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:13:41.935891+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:14:15.095388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:14:42.512355+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:15:08.113775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:15:43.828550+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:16:10.901909+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:16:36.422962+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:17:13.924446+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:17:42.814401+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:18:08.672793+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/3090852580":"2024-04-25T09:10:57.377655+0000","172.21.15.90:0/687197394":"2024-04-25T09:10:57.377655+0000","172.21.15.90:6801/1809038230":"2024-04-25T09:10:57.377655+0000","172.21.15.90:6800/1809038230":"2024-04-25T09:10:57.377655+0000","172.21.15.90:0/2964149071":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/3617388113":"2024-04-25T09:10:22.599674+0000","172.21.15.90:6801/3089140842":"2024-04-25T09:10:22.599674+0000","172.21.15.90:6800/3089140842":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/2840416149":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/305432710":"2024-04-25T09:10:57.377655+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T09:20:59.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.698+0000 7f866b000640 1 -- 172.21.15.90:0/3927752049 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f864c0a3bb0 msgr2=0x7f864c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:59.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.698+0000 7f866b000640 1 --2- 172.21.15.90:0/3927752049 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f864c0a3bb0 0x7f864c0a6070 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f86641094d0 tx=0x7f8658009110 comp rx=0 tx=0).stop 2024-04-24T09:20:59.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.698+0000 7f866b000640 1 -- 172.21.15.90:0/3927752049 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f866410a030 msgr2=0x7f86641a5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:59.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.698+0000 7f866b000640 1 --2- 172.21.15.90:0/3927752049 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f866410a030 0x7f86641a5d60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f865c009430 tx=0x7f865c003970 comp rx=0 tx=0).stop 2024-04-24T09:20:59.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.698+0000 7f866b000640 1 -- 172.21.15.90:0/3927752049 shutdown_connections 2024-04-24T09:20:59.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.698+0000 7f866b000640 1 --2- 172.21.15.90:0/3927752049 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f864c0a3bb0 0x7f864c0a6070 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:59.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.698+0000 7f866b000640 1 --2- 172.21.15.90:0/3927752049 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f866410a030 0x7f86641a5d60 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:59.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.698+0000 7f866b000640 1 --2- 172.21.15.90:0/3927752049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8664109670 0x7f866419ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:59.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.698+0000 7f866b000640 1 --2- 172.21.15.90:0/3927752049 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8664108480 0x7f866419e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:59.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.698+0000 7f866b000640 1 -- 172.21.15.90:0/3927752049 >> 172.21.15.90:0/3927752049 conn(0x7f8664077440 msgr2=0x7f866410c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:20:59.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.699+0000 7f866b000640 1 -- 172.21.15.90:0/3927752049 shutdown_connections 2024-04-24T09:20:59.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.699+0000 7f866b000640 1 -- 172.21.15.90:0/3927752049 wait complete. 2024-04-24T09:20:59.718 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T09:13:45.548774+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T09:19:07.185580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T09:19:08.374818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T09:20:59.718 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T09:20:59.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.911+0000 7f5816e00640 1 -- 172.21.15.90:0/3815056709 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5810104140 msgr2=0x7f5810104560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:59.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.911+0000 7f5816e00640 1 --2- 172.21.15.90:0/3815056709 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5810104140 0x7f5810104560 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5804009900 tx=0x7f580402a410 comp rx=0 tx=0).stop 2024-04-24T09:20:59.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.911+0000 7f5816e00640 1 -- 172.21.15.90:0/3815056709 shutdown_connections 2024-04-24T09:20:59.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.911+0000 7f5816e00640 1 --2- 172.21.15.90:0/3815056709 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5810105cf0 0x7f581010c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:59.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.911+0000 7f5816e00640 1 --2- 172.21.15.90:0/3815056709 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5810105330 0x7f58101057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:59.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.911+0000 7f5816e00640 1 --2- 172.21.15.90:0/3815056709 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5810104140 0x7f5810104560 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:59.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.911+0000 7f5816e00640 1 -- 172.21.15.90:0/3815056709 >> 172.21.15.90:0/3815056709 conn(0x7f58100ff6b0 msgr2=0x7f5810101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:20:59.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.911+0000 7f5816e00640 1 -- 172.21.15.90:0/3815056709 shutdown_connections 2024-04-24T09:20:59.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.911+0000 7f5816e00640 1 -- 172.21.15.90:0/3815056709 wait complete. 2024-04-24T09:20:59.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.912+0000 7f5816e00640 1 Processor -- start 2024-04-24T09:20:59.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.913+0000 7f5816e00640 1 -- start start 2024-04-24T09:20:59.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.913+0000 7f5816e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5810104140 0x7f581019a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:59.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.913+0000 7f5816e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5810105330 0x7f581019a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:59.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.913+0000 7f5816e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5810105cf0 0x7f58101a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:59.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.913+0000 7f5816e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f581010fe20 con 0x7f5810105330 2024-04-24T09:20:59.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.913+0000 7f5816e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f581010fca0 con 0x7f5810105cf0 2024-04-24T09:20:59.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.913+0000 7f5816e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f581010ffa0 con 0x7f5810104140 2024-04-24T09:20:59.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.913+0000 7f5815000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5810105330 0x7f581019a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:59.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.913+0000 7f5815000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5810105330 0x7f581019a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45276/0 (socket says 172.21.15.90:45276) 2024-04-24T09:20:59.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.913+0000 7f5815000640 1 -- 172.21.15.90:0/1661823421 learned_addr learned my addr 172.21.15.90:0/1661823421 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:20:59.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.913+0000 7f5815a00640 1 --2- 172.21.15.90:0/1661823421 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5810104140 0x7f581019a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:59.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.913+0000 7f5816400640 1 --2- 172.21.15.90:0/1661823421 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5810105cf0 0x7f58101a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:59.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.914+0000 7f5815000640 1 -- 172.21.15.90:0/1661823421 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5810104140 msgr2=0x7f581019a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:59.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.914+0000 7f5815000640 1 --2- 172.21.15.90:0/1661823421 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5810104140 0x7f581019a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:59.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.914+0000 7f5815000640 1 -- 172.21.15.90:0/1661823421 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5810105cf0 msgr2=0x7f58101a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:20:59.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.914+0000 7f5815000640 1 --2- 172.21.15.90:0/1661823421 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5810105cf0 0x7f58101a19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:20:59.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.914+0000 7f5815000640 1 -- 172.21.15.90:0/1661823421 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58101a2040 con 0x7f5810105330 2024-04-24T09:20:59.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.914+0000 7f5815000640 1 --2- 172.21.15.90:0/1661823421 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5810105330 0x7f581019a960 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00e9f0 tx=0x7f57fc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:20:59.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.914+0000 7f580e000640 1 -- 172.21.15.90:0/1661823421 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc00cd80 con 0x7f5810105330 2024-04-24T09:20:59.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.914+0000 7f580e000640 1 -- 172.21.15.90:0/1661823421 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57fc010910 con 0x7f5810105330 2024-04-24T09:20:59.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.914+0000 7f5816e00640 1 -- 172.21.15.90:0/1661823421 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58101a2330 con 0x7f5810105330 2024-04-24T09:20:59.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.914+0000 7f580e000640 1 -- 172.21.15.90:0/1661823421 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc018990 con 0x7f5810105330 2024-04-24T09:20:59.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.914+0000 7f5816e00640 1 -- 172.21.15.90:0/1661823421 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58101a28d0 con 0x7f5810105330 2024-04-24T09:20:59.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.915+0000 7f5816e00640 1 -- 172.21.15.90:0/1661823421 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f581010b400 con 0x7f5810105330 2024-04-24T09:20:59.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.917+0000 7f580e000640 1 -- 172.21.15.90:0/1661823421 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f57fc010ac0 con 0x7f5810105330 2024-04-24T09:20:59.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.919+0000 7f580e000640 1 --2- 172.21.15.90:0/1661823421 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f57f80a3c30 0x7f57f80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:20:59.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.919+0000 7f580e000640 1 -- 172.21.15.90:0/1661823421 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f57fc0d0760 con 0x7f5810105330 2024-04-24T09:20:59.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.919+0000 7f5815a00640 1 --2- 172.21.15.90:0/1661823421 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f57f80a3c30 0x7f57f80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:20:59.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.920+0000 7f5815a00640 1 --2- 172.21.15.90:0/1661823421 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f57f80a3c30 0x7f57f80a60f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f580402aa00 tx=0x7f58040315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:20:59.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:20:59.923+0000 7f580e000640 1 -- 172.21.15.90:0/1661823421 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f57fc09d100 con 0x7f5810105330 2024-04-24T09:21:00.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.124+0000 7f5816e00640 1 -- 172.21.15.90:0/1661823421 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5810062c00 con 0x7f5810105330 2024-04-24T09:21:00.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.126+0000 7f580e000640 1 -- 172.21.15.90:0/1661823421 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f57fc09ca00 con 0x7f5810105330 2024-04-24T09:21:00.125 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 1 2024-04-24T09:21:00.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.128+0000 7f5816e00640 1 -- 172.21.15.90:0/1661823421 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f57f80a3c30 msgr2=0x7f57f80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:00.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.128+0000 7f5816e00640 1 --2- 172.21.15.90:0/1661823421 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f57f80a3c30 0x7f57f80a60f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f580402aa00 tx=0x7f58040315b0 comp rx=0 tx=0).stop 2024-04-24T09:21:00.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.128+0000 7f5816e00640 1 -- 172.21.15.90:0/1661823421 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5810105330 msgr2=0x7f581019a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:00.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.128+0000 7f5816e00640 1 --2- 172.21.15.90:0/1661823421 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5810105330 0x7f581019a960 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00e9f0 tx=0x7f57fc00eeb0 comp rx=0 tx=0).stop 2024-04-24T09:21:00.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.128+0000 7f5816e00640 1 -- 172.21.15.90:0/1661823421 shutdown_connections 2024-04-24T09:21:00.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.128+0000 7f5816e00640 1 --2- 172.21.15.90:0/1661823421 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f57f80a3c30 0x7f57f80a60f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:00.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.128+0000 7f5816e00640 1 --2- 172.21.15.90:0/1661823421 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5810105cf0 0x7f58101a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:00.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.128+0000 7f5816e00640 1 --2- 172.21.15.90:0/1661823421 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5810105330 0x7f581019a960 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:00.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.128+0000 7f5816e00640 1 --2- 172.21.15.90:0/1661823421 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5810104140 0x7f581019a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:00.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.128+0000 7f5816e00640 1 -- 172.21.15.90:0/1661823421 >> 172.21.15.90:0/1661823421 conn(0x7f58100ff6b0 msgr2=0x7f58101085d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:00.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.128+0000 7f5816e00640 1 -- 172.21.15.90:0/1661823421 shutdown_connections 2024-04-24T09:21:00.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.129+0000 7f5816e00640 1 -- 172.21.15.90:0/1661823421 wait complete. 2024-04-24T09:21:00.148 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T09:21:00.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.348+0000 7f05e1600640 1 -- 172.21.15.90:0/90784034 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05dc107d40 msgr2=0x7f05dc10a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:00.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.348+0000 7f05e1600640 1 --2- 172.21.15.90:0/90784034 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05dc107d40 0x7f05dc10a190 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f05d4009ca0 tx=0x7f05d402a8c0 comp rx=0 tx=0).stop 2024-04-24T09:21:00.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.348+0000 7f05e1600640 1 -- 172.21.15.90:0/90784034 shutdown_connections 2024-04-24T09:21:00.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.348+0000 7f05e1600640 1 --2- 172.21.15.90:0/90784034 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05dc107d40 0x7f05dc10a190 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:00.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.348+0000 7f05e1600640 1 --2- 172.21.15.90:0/90784034 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05dc105410 0x7f05dc107800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:00.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.348+0000 7f05e1600640 1 --2- 172.21.15.90:0/90784034 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05dc102ae0 0x7f05dc104ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:00.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.348+0000 7f05e1600640 1 -- 172.21.15.90:0/90784034 >> 172.21.15.90:0/90784034 conn(0x7f05dc0fc4d0 msgr2=0x7f05dc0fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:00.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.348+0000 7f05e1600640 1 -- 172.21.15.90:0/90784034 shutdown_connections 2024-04-24T09:21:00.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.348+0000 7f05e1600640 1 -- 172.21.15.90:0/90784034 wait complete. 2024-04-24T09:21:00.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.349+0000 7f05e1600640 1 Processor -- start 2024-04-24T09:21:00.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.349+0000 7f05e1600640 1 -- start start 2024-04-24T09:21:00.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.349+0000 7f05e1600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05dc102ae0 0x7f05dc198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:00.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.349+0000 7f05e1600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05dc105410 0x7f05dc1988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:00.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.350+0000 7f05e1600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05dc107d40 0x7f05dc19f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:00.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.350+0000 7f05e1600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f05dc10dd40 con 0x7f05dc107d40 2024-04-24T09:21:00.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.350+0000 7f05e1600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f05dc10dbc0 con 0x7f05dc105410 2024-04-24T09:21:00.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.350+0000 7f05e1600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f05dc10dec0 con 0x7f05dc102ae0 2024-04-24T09:21:00.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.350+0000 7f05db400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05dc107d40 0x7f05dc19f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:00.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.350+0000 7f05d3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05dc105410 0x7f05dc1988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:00.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.350+0000 7f05db400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05dc107d40 0x7f05dc19f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45284/0 (socket says 172.21.15.90:45284) 2024-04-24T09:21:00.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.350+0000 7f05db400640 1 -- 172.21.15.90:0/940309355 learned_addr learned my addr 172.21.15.90:0/940309355 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:00.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.350+0000 7f05daa00640 1 --2- 172.21.15.90:0/940309355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05dc102ae0 0x7f05dc198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:00.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.350+0000 7f05d3e00640 1 -- 172.21.15.90:0/940309355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05dc102ae0 msgr2=0x7f05dc198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:00.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.350+0000 7f05d3e00640 1 --2- 172.21.15.90:0/940309355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05dc102ae0 0x7f05dc198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:00.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.350+0000 7f05d3e00640 1 -- 172.21.15.90:0/940309355 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05dc107d40 msgr2=0x7f05dc19f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:00.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.350+0000 7f05d3e00640 1 --2- 172.21.15.90:0/940309355 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05dc107d40 0x7f05dc19f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:00.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.350+0000 7f05d3e00640 1 -- 172.21.15.90:0/940309355 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05dc19ffa0 con 0x7f05dc105410 2024-04-24T09:21:00.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.350+0000 7f05db400640 1 --2- 172.21.15.90:0/940309355 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05dc107d40 0x7f05dc19f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:21:00.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.351+0000 7f05d3e00640 1 --2- 172.21.15.90:0/940309355 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05dc105410 0x7f05dc1988c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f05c400e9c0 tx=0x7f05c400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:00.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.351+0000 7f05d3400640 1 -- 172.21.15.90:0/940309355 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c400ccc0 con 0x7f05dc105410 2024-04-24T09:21:00.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.351+0000 7f05d3400640 1 -- 172.21.15.90:0/940309355 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05c4005020 con 0x7f05dc105410 2024-04-24T09:21:00.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.351+0000 7f05e1600640 1 -- 172.21.15.90:0/940309355 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05dc1a0290 con 0x7f05dc105410 2024-04-24T09:21:00.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.351+0000 7f05d3400640 1 -- 172.21.15.90:0/940309355 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c4004a00 con 0x7f05dc105410 2024-04-24T09:21:00.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.351+0000 7f05e1600640 1 -- 172.21.15.90:0/940309355 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05dc1a0790 con 0x7f05dc105410 2024-04-24T09:21:00.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.352+0000 7f05e1600640 1 -- 172.21.15.90:0/940309355 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05dc0fdfc0 con 0x7f05dc105410 2024-04-24T09:21:00.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.354+0000 7f05d3400640 1 -- 172.21.15.90:0/940309355 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f05c40051d0 con 0x7f05dc105410 2024-04-24T09:21:00.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.356+0000 7f05d3400640 1 --2- 172.21.15.90:0/940309355 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f05c00a3bb0 0x7f05c00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:00.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.356+0000 7f05d3400640 1 -- 172.21.15.90:0/940309355 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f05c40d0530 con 0x7f05dc105410 2024-04-24T09:21:00.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.356+0000 7f05daa00640 1 --2- 172.21.15.90:0/940309355 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f05c00a3bb0 0x7f05c00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:00.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.357+0000 7f05daa00640 1 --2- 172.21.15.90:0/940309355 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f05c00a3bb0 0x7f05c00a6070 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f05cc009bd0 tx=0x7f05cc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:00.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.358+0000 7f05d3400640 1 -- 172.21.15.90:0/940309355 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f05c409ced0 con 0x7f05dc105410 2024-04-24T09:21:00.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.559+0000 7f05e1600640 1 -- 172.21.15.90:0/940309355 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f05dc02d870 con 0x7f05dc105410 2024-04-24T09:21:00.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.560+0000 7f05d3400640 1 -- 172.21.15.90:0/940309355 <== mon.2 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f05c409c7d0 con 0x7f05dc105410 2024-04-24T09:21:00.560 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 32 2024-04-24T09:21:00.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.562+0000 7f05e1600640 1 -- 172.21.15.90:0/940309355 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f05c00a3bb0 msgr2=0x7f05c00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:00.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.562+0000 7f05e1600640 1 --2- 172.21.15.90:0/940309355 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f05c00a3bb0 0x7f05c00a6070 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f05cc009bd0 tx=0x7f05cc005f90 comp rx=0 tx=0).stop 2024-04-24T09:21:00.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.562+0000 7f05e1600640 1 -- 172.21.15.90:0/940309355 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05dc105410 msgr2=0x7f05dc1988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:00.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.562+0000 7f05e1600640 1 --2- 172.21.15.90:0/940309355 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05dc105410 0x7f05dc1988c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f05c400e9c0 tx=0x7f05c400ee80 comp rx=0 tx=0).stop 2024-04-24T09:21:00.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.563+0000 7f05e1600640 1 -- 172.21.15.90:0/940309355 shutdown_connections 2024-04-24T09:21:00.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.563+0000 7f05e1600640 1 --2- 172.21.15.90:0/940309355 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f05c00a3bb0 0x7f05c00a6070 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:00.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.563+0000 7f05e1600640 1 --2- 172.21.15.90:0/940309355 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05dc107d40 0x7f05dc19f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:00.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.563+0000 7f05e1600640 1 --2- 172.21.15.90:0/940309355 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05dc105410 0x7f05dc1988c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:00.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.563+0000 7f05e1600640 1 --2- 172.21.15.90:0/940309355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05dc102ae0 0x7f05dc198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:00.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.563+0000 7f05e1600640 1 -- 172.21.15.90:0/940309355 >> 172.21.15.90:0/940309355 conn(0x7f05dc0fc4d0 msgr2=0x7f05dc103420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:00.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.563+0000 7f05e1600640 1 -- 172.21.15.90:0/940309355 shutdown_connections 2024-04-24T09:21:00.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.563+0000 7f05e1600640 1 -- 172.21.15.90:0/940309355 wait complete. 2024-04-24T09:21:00.583 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T09:21:00.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.770+0000 7f514be00640 1 -- 172.21.15.90:0/3579024825 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5144105450 msgr2=0x7f51441058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:00.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.770+0000 7f514be00640 1 --2- 172.21.15.90:0/3579024825 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5144105450 0x7f51441058d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5130009900 tx=0x7f513002a410 comp rx=0 tx=0).stop 2024-04-24T09:21:00.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.770+0000 7f514be00640 1 -- 172.21.15.90:0/3579024825 shutdown_connections 2024-04-24T09:21:00.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.770+0000 7f514be00640 1 --2- 172.21.15.90:0/3579024825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5144105e10 0x7f514410c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:00.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.770+0000 7f514be00640 1 --2- 172.21.15.90:0/3579024825 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5144105450 0x7f51441058d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:00.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.770+0000 7f514be00640 1 --2- 172.21.15.90:0/3579024825 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5144104260 0x7f5144104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:00.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.770+0000 7f514be00640 1 -- 172.21.15.90:0/3579024825 >> 172.21.15.90:0/3579024825 conn(0x7f51440ff7b0 msgr2=0x7f5144101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:00.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.770+0000 7f514be00640 1 -- 172.21.15.90:0/3579024825 shutdown_connections 2024-04-24T09:21:00.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.770+0000 7f514be00640 1 -- 172.21.15.90:0/3579024825 wait complete. 2024-04-24T09:21:00.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.771+0000 7f514be00640 1 Processor -- start 2024-04-24T09:21:00.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.771+0000 7f514be00640 1 -- start start 2024-04-24T09:21:00.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.771+0000 7f514be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5144104260 0x7f514419a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:00.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.771+0000 7f514be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5144105450 0x7f514419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:00.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.771+0000 7f514be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5144105e10 0x7f51441a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:00.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.771+0000 7f514be00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f514410ff40 con 0x7f5144104260 2024-04-24T09:21:00.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.771+0000 7f514be00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f514410fdc0 con 0x7f5144105450 2024-04-24T09:21:00.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.771+0000 7f514be00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f51441100c0 con 0x7f5144105e10 2024-04-24T09:21:00.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.771+0000 7f5149400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5144104260 0x7f514419a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:00.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.772+0000 7f5149e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5144105e10 0x7f51441a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:00.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.772+0000 7f5149e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5144105e10 0x7f51441a1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:37928/0 (socket says 172.21.15.90:37928) 2024-04-24T09:21:00.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.772+0000 7f5149e00640 1 -- 172.21.15.90:0/1560122966 learned_addr learned my addr 172.21.15.90:0/1560122966 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:00.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.772+0000 7f5148a00640 1 --2- 172.21.15.90:0/1560122966 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5144105450 0x7f514419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:00.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.772+0000 7f5149e00640 1 -- 172.21.15.90:0/1560122966 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5144105450 msgr2=0x7f514419aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:00.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.772+0000 7f5149e00640 1 --2- 172.21.15.90:0/1560122966 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5144105450 0x7f514419aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:00.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.772+0000 7f5149e00640 1 -- 172.21.15.90:0/1560122966 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5144104260 msgr2=0x7f514419a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:00.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.772+0000 7f5149e00640 1 --2- 172.21.15.90:0/1560122966 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5144104260 0x7f514419a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:00.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.772+0000 7f5149e00640 1 -- 172.21.15.90:0/1560122966 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51441a2150 con 0x7f5144105e10 2024-04-24T09:21:00.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.772+0000 7f5149e00640 1 --2- 172.21.15.90:0/1560122966 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5144105e10 0x7f51441a1ae0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f513c00e970 tx=0x7f513c00ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:00.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.772+0000 7f5142000640 1 -- 172.21.15.90:0/1560122966 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513c00ccc0 con 0x7f5144105e10 2024-04-24T09:21:00.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.773+0000 7f514be00640 1 -- 172.21.15.90:0/1560122966 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51441a2440 con 0x7f5144105e10 2024-04-24T09:21:00.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.773+0000 7f5142000640 1 -- 172.21.15.90:0/1560122966 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f513c00ce60 con 0x7f5144105e10 2024-04-24T09:21:00.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.773+0000 7f5142000640 1 -- 172.21.15.90:0/1560122966 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513c010450 con 0x7f5144105e10 2024-04-24T09:21:00.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.773+0000 7f514be00640 1 -- 172.21.15.90:0/1560122966 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51441a29b0 con 0x7f5144105e10 2024-04-24T09:21:00.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.774+0000 7f514be00640 1 -- 172.21.15.90:0/1560122966 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f514410b580 con 0x7f5144105e10 2024-04-24T09:21:00.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.776+0000 7f5142000640 1 -- 172.21.15.90:0/1560122966 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f513c005020 con 0x7f5144105e10 2024-04-24T09:21:00.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.777+0000 7f5142000640 1 --2- 172.21.15.90:0/1560122966 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f51280a3c30 0x7f51280a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:00.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.777+0000 7f5142000640 1 -- 172.21.15.90:0/1560122966 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f513c0d07d0 con 0x7f5144105e10 2024-04-24T09:21:00.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.777+0000 7f5149400640 1 --2- 172.21.15.90:0/1560122966 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f51280a3c30 0x7f51280a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:00.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.778+0000 7f5149400640 1 --2- 172.21.15.90:0/1560122966 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f51280a3c30 0x7f51280a60f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f51441052b0 tx=0x7f5134009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:00.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.781+0000 7f5142000640 1 -- 172.21.15.90:0/1560122966 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f513c09d170 con 0x7f5144105e10 2024-04-24T09:21:00.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:00 smithi090 ceph-mon[27825]: pgmap v379: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-24T09:21:00.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:00 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/3927752049' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:21:00.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:00 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/1661823421' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:21:00.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:00 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/940309355' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:21:00.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:00 smithi114 ceph-mon[32501]: pgmap v379: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-24T09:21:00.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:00 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/3927752049' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:21:00.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:00 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/1661823421' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:21:00.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:00 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/940309355' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:21:00.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.986+0000 7f514be00640 1 -- 172.21.15.90:0/1560122966 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5144062c00 con 0x7f5144105e10 2024-04-24T09:21:00.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.986+0000 7f5142000640 1 -- 172.21.15.90:0/1560122966 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f513c09ca70 con 0x7f5144105e10 2024-04-24T09:21:00.986 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 64 2024-04-24T09:21:00.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.989+0000 7f514be00640 1 -- 172.21.15.90:0/1560122966 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f51280a3c30 msgr2=0x7f51280a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:00.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.989+0000 7f514be00640 1 --2- 172.21.15.90:0/1560122966 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f51280a3c30 0x7f51280a60f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f51441052b0 tx=0x7f5134009500 comp rx=0 tx=0).stop 2024-04-24T09:21:00.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.989+0000 7f514be00640 1 -- 172.21.15.90:0/1560122966 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5144105e10 msgr2=0x7f51441a1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:00.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.989+0000 7f514be00640 1 --2- 172.21.15.90:0/1560122966 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5144105e10 0x7f51441a1ae0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f513c00e970 tx=0x7f513c00ed70 comp rx=0 tx=0).stop 2024-04-24T09:21:00.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.989+0000 7f514be00640 1 -- 172.21.15.90:0/1560122966 shutdown_connections 2024-04-24T09:21:00.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.989+0000 7f514be00640 1 --2- 172.21.15.90:0/1560122966 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f51280a3c30 0x7f51280a60f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:00.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.989+0000 7f514be00640 1 --2- 172.21.15.90:0/1560122966 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5144105e10 0x7f51441a1ae0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:00.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.989+0000 7f514be00640 1 --2- 172.21.15.90:0/1560122966 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5144105450 0x7f514419aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:00.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.989+0000 7f514be00640 1 --2- 172.21.15.90:0/1560122966 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5144104260 0x7f514419a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:00.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.989+0000 7f514be00640 1 -- 172.21.15.90:0/1560122966 >> 172.21.15.90:0/1560122966 conn(0x7f51440ff7b0 msgr2=0x7f5144108750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:00.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.989+0000 7f514be00640 1 -- 172.21.15.90:0/1560122966 shutdown_connections 2024-04-24T09:21:00.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:00.989+0000 7f514be00640 1 -- 172.21.15.90:0/1560122966 wait complete. 2024-04-24T09:21:01.005 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:21:01.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:00 smithi149 ceph-mon[32592]: pgmap v379: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-24T09:21:01.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:00 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/3927752049' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:21:01.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:00 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/1661823421' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:21:01.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:00 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/940309355' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:21:01.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.192+0000 7f7262600640 1 -- 172.21.15.90:0/1593363755 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f725c101f70 msgr2=0x7f725c1023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:01.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.192+0000 7f7262600640 1 --2- 172.21.15.90:0/1593363755 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f725c101f70 0x7f725c1023f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7248009900 tx=0x7f724802a3f0 comp rx=0 tx=0).stop 2024-04-24T09:21:01.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.192+0000 7f7262600640 1 -- 172.21.15.90:0/1593363755 shutdown_connections 2024-04-24T09:21:01.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.192+0000 7f7262600640 1 --2- 172.21.15.90:0/1593363755 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f725c102930 0x7f725c1091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:01.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.192+0000 7f7262600640 1 --2- 172.21.15.90:0/1593363755 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f725c101f70 0x7f725c1023f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:01.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.192+0000 7f7262600640 1 --2- 172.21.15.90:0/1593363755 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f725c100d80 0x7f725c1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:01.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.192+0000 7f7262600640 1 -- 172.21.15.90:0/1593363755 >> 172.21.15.90:0/1593363755 conn(0x7f725c0fc2d0 msgr2=0x7f725c0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:01.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.192+0000 7f7262600640 1 -- 172.21.15.90:0/1593363755 shutdown_connections 2024-04-24T09:21:01.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.192+0000 7f7262600640 1 -- 172.21.15.90:0/1593363755 wait complete. 2024-04-24T09:21:01.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.193+0000 7f7262600640 1 Processor -- start 2024-04-24T09:21:01.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.193+0000 7f7262600640 1 -- start start 2024-04-24T09:21:01.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.194+0000 7f7262600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f725c100d80 0x7f725c195a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:01.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.194+0000 7f7262600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f725c101f70 0x7f725c195f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:01.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.194+0000 7f7262600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f725c102930 0x7f725c1925c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:01.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.194+0000 7f7262600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f725c10ca00 con 0x7f725c101f70 2024-04-24T09:21:01.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.194+0000 7f7262600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f725c10c880 con 0x7f725c100d80 2024-04-24T09:21:01.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.194+0000 7f7262600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f725c10cb80 con 0x7f725c102930 2024-04-24T09:21:01.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.194+0000 7f725be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f725c101f70 0x7f725c195f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:01.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.194+0000 7f725be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f725c101f70 0x7f725c195f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45322/0 (socket says 172.21.15.90:45322) 2024-04-24T09:21:01.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.194+0000 7f725be00640 1 -- 172.21.15.90:0/4159742519 learned_addr learned my addr 172.21.15.90:0/4159742519 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:01.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.194+0000 7f7261200640 1 --2- 172.21.15.90:0/4159742519 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f725c100d80 0x7f725c195a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:01.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.194+0000 7f7261c00640 1 --2- 172.21.15.90:0/4159742519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f725c102930 0x7f725c1925c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:01.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.194+0000 7f725be00640 1 -- 172.21.15.90:0/4159742519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f725c102930 msgr2=0x7f725c1925c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:01.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.194+0000 7f725be00640 1 --2- 172.21.15.90:0/4159742519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f725c102930 0x7f725c1925c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:01.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.194+0000 7f725be00640 1 -- 172.21.15.90:0/4159742519 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f725c100d80 msgr2=0x7f725c195a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:01.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.194+0000 7f725be00640 1 --2- 172.21.15.90:0/4159742519 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f725c100d80 0x7f725c195a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:01.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.194+0000 7f725be00640 1 -- 172.21.15.90:0/4159742519 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f725c192e80 con 0x7f725c101f70 2024-04-24T09:21:01.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.195+0000 7f725be00640 1 --2- 172.21.15.90:0/4159742519 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f725c101f70 0x7f725c195f90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7248009230 tx=0x7f72480042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:01.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.195+0000 7f7259600640 1 -- 172.21.15.90:0/4159742519 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724803f030 con 0x7f725c101f70 2024-04-24T09:21:01.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.195+0000 7f7259600640 1 -- 172.21.15.90:0/4159742519 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7248003ce0 con 0x7f725c101f70 2024-04-24T09:21:01.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.195+0000 7f7262600640 1 -- 172.21.15.90:0/4159742519 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f725c1a68a0 con 0x7f725c101f70 2024-04-24T09:21:01.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.195+0000 7f7259600640 1 -- 172.21.15.90:0/4159742519 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7248035070 con 0x7f725c101f70 2024-04-24T09:21:01.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.195+0000 7f7262600640 1 -- 172.21.15.90:0/4159742519 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f725c1a6c50 con 0x7f725c101f70 2024-04-24T09:21:01.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.196+0000 7f7262600640 1 -- 172.21.15.90:0/4159742519 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f725c1a6f40 con 0x7f725c101f70 2024-04-24T09:21:01.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.198+0000 7f7259600640 1 -- 172.21.15.90:0/4159742519 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7248042030 con 0x7f725c101f70 2024-04-24T09:21:01.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.199+0000 7f7259600640 1 --2- 172.21.15.90:0/4159742519 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f72440a3bb0 0x7f72440a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:01.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.200+0000 7f7259600640 1 -- 172.21.15.90:0/4159742519 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f72480f0850 con 0x7f725c101f70 2024-04-24T09:21:01.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.200+0000 7f7261200640 1 --2- 172.21.15.90:0/4159742519 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f72440a3bb0 0x7f72440a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:01.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.200+0000 7f7261200640 1 --2- 172.21.15.90:0/4159742519 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f72440a3bb0 0x7f72440a6070 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f725c101dd0 tx=0x7f7250005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:01.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.203+0000 7f7259600640 1 -- 172.21.15.90:0/4159742519 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f72480bcbe0 con 0x7f725c101f70 2024-04-24T09:21:01.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.444+0000 7f7262600640 1 -- 172.21.15.90:0/4159742519 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f725c1a71b0 con 0x7f725c101f70 2024-04-24T09:21:01.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.445+0000 7f7259600640 1 -- 172.21.15.90:0/4159742519 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11928 (secure 0 0 0) 0x7f72480c0ee0 con 0x7f725c101f70 2024-04-24T09:21:01.446 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:21:01.446 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/227251251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":227251251},{"type":"v1","addr":"172.21.15.90:6837","nonce":227251251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/1904617971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":1904617971},{"type":"v1","addr":"172.21.15.90:6839","nonce":1904617971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T09:19:09.430875+0000","modified":"2024-04-24T09:19:17.259940+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24547,"mds_2":24545,"mds_3":24526,"mds_4":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6833/219377141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":219377141},{"type":"v1","addr":"172.21.15.114:6833","nonce":219377141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"j","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.90:6835/2396995634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2396995634},{"type":"v1","addr":"172.21.15.90:6835","nonce":2396995634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/779642417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":779642417},{"type":"v1","addr":"172.21.15.149:6833","nonce":779642417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6835/2565340800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":2565340800},{"type":"v1","addr":"172.21.15.114:6835","nonce":2565340800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6837/875324280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":875324280},{"type":"v1","addr":"172.21.15.114:6837","nonce":875324280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/3157375624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3157375624},{"type":"v1","addr":"172.21.15.149:6835","nonce":3157375624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.149:6837/2250310587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2250310587},{"type":"v1","addr":"172.21.15.149:6837","nonce":2250310587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.90:6841/397593398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":397593398},{"type":"v1","addr":"172.21.15.90:6841","nonce":397593398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.149:6839/652734263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":652734263},{"type":"v1","addr":"172.21.15.149:6839","nonce":652734263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6839/1025403710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":1025403710},{"type":"v1","addr":"172.21.15.114:6839","nonce":1025403710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24545,24557,24446,24526,24547]},"id":1}]} 2024-04-24T09:21:01.446 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 27 2024-04-24T09:21:01.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.448+0000 7f7262600640 1 -- 172.21.15.90:0/4159742519 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f72440a3bb0 msgr2=0x7f72440a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:01.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.448+0000 7f7262600640 1 --2- 172.21.15.90:0/4159742519 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f72440a3bb0 0x7f72440a6070 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f725c101dd0 tx=0x7f7250005f90 comp rx=0 tx=0).stop 2024-04-24T09:21:01.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.448+0000 7f7262600640 1 -- 172.21.15.90:0/4159742519 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f725c101f70 msgr2=0x7f725c195f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:01.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.448+0000 7f7262600640 1 --2- 172.21.15.90:0/4159742519 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f725c101f70 0x7f725c195f90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7248009230 tx=0x7f72480042f0 comp rx=0 tx=0).stop 2024-04-24T09:21:01.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.448+0000 7f7262600640 1 -- 172.21.15.90:0/4159742519 shutdown_connections 2024-04-24T09:21:01.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.448+0000 7f7262600640 1 --2- 172.21.15.90:0/4159742519 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f72440a3bb0 0x7f72440a6070 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:01.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.448+0000 7f7262600640 1 --2- 172.21.15.90:0/4159742519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f725c102930 0x7f725c1925c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:01.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.448+0000 7f7262600640 1 --2- 172.21.15.90:0/4159742519 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f725c101f70 0x7f725c195f90 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:01.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.448+0000 7f7262600640 1 --2- 172.21.15.90:0/4159742519 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f725c100d80 0x7f725c195a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:01.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.448+0000 7f7262600640 1 -- 172.21.15.90:0/4159742519 >> 172.21.15.90:0/4159742519 conn(0x7f725c0fc2d0 msgr2=0x7f725c0fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:01.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.448+0000 7f7262600640 1 -- 172.21.15.90:0/4159742519 shutdown_connections 2024-04-24T09:21:01.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.448+0000 7f7262600640 1 -- 172.21.15.90:0/4159742519 wait complete. 2024-04-24T09:21:01.466 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T09:21:01.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.651+0000 7f081b800640 1 -- 172.21.15.90:0/3883701554 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0814105450 msgr2=0x7f08141058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:01.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.651+0000 7f081b800640 1 --2- 172.21.15.90:0/3883701554 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0814105450 0x7f08141058d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0800009900 tx=0x7f080002a3f0 comp rx=0 tx=0).stop 2024-04-24T09:21:01.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.651+0000 7f081b800640 1 -- 172.21.15.90:0/3883701554 shutdown_connections 2024-04-24T09:21:01.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.651+0000 7f081b800640 1 --2- 172.21.15.90:0/3883701554 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0814105e10 0x7f081410c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:01.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.651+0000 7f081b800640 1 --2- 172.21.15.90:0/3883701554 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0814105450 0x7f08141058d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:01.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.651+0000 7f081b800640 1 --2- 172.21.15.90:0/3883701554 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0814104260 0x7f0814104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:01.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.651+0000 7f081b800640 1 -- 172.21.15.90:0/3883701554 >> 172.21.15.90:0/3883701554 conn(0x7f08140ff7f0 msgr2=0x7f0814101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:01.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.651+0000 7f081b800640 1 -- 172.21.15.90:0/3883701554 shutdown_connections 2024-04-24T09:21:01.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.651+0000 7f081b800640 1 -- 172.21.15.90:0/3883701554 wait complete. 2024-04-24T09:21:01.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.652+0000 7f081b800640 1 Processor -- start 2024-04-24T09:21:01.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.653+0000 7f081b800640 1 -- start start 2024-04-24T09:21:01.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.653+0000 7f081b800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0814104260 0x7f081419a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:01.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.653+0000 7f081b800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0814105450 0x7f081419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:01.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.653+0000 7f081b800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0814105e10 0x7f08141a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:01.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.653+0000 7f081b800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f081410ff40 con 0x7f0814104260 2024-04-24T09:21:01.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.653+0000 7f081b800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f081410fdc0 con 0x7f0814105450 2024-04-24T09:21:01.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.653+0000 7f081b800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f08141100c0 con 0x7f0814105e10 2024-04-24T09:21:01.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.653+0000 7f0818e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0814104260 0x7f081419a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:01.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.653+0000 7f0818e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0814104260 0x7f081419a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45342/0 (socket says 172.21.15.90:45342) 2024-04-24T09:21:01.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.653+0000 7f0818e00640 1 -- 172.21.15.90:0/4144291439 learned_addr learned my addr 172.21.15.90:0/4144291439 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:01.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.653+0000 7f0813e00640 1 --2- 172.21.15.90:0/4144291439 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0814105450 0x7f081419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:01.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.653+0000 7f0819800640 1 --2- 172.21.15.90:0/4144291439 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0814105e10 0x7f08141a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:01.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.654+0000 7f0818e00640 1 -- 172.21.15.90:0/4144291439 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0814105e10 msgr2=0x7f08141a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:01.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.654+0000 7f0818e00640 1 --2- 172.21.15.90:0/4144291439 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0814105e10 0x7f08141a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:01.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.654+0000 7f0818e00640 1 -- 172.21.15.90:0/4144291439 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0814105450 msgr2=0x7f081419aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:01.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.654+0000 7f0818e00640 1 --2- 172.21.15.90:0/4144291439 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0814105450 0x7f081419aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:01.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.654+0000 7f0818e00640 1 -- 172.21.15.90:0/4144291439 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08141a2160 con 0x7f0814104260 2024-04-24T09:21:01.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.654+0000 7f0818e00640 1 --2- 172.21.15.90:0/4144291439 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0814104260 0x7f081419a540 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f080800e9c0 tx=0x7f080800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:01.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.654+0000 7f0811600640 1 -- 172.21.15.90:0/4144291439 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080800cde0 con 0x7f0814104260 2024-04-24T09:21:01.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.654+0000 7f0811600640 1 -- 172.21.15.90:0/4144291439 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0808010910 con 0x7f0814104260 2024-04-24T09:21:01.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.654+0000 7f081b800640 1 -- 172.21.15.90:0/4144291439 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08141a2450 con 0x7f0814104260 2024-04-24T09:21:01.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.654+0000 7f0811600640 1 -- 172.21.15.90:0/4144291439 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0808018a00 con 0x7f0814104260 2024-04-24T09:21:01.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.654+0000 7f081b800640 1 -- 172.21.15.90:0/4144291439 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08141a29c0 con 0x7f0814104260 2024-04-24T09:21:01.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.655+0000 7f081b800640 1 -- 172.21.15.90:0/4144291439 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f081410b580 con 0x7f0814104260 2024-04-24T09:21:01.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.658+0000 7f0811600640 1 -- 172.21.15.90:0/4144291439 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f080801f020 con 0x7f0814104260 2024-04-24T09:21:01.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.660+0000 7f0811600640 1 --2- 172.21.15.90:0/4144291439 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f07fc0a3c30 0x7f07fc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:01.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.660+0000 7f0811600640 1 -- 172.21.15.90:0/4144291439 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f08080d0590 con 0x7f0814104260 2024-04-24T09:21:01.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.660+0000 7f0813e00640 1 --2- 172.21.15.90:0/4144291439 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f07fc0a3c30 0x7f07fc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:01.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.661+0000 7f0813e00640 1 --2- 172.21.15.90:0/4144291439 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f07fc0a3c30 0x7f07fc0a60f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0800031440 tx=0x7f08000315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:01.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.666+0000 7f0811600640 1 -- 172.21.15.90:0/4144291439 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f080809cf30 con 0x7f0814104260 2024-04-24T09:21:01.731 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:01 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/1560122966' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:21:01.731 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:01 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/4159742519' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:21:01.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.885+0000 7f081b800640 1 -- 172.21.15.90:0/4144291439 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0814062c00 con 0x7f0814104260 2024-04-24T09:21:01.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.887+0000 7f0811600640 1 -- 172.21.15.90:0/4144291439 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22652 (secure 0 0 0) 0x7f080809c830 con 0x7f0814104260 2024-04-24T09:21:01.887 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:21:01.888 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":88,"fsid":"30ee9be2-021a-11ef-bc93-c7b262605968","created":"2024-04-24T09:09:49.496019+0000","modified":"2024-04-24T09:19:58.609640+0000","last_up_change":"2024-04-24T09:18:10.455632+0000","last_in_change":"2024-04-24T09:17:50.600366+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T09:13:45.548774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T09:19:07.185580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T09:19:08.374818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13a59b9f-b46e-4ec4-90af-bb7a3d5c753b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6803","nonce":2864288066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6805","nonce":2864288066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6809","nonce":2864288066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6807","nonce":2864288066}]},"public_addr":"172.21.15.90:6803/2864288066","cluster_addr":"172.21.15.90:6805/2864288066","heartbeat_back_addr":"172.21.15.90:6809/2864288066","heartbeat_front_addr":"172.21.15.90:6807/2864288066","state":["exists","up"]},{"osd":1,"uuid":"1567ce8f-24e6-42cf-bf64-f6f0c534a9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6801","nonce":344222414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6803","nonce":344222414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6807","nonce":344222414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6805","nonce":344222414}]},"public_addr":"172.21.15.114:6801/344222414","cluster_addr":"172.21.15.114:6803/344222414","heartbeat_back_addr":"172.21.15.114:6807/344222414","heartbeat_front_addr":"172.21.15.114:6805/344222414","state":["exists","up"]},{"osd":2,"uuid":"f861b304-9c7d-4629-8522-37f33a747945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6801","nonce":3257067419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6803","nonce":3257067419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6807","nonce":3257067419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6805","nonce":3257067419}]},"public_addr":"172.21.15.149:6801/3257067419","cluster_addr":"172.21.15.149:6803/3257067419","heartbeat_back_addr":"172.21.15.149:6807/3257067419","heartbeat_front_addr":"172.21.15.149:6805/3257067419","state":["exists","up"]},{"osd":3,"uuid":"9377ab6e-5996-45fb-93a9-475a2e5f17a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6811","nonce":3068524970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6813","nonce":3068524970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6817","nonce":3068524970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6815","nonce":3068524970}]},"public_addr":"172.21.15.90:6811/3068524970","cluster_addr":"172.21.15.90:6813/3068524970","heartbeat_back_addr":"172.21.15.90:6817/3068524970","heartbeat_front_addr":"172.21.15.90:6815/3068524970","state":["exists","up"]},{"osd":4,"uuid":"e2ddd19e-e641-4d61-b342-02b5f12b1c4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6809","nonce":2782015522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6811","nonce":2782015522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6815","nonce":2782015522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6813","nonce":2782015522}]},"public_addr":"172.21.15.114:6809/2782015522","cluster_addr":"172.21.15.114:6811/2782015522","heartbeat_back_addr":"172.21.15.114:6815/2782015522","heartbeat_front_addr":"172.21.15.114:6813/2782015522","state":["exists","up"]},{"osd":5,"uuid":"b6ff9d10-5210-4b46-a192-6dcbf9d1f156","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6809","nonce":4266460479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6811","nonce":4266460479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6815","nonce":4266460479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6813","nonce":4266460479}]},"public_addr":"172.21.15.149:6809/4266460479","cluster_addr":"172.21.15.149:6811/4266460479","heartbeat_back_addr":"172.21.15.149:6815/4266460479","heartbeat_front_addr":"172.21.15.149:6813/4266460479","state":["exists","up"]},{"osd":6,"uuid":"4287b5a1-f953-421c-bf62-5e390bee834d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6819","nonce":1121461276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6821","nonce":1121461276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6825","nonce":1121461276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6823","nonce":1121461276}]},"public_addr":"172.21.15.90:6819/1121461276","cluster_addr":"172.21.15.90:6821/1121461276","heartbeat_back_addr":"172.21.15.90:6825/1121461276","heartbeat_front_addr":"172.21.15.90:6823/1121461276","state":["exists","up"]},{"osd":7,"uuid":"a54e507d-d6ba-4bf2-9a7f-a7b95c3859bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6817","nonce":3222546280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6819","nonce":3222546280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6823","nonce":3222546280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6821","nonce":3222546280}]},"public_addr":"172.21.15.114:6817/3222546280","cluster_addr":"172.21.15.114:6819/3222546280","heartbeat_back_addr":"172.21.15.114:6823/3222546280","heartbeat_front_addr":"172.21.15.114:6821/3222546280","state":["exists","up"]},{"osd":8,"uuid":"9950b010-5865-4109-ae37-582b1dbd2f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6817","nonce":157615149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6819","nonce":157615149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6823","nonce":157615149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6821","nonce":157615149}]},"public_addr":"172.21.15.149:6817/157615149","cluster_addr":"172.21.15.149:6819/157615149","heartbeat_back_addr":"172.21.15.149:6823/157615149","heartbeat_front_addr":"172.21.15.149:6821/157615149","state":["exists","up"]},{"osd":9,"uuid":"31e2635c-e234-421f-9757-592fd673c936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6827","nonce":1307318076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6829","nonce":1307318076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6833","nonce":1307318076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6831","nonce":1307318076}]},"public_addr":"172.21.15.90:6827/1307318076","cluster_addr":"172.21.15.90:6829/1307318076","heartbeat_back_addr":"172.21.15.90:6833/1307318076","heartbeat_front_addr":"172.21.15.90:6831/1307318076","state":["exists","up"]},{"osd":10,"uuid":"46a3b2a6-e6e5-4578-b4bb-f3e998b9bc95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6825","nonce":1719802140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6827","nonce":1719802140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6831","nonce":1719802140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6829","nonce":1719802140}]},"public_addr":"172.21.15.114:6825/1719802140","cluster_addr":"172.21.15.114:6827/1719802140","heartbeat_back_addr":"172.21.15.114:6831/1719802140","heartbeat_front_addr":"172.21.15.114:6829/1719802140","state":["exists","up"]},{"osd":11,"uuid":"ccd4456b-e947-46cf-a779-148ffa289b58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6825","nonce":1737275166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6827","nonce":1737275166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6831","nonce":1737275166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6829","nonce":1737275166}]},"public_addr":"172.21.15.149:6825/1737275166","cluster_addr":"172.21.15.149:6827/1737275166","heartbeat_back_addr":"172.21.15.149:6831/1737275166","heartbeat_front_addr":"172.21.15.149:6829/1737275166","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:12:53.015078+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:13:18.047053+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:13:41.935891+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:14:15.095388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:14:42.512355+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:15:08.113775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:15:43.828550+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:16:10.901909+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:16:36.422962+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:17:13.924446+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:17:42.814401+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:18:08.672793+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/3090852580":"2024-04-25T09:10:57.377655+0000","172.21.15.90:0/687197394":"2024-04-25T09:10:57.377655+0000","172.21.15.90:6801/1809038230":"2024-04-25T09:10:57.377655+0000","172.21.15.90:6800/1809038230":"2024-04-25T09:10:57.377655+0000","172.21.15.90:0/2964149071":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/3617388113":"2024-04-25T09:10:22.599674+0000","172.21.15.90:6801/3089140842":"2024-04-25T09:10:22.599674+0000","172.21.15.90:6800/3089140842":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/2840416149":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/305432710":"2024-04-25T09:10:57.377655+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T09:21:01.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.890+0000 7f081b800640 1 -- 172.21.15.90:0/4144291439 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f07fc0a3c30 msgr2=0x7f07fc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:01.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.890+0000 7f081b800640 1 --2- 172.21.15.90:0/4144291439 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f07fc0a3c30 0x7f07fc0a60f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0800031440 tx=0x7f08000315b0 comp rx=0 tx=0).stop 2024-04-24T09:21:01.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.890+0000 7f081b800640 1 -- 172.21.15.90:0/4144291439 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0814104260 msgr2=0x7f081419a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:01.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.890+0000 7f081b800640 1 --2- 172.21.15.90:0/4144291439 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0814104260 0x7f081419a540 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f080800e9c0 tx=0x7f080800ee80 comp rx=0 tx=0).stop 2024-04-24T09:21:01.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.891+0000 7f081b800640 1 -- 172.21.15.90:0/4144291439 shutdown_connections 2024-04-24T09:21:01.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.891+0000 7f081b800640 1 --2- 172.21.15.90:0/4144291439 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f07fc0a3c30 0x7f07fc0a60f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:01.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.891+0000 7f081b800640 1 --2- 172.21.15.90:0/4144291439 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0814105e10 0x7f08141a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:01.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.891+0000 7f081b800640 1 --2- 172.21.15.90:0/4144291439 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0814105450 0x7f081419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:01.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.891+0000 7f081b800640 1 --2- 172.21.15.90:0/4144291439 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0814104260 0x7f081419a540 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:01.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.891+0000 7f081b800640 1 -- 172.21.15.90:0/4144291439 >> 172.21.15.90:0/4144291439 conn(0x7f08140ff7f0 msgr2=0x7f0814108750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:01.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.891+0000 7f081b800640 1 -- 172.21.15.90:0/4144291439 shutdown_connections 2024-04-24T09:21:01.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:01.891+0000 7f081b800640 1 -- 172.21.15.90:0/4144291439 wait complete. 2024-04-24T09:21:01.909 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T09:13:45.548774+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T09:19:07.185580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T09:19:08.374818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T09:21:01.909 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T09:21:01.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:01 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/1560122966' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:21:01.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:01 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/4159742519' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:21:02.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.097+0000 7f277da00640 1 -- 172.21.15.90:0/4122761469 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2778103a40 msgr2=0x7f277810a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:02.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.097+0000 7f277da00640 1 --2- 172.21.15.90:0/4122761469 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2778103a40 0x7f277810a360 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2770009ca0 tx=0x7f277002a8c0 comp rx=0 tx=0).stop 2024-04-24T09:21:02.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.097+0000 7f277da00640 1 -- 172.21.15.90:0/4122761469 shutdown_connections 2024-04-24T09:21:02.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.097+0000 7f277da00640 1 --2- 172.21.15.90:0/4122761469 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2778103a40 0x7f277810a360 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:02.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.097+0000 7f277da00640 1 --2- 172.21.15.90:0/4122761469 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2778103080 0x7f2778103500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:02.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.097+0000 7f277da00640 1 --2- 172.21.15.90:0/4122761469 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2778101f30 0x7f2778102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:02.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.097+0000 7f277da00640 1 -- 172.21.15.90:0/4122761469 >> 172.21.15.90:0/4122761469 conn(0x7f27780fd4a0 msgr2=0x7f27780ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:02.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.097+0000 7f277da00640 1 -- 172.21.15.90:0/4122761469 shutdown_connections 2024-04-24T09:21:02.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.097+0000 7f277da00640 1 -- 172.21.15.90:0/4122761469 wait complete. 2024-04-24T09:21:02.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.098+0000 7f277da00640 1 Processor -- start 2024-04-24T09:21:02.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.098+0000 7f277da00640 1 -- start start 2024-04-24T09:21:02.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.098+0000 7f277da00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2778101f30 0x7f277819a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:02.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.098+0000 7f277da00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2778103080 0x7f277819a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:02.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.099+0000 7f277da00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2778103a40 0x7f27781a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:02.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.099+0000 7f277da00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f277810dc10 con 0x7f2778101f30 2024-04-24T09:21:02.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.099+0000 7f277da00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f277810da90 con 0x7f2778103080 2024-04-24T09:21:02.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.099+0000 7f277da00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f277810dd90 con 0x7f2778103a40 2024-04-24T09:21:02.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.099+0000 7f2777e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2778101f30 0x7f277819a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:02.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.099+0000 7f2777e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2778101f30 0x7f277819a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45368/0 (socket says 172.21.15.90:45368) 2024-04-24T09:21:02.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.099+0000 7f2777e00640 1 -- 172.21.15.90:0/1681337241 learned_addr learned my addr 172.21.15.90:0/1681337241 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:02.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.099+0000 7f277d000640 1 --2- 172.21.15.90:0/1681337241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2778103a40 0x7f27781a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:02.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.099+0000 7f2777400640 1 --2- 172.21.15.90:0/1681337241 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2778103080 0x7f277819a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:02.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.099+0000 7f2777e00640 1 -- 172.21.15.90:0/1681337241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2778103a40 msgr2=0x7f27781a1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:02.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.099+0000 7f2777e00640 1 --2- 172.21.15.90:0/1681337241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2778103a40 0x7f27781a1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:02.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.099+0000 7f2777e00640 1 -- 172.21.15.90:0/1681337241 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2778103080 msgr2=0x7f277819a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:02.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.099+0000 7f2777e00640 1 --2- 172.21.15.90:0/1681337241 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2778103080 0x7f277819a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:02.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.099+0000 7f2777e00640 1 -- 172.21.15.90:0/1681337241 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27781a2000 con 0x7f2778101f30 2024-04-24T09:21:02.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.100+0000 7f2777e00640 1 --2- 172.21.15.90:0/1681337241 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2778101f30 0x7f277819a3e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f276800e9f0 tx=0x7f276800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:02.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.100+0000 7f2774c00640 1 -- 172.21.15.90:0/1681337241 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276800cd80 con 0x7f2778101f30 2024-04-24T09:21:02.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.100+0000 7f2774c00640 1 -- 172.21.15.90:0/1681337241 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2768010910 con 0x7f2778101f30 2024-04-24T09:21:02.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.100+0000 7f277da00640 1 -- 172.21.15.90:0/1681337241 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27781a22f0 con 0x7f2778101f30 2024-04-24T09:21:02.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.100+0000 7f2774c00640 1 -- 172.21.15.90:0/1681337241 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2768018990 con 0x7f2778101f30 2024-04-24T09:21:02.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.100+0000 7f277da00640 1 -- 172.21.15.90:0/1681337241 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27781a2840 con 0x7f2778101f30 2024-04-24T09:21:02.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.101+0000 7f277da00640 1 -- 172.21.15.90:0/1681337241 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27781a2aa0 con 0x7f2778101f30 2024-04-24T09:21:02.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.103+0000 7f2774c00640 1 -- 172.21.15.90:0/1681337241 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2768010ac0 con 0x7f2778101f30 2024-04-24T09:21:02.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.105+0000 7f2774c00640 1 --2- 172.21.15.90:0/1681337241 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f275c0a3bb0 0x7f275c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:02.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.105+0000 7f2774c00640 1 -- 172.21.15.90:0/1681337241 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f27680d0760 con 0x7f2778101f30 2024-04-24T09:21:02.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.105+0000 7f2777400640 1 --2- 172.21.15.90:0/1681337241 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f275c0a3bb0 0x7f275c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:02.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.106+0000 7f2777400640 1 --2- 172.21.15.90:0/1681337241 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f275c0a3bb0 0x7f275c0a6070 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f277819b8f0 tx=0x7f2764005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:02.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.109+0000 7f2774c00640 1 -- 172.21.15.90:0/1681337241 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f276809cab0 con 0x7f2778101f30 2024-04-24T09:21:02.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:01 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/1560122966' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:21:02.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:01 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/4159742519' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:21:02.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.310+0000 7f277da00640 1 -- 172.21.15.90:0/1681337241 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f27781a2c70 con 0x7f2778101f30 2024-04-24T09:21:02.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.311+0000 7f2774c00640 1 -- 172.21.15.90:0/1681337241 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f27680a0db0 con 0x7f2778101f30 2024-04-24T09:21:02.311 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 1 2024-04-24T09:21:02.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.314+0000 7f277da00640 1 -- 172.21.15.90:0/1681337241 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f275c0a3bb0 msgr2=0x7f275c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:02.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.314+0000 7f277da00640 1 --2- 172.21.15.90:0/1681337241 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f275c0a3bb0 0x7f275c0a6070 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f277819b8f0 tx=0x7f2764005f90 comp rx=0 tx=0).stop 2024-04-24T09:21:02.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.314+0000 7f277da00640 1 -- 172.21.15.90:0/1681337241 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2778101f30 msgr2=0x7f277819a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:02.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.314+0000 7f277da00640 1 --2- 172.21.15.90:0/1681337241 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2778101f30 0x7f277819a3e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f276800e9f0 tx=0x7f276800eeb0 comp rx=0 tx=0).stop 2024-04-24T09:21:02.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.314+0000 7f277da00640 1 -- 172.21.15.90:0/1681337241 shutdown_connections 2024-04-24T09:21:02.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.314+0000 7f277da00640 1 --2- 172.21.15.90:0/1681337241 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f275c0a3bb0 0x7f275c0a6070 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:02.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.314+0000 7f277da00640 1 --2- 172.21.15.90:0/1681337241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2778103a40 0x7f27781a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:02.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.314+0000 7f277da00640 1 --2- 172.21.15.90:0/1681337241 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2778103080 0x7f277819a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:02.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.314+0000 7f277da00640 1 --2- 172.21.15.90:0/1681337241 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2778101f30 0x7f277819a3e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:02.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.314+0000 7f277da00640 1 -- 172.21.15.90:0/1681337241 >> 172.21.15.90:0/1681337241 conn(0x7f27780fd4a0 msgr2=0x7f2778074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:02.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.314+0000 7f277da00640 1 -- 172.21.15.90:0/1681337241 shutdown_connections 2024-04-24T09:21:02.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.314+0000 7f277da00640 1 -- 172.21.15.90:0/1681337241 wait complete. 2024-04-24T09:21:02.332 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T09:21:02.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.535+0000 7fb918600640 1 -- 172.21.15.90:0/1763781619 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb910109850 msgr2=0x7fb910109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:02.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.535+0000 7fb918600640 1 --2- 172.21.15.90:0/1763781619 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb910109850 0x7fb910109cd0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc009860 tx=0x7fb8fc02a340 comp rx=0 tx=0).stop 2024-04-24T09:21:02.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.535+0000 7fb918600640 1 -- 172.21.15.90:0/1763781619 shutdown_connections 2024-04-24T09:21:02.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.535+0000 7fb918600640 1 --2- 172.21.15.90:0/1763781619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb91010a210 0x7fb910110aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:02.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.535+0000 7fb918600640 1 --2- 172.21.15.90:0/1763781619 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb910109850 0x7fb910109cd0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:02.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.535+0000 7fb918600640 1 --2- 172.21.15.90:0/1763781619 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb910108660 0x7fb910108a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:02.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.535+0000 7fb918600640 1 -- 172.21.15.90:0/1763781619 >> 172.21.15.90:0/1763781619 conn(0x7fb910077600 msgr2=0x7fb910079a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:02.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.535+0000 7fb918600640 1 -- 172.21.15.90:0/1763781619 shutdown_connections 2024-04-24T09:21:02.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.535+0000 7fb918600640 1 -- 172.21.15.90:0/1763781619 wait complete. 2024-04-24T09:21:02.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.537+0000 7fb918600640 1 Processor -- start 2024-04-24T09:21:02.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.537+0000 7fb918600640 1 -- start start 2024-04-24T09:21:02.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.537+0000 7fb918600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb910108660 0x7fb91019e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:02.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.537+0000 7fb918600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb910109850 0x7fb91019eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:02.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.537+0000 7fb918600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb91010a210 0x7fb9101a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:02.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.537+0000 7fb918600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb9101142d0 con 0x7fb91010a210 2024-04-24T09:21:02.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.537+0000 7fb918600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb910114150 con 0x7fb910109850 2024-04-24T09:21:02.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.537+0000 7fb918600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb910114450 con 0x7fb910108660 2024-04-24T09:21:02.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.537+0000 7fb916600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb91010a210 0x7fb9101a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:02.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.537+0000 7fb916600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb91010a210 0x7fb9101a5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45378/0 (socket says 172.21.15.90:45378) 2024-04-24T09:21:02.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.537+0000 7fb916600640 1 -- 172.21.15.90:0/2565198844 learned_addr learned my addr 172.21.15.90:0/2565198844 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:02.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.537+0000 7fb915200640 1 --2- 172.21.15.90:0/2565198844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb910109850 0x7fb91019eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:02.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.537+0000 7fb915c00640 1 --2- 172.21.15.90:0/2565198844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb910108660 0x7fb91019e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:02.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.538+0000 7fb915200640 1 -- 172.21.15.90:0/2565198844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb910108660 msgr2=0x7fb91019e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:02.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.538+0000 7fb915200640 1 --2- 172.21.15.90:0/2565198844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb910108660 0x7fb91019e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:02.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.538+0000 7fb915200640 1 -- 172.21.15.90:0/2565198844 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb91010a210 msgr2=0x7fb9101a5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:02.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.538+0000 7fb915200640 1 --2- 172.21.15.90:0/2565198844 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb91010a210 0x7fb9101a5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:02.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.538+0000 7fb915200640 1 -- 172.21.15.90:0/2565198844 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9101a6590 con 0x7fb910109850 2024-04-24T09:21:02.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.538+0000 7fb916600640 1 --2- 172.21.15.90:0/2565198844 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb91010a210 0x7fb9101a5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:21:02.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.538+0000 7fb915200640 1 --2- 172.21.15.90:0/2565198844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb910109850 0x7fb91019eeb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc035040 tx=0x7fb8fc033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:02.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.538+0000 7fb90e000640 1 -- 172.21.15.90:0/2565198844 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc037040 con 0x7fb910109850 2024-04-24T09:21:02.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.539+0000 7fb918600640 1 -- 172.21.15.90:0/2565198844 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9101a6820 con 0x7fb910109850 2024-04-24T09:21:02.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.539+0000 7fb90e000640 1 -- 172.21.15.90:0/2565198844 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8fc030850 con 0x7fb910109850 2024-04-24T09:21:02.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.539+0000 7fb90e000640 1 -- 172.21.15.90:0/2565198844 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc004920 con 0x7fb910109850 2024-04-24T09:21:02.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.539+0000 7fb918600640 1 -- 172.21.15.90:0/2565198844 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9101a6d30 con 0x7fb910109850 2024-04-24T09:21:02.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.540+0000 7fb918600640 1 -- 172.21.15.90:0/2565198844 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb91010f980 con 0x7fb910109850 2024-04-24T09:21:02.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.542+0000 7fb90e000640 1 -- 172.21.15.90:0/2565198844 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb8fc004050 con 0x7fb910109850 2024-04-24T09:21:02.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.543+0000 7fb90e000640 1 --2- 172.21.15.90:0/2565198844 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fb8f80a3c30 0x7fb8f80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:02.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.543+0000 7fb90e000640 1 -- 172.21.15.90:0/2565198844 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fb8fc0f0170 con 0x7fb910109850 2024-04-24T09:21:02.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.544+0000 7fb915c00640 1 --2- 172.21.15.90:0/2565198844 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fb8f80a3c30 0x7fb8f80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:02.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.544+0000 7fb915c00640 1 --2- 172.21.15.90:0/2565198844 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fb8f80a3c30 0x7fb8f80a60f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb9101096b0 tx=0x7fb904009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:02.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.546+0000 7fb90e000640 1 -- 172.21.15.90:0/2565198844 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb8fc0bcb10 con 0x7fb910109850 2024-04-24T09:21:02.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.749+0000 7fb918600640 1 -- 172.21.15.90:0/2565198844 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb91002d870 con 0x7fb910109850 2024-04-24T09:21:02.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.750+0000 7fb90e000640 1 -- 172.21.15.90:0/2565198844 <== mon.2 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7fb8fc0bc410 con 0x7fb910109850 2024-04-24T09:21:02.750 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 32 2024-04-24T09:21:02.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.753+0000 7fb918600640 1 -- 172.21.15.90:0/2565198844 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fb8f80a3c30 msgr2=0x7fb8f80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:02.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.753+0000 7fb918600640 1 --2- 172.21.15.90:0/2565198844 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fb8f80a3c30 0x7fb8f80a60f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb9101096b0 tx=0x7fb904009500 comp rx=0 tx=0).stop 2024-04-24T09:21:02.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.753+0000 7fb918600640 1 -- 172.21.15.90:0/2565198844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb910109850 msgr2=0x7fb91019eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:02.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.753+0000 7fb918600640 1 --2- 172.21.15.90:0/2565198844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb910109850 0x7fb91019eeb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc035040 tx=0x7fb8fc033d90 comp rx=0 tx=0).stop 2024-04-24T09:21:02.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.753+0000 7fb918600640 1 -- 172.21.15.90:0/2565198844 shutdown_connections 2024-04-24T09:21:02.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.753+0000 7fb918600640 1 --2- 172.21.15.90:0/2565198844 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fb8f80a3c30 0x7fb8f80a60f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:02.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.753+0000 7fb918600640 1 --2- 172.21.15.90:0/2565198844 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb91010a210 0x7fb9101a5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:02.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.753+0000 7fb918600640 1 --2- 172.21.15.90:0/2565198844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb910109850 0x7fb91019eeb0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:02.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.753+0000 7fb918600640 1 --2- 172.21.15.90:0/2565198844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb910108660 0x7fb91019e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:02.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.753+0000 7fb918600640 1 -- 172.21.15.90:0/2565198844 >> 172.21.15.90:0/2565198844 conn(0x7fb910077600 msgr2=0x7fb91010cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:02.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.753+0000 7fb918600640 1 -- 172.21.15.90:0/2565198844 shutdown_connections 2024-04-24T09:21:02.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.753+0000 7fb918600640 1 -- 172.21.15.90:0/2565198844 wait complete. 2024-04-24T09:21:02.770 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T09:21:02.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:02 smithi114 ceph-mon[32501]: pgmap v380: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T09:21:02.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:02 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/4144291439' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:21:02.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:02 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/1681337241' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:21:02.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.966+0000 7f75afe00640 1 -- 172.21.15.90:0/1937499640 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75b0103060 msgr2=0x7f75b01034e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:02.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.966+0000 7f75afe00640 1 --2- 172.21.15.90:0/1937499640 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75b0103060 0x7f75b01034e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7598009900 tx=0x7f759802a400 comp rx=0 tx=0).stop 2024-04-24T09:21:02.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.967+0000 7f75afe00640 1 -- 172.21.15.90:0/1937499640 shutdown_connections 2024-04-24T09:21:02.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.967+0000 7f75afe00640 1 --2- 172.21.15.90:0/1937499640 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75b0103a20 0x7f75b010a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:02.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.967+0000 7f75afe00640 1 --2- 172.21.15.90:0/1937499640 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75b0103060 0x7f75b01034e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:02.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.967+0000 7f75afe00640 1 --2- 172.21.15.90:0/1937499640 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f75b0101f10 0x7f75b0102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:02.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.967+0000 7f75afe00640 1 -- 172.21.15.90:0/1937499640 >> 172.21.15.90:0/1937499640 conn(0x7f75b00fd4a0 msgr2=0x7f75b00ff8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:02.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.967+0000 7f75afe00640 1 -- 172.21.15.90:0/1937499640 shutdown_connections 2024-04-24T09:21:02.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.967+0000 7f75afe00640 1 -- 172.21.15.90:0/1937499640 wait complete. 2024-04-24T09:21:02.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.968+0000 7f75afe00640 1 Processor -- start 2024-04-24T09:21:02.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.968+0000 7f75afe00640 1 -- start start 2024-04-24T09:21:02.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.968+0000 7f75afe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f75b0101f10 0x7f75b01982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:02.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.968+0000 7f75afe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75b0103060 0x7f75b01987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:02.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.968+0000 7f75afe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75b0103a20 0x7f75b019f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:02.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.968+0000 7f75afe00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f75b010dbf0 con 0x7f75b0103060 2024-04-24T09:21:02.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.968+0000 7f75afe00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f75b010da70 con 0x7f75b0101f10 2024-04-24T09:21:02.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.968+0000 7f75afe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f75b010dd70 con 0x7f75b0103a20 2024-04-24T09:21:02.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.968+0000 7f75ae000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75b0103060 0x7f75b01987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:02.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.968+0000 7f75ae000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75b0103060 0x7f75b01987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45400/0 (socket says 172.21.15.90:45400) 2024-04-24T09:21:02.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.968+0000 7f75ae000640 1 -- 172.21.15.90:0/3206770712 learned_addr learned my addr 172.21.15.90:0/3206770712 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:02.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.968+0000 7f75af400640 1 --2- 172.21.15.90:0/3206770712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75b0103a20 0x7f75b019f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:02.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.969+0000 7f75aea00640 1 --2- 172.21.15.90:0/3206770712 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f75b0101f10 0x7f75b01982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:02.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.969+0000 7f75ae000640 1 -- 172.21.15.90:0/3206770712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75b0103a20 msgr2=0x7f75b019f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:02.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.969+0000 7f75ae000640 1 --2- 172.21.15.90:0/3206770712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75b0103a20 0x7f75b019f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:02.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.969+0000 7f75ae000640 1 -- 172.21.15.90:0/3206770712 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f75b0101f10 msgr2=0x7f75b01982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:02.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.969+0000 7f75ae000640 1 --2- 172.21.15.90:0/3206770712 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f75b0101f10 0x7f75b01982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:02.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.969+0000 7f75ae000640 1 -- 172.21.15.90:0/3206770712 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b019fec0 con 0x7f75b0103060 2024-04-24T09:21:02.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.969+0000 7f75af400640 1 --2- 172.21.15.90:0/3206770712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75b0103a20 0x7f75b019f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:21:02.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.969+0000 7f75ae000640 1 --2- 172.21.15.90:0/3206770712 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75b0103060 0x7f75b01987e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7598031440 tx=0x7f7598032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:02.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.969+0000 7f75a7400640 1 -- 172.21.15.90:0/3206770712 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759803f030 con 0x7f75b0103060 2024-04-24T09:21:02.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.970+0000 7f75afe00640 1 -- 172.21.15.90:0/3206770712 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75b01a0150 con 0x7f75b0103060 2024-04-24T09:21:02.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.970+0000 7f75a7400640 1 -- 172.21.15.90:0/3206770712 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7598003ce0 con 0x7f75b0103060 2024-04-24T09:21:02.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.970+0000 7f75a7400640 1 -- 172.21.15.90:0/3206770712 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7598035070 con 0x7f75b0103060 2024-04-24T09:21:02.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.970+0000 7f75afe00640 1 -- 172.21.15.90:0/3206770712 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75b01a0640 con 0x7f75b0103060 2024-04-24T09:21:02.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.971+0000 7f75afe00640 1 -- 172.21.15.90:0/3206770712 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75b01091c0 con 0x7f75b0103060 2024-04-24T09:21:02.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.973+0000 7f75a7400640 1 -- 172.21.15.90:0/3206770712 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7598042030 con 0x7f75b0103060 2024-04-24T09:21:02.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.974+0000 7f75a7400640 1 --2- 172.21.15.90:0/3206770712 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f75940a3bb0 0x7f75940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:02.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.974+0000 7f75a7400640 1 -- 172.21.15.90:0/3206770712 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f75980f07a0 con 0x7f75b0103060 2024-04-24T09:21:02.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.975+0000 7f75aea00640 1 --2- 172.21.15.90:0/3206770712 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f75940a3bb0 0x7f75940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:02.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.975+0000 7f75aea00640 1 --2- 172.21.15.90:0/3206770712 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f75940a3bb0 0x7f75940a6070 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f75a0009bd0 tx=0x7f75a0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:02.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:02.978+0000 7f75a7400640 1 -- 172.21.15.90:0/3206770712 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f75980bcaf0 con 0x7f75b0103060 2024-04-24T09:21:03.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:02 smithi090 ceph-mon[27825]: pgmap v380: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T09:21:03.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:02 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/4144291439' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:21:03.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:02 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/1681337241' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:21:03.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:02 smithi149 ceph-mon[32592]: pgmap v380: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T09:21:03.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:02 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/4144291439' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:21:03.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:02 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/1681337241' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:21:03.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.179+0000 7f75afe00640 1 -- 172.21.15.90:0/3206770712 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f75b0062c00 con 0x7f75b0103060 2024-04-24T09:21:03.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.180+0000 7f75a7400640 1 -- 172.21.15.90:0/3206770712 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f75980c0df0 con 0x7f75b0103060 2024-04-24T09:21:03.180 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 64 2024-04-24T09:21:03.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.183+0000 7f75afe00640 1 -- 172.21.15.90:0/3206770712 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f75940a3bb0 msgr2=0x7f75940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:03.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.183+0000 7f75afe00640 1 --2- 172.21.15.90:0/3206770712 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f75940a3bb0 0x7f75940a6070 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f75a0009bd0 tx=0x7f75a0005f90 comp rx=0 tx=0).stop 2024-04-24T09:21:03.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.183+0000 7f75afe00640 1 -- 172.21.15.90:0/3206770712 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75b0103060 msgr2=0x7f75b01987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:03.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.183+0000 7f75afe00640 1 --2- 172.21.15.90:0/3206770712 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75b0103060 0x7f75b01987e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7598031440 tx=0x7f7598032ce0 comp rx=0 tx=0).stop 2024-04-24T09:21:03.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.183+0000 7f75afe00640 1 -- 172.21.15.90:0/3206770712 shutdown_connections 2024-04-24T09:21:03.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.183+0000 7f75afe00640 1 --2- 172.21.15.90:0/3206770712 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f75940a3bb0 0x7f75940a6070 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:03.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.183+0000 7f75afe00640 1 --2- 172.21.15.90:0/3206770712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75b0103a20 0x7f75b019f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:03.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.183+0000 7f75afe00640 1 --2- 172.21.15.90:0/3206770712 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75b0103060 0x7f75b01987e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:03.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.183+0000 7f75afe00640 1 --2- 172.21.15.90:0/3206770712 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f75b0101f10 0x7f75b01982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:03.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.183+0000 7f75afe00640 1 -- 172.21.15.90:0/3206770712 >> 172.21.15.90:0/3206770712 conn(0x7f75b00fd4a0 msgr2=0x7f75b0106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:03.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.183+0000 7f75afe00640 1 -- 172.21.15.90:0/3206770712 shutdown_connections 2024-04-24T09:21:03.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.183+0000 7f75afe00640 1 -- 172.21.15.90:0/3206770712 wait complete. 2024-04-24T09:21:03.201 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:21:03.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.402+0000 7f3863400640 1 -- 172.21.15.90:0/4061207565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f385c105e10 msgr2=0x7f385c10c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:03.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.402+0000 7f3863400640 1 --2- 172.21.15.90:0/4061207565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f385c105e10 0x7f385c10c6a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3854009be0 tx=0x7f385402a6c0 comp rx=0 tx=0).stop 2024-04-24T09:21:03.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.402+0000 7f3863400640 1 -- 172.21.15.90:0/4061207565 shutdown_connections 2024-04-24T09:21:03.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.402+0000 7f3863400640 1 --2- 172.21.15.90:0/4061207565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f385c105e10 0x7f385c10c6a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:03.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.402+0000 7f3863400640 1 --2- 172.21.15.90:0/4061207565 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f385c105450 0x7f385c1058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:03.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.402+0000 7f3863400640 1 --2- 172.21.15.90:0/4061207565 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f385c104260 0x7f385c104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:03.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.402+0000 7f3863400640 1 -- 172.21.15.90:0/4061207565 >> 172.21.15.90:0/4061207565 conn(0x7f385c0ff7f0 msgr2=0x7f385c101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:03.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.402+0000 7f3863400640 1 -- 172.21.15.90:0/4061207565 shutdown_connections 2024-04-24T09:21:03.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.402+0000 7f3863400640 1 -- 172.21.15.90:0/4061207565 wait complete. 2024-04-24T09:21:03.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.403+0000 7f3863400640 1 Processor -- start 2024-04-24T09:21:03.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.403+0000 7f3863400640 1 -- start start 2024-04-24T09:21:03.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.404+0000 7f3863400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f385c104260 0x7f385c19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:03.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.404+0000 7f3863400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f385c105450 0x7f385c19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:03.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.404+0000 7f3863400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f385c105e10 0x7f385c1a1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:03.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.404+0000 7f3863400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f385c10fec0 con 0x7f385c105450 2024-04-24T09:21:03.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.404+0000 7f3863400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f385c10fd40 con 0x7f385c104260 2024-04-24T09:21:03.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.404+0000 7f3863400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f385c110040 con 0x7f385c105e10 2024-04-24T09:21:03.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.404+0000 7f3860a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f385c104260 0x7f385c19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:03.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.404+0000 7f3853e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f385c105450 0x7f385c19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:03.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.404+0000 7f3860a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f385c104260 0x7f385c19a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.90:38724/0 (socket says 172.21.15.90:38724) 2024-04-24T09:21:03.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.404+0000 7f3860a00640 1 -- 172.21.15.90:0/2879931922 learned_addr learned my addr 172.21.15.90:0/2879931922 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:03.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.404+0000 7f3861400640 1 --2- 172.21.15.90:0/2879931922 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f385c105e10 0x7f385c1a1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:03.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.404+0000 7f3860a00640 1 -- 172.21.15.90:0/2879931922 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f385c105e10 msgr2=0x7f385c1a1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:03.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.404+0000 7f3860a00640 1 --2- 172.21.15.90:0/2879931922 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f385c105e10 0x7f385c1a1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:03.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.404+0000 7f3860a00640 1 -- 172.21.15.90:0/2879931922 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f385c105450 msgr2=0x7f385c19aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:03.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.404+0000 7f3860a00640 1 --2- 172.21.15.90:0/2879931922 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f385c105450 0x7f385c19aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:03.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.404+0000 7f3860a00640 1 -- 172.21.15.90:0/2879931922 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f385c1a2120 con 0x7f385c104260 2024-04-24T09:21:03.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.405+0000 7f3860a00640 1 --2- 172.21.15.90:0/2879931922 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f385c104260 0x7f385c19a500 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f384c00e9c0 tx=0x7f384c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:03.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.405+0000 7f385a000640 1 -- 172.21.15.90:0/2879931922 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384c00cd80 con 0x7f385c104260 2024-04-24T09:21:03.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.405+0000 7f385a000640 1 -- 172.21.15.90:0/2879931922 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f384c0035f0 con 0x7f385c104260 2024-04-24T09:21:03.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.405+0000 7f3863400640 1 -- 172.21.15.90:0/2879931922 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f385c1a2410 con 0x7f385c104260 2024-04-24T09:21:03.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.405+0000 7f385a000640 1 -- 172.21.15.90:0/2879931922 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384c010690 con 0x7f385c104260 2024-04-24T09:21:03.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.405+0000 7f3863400640 1 -- 172.21.15.90:0/2879931922 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f385c1a2960 con 0x7f385c104260 2024-04-24T09:21:03.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.406+0000 7f3863400640 1 -- 172.21.15.90:0/2879931922 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f385c10b520 con 0x7f385c104260 2024-04-24T09:21:03.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.408+0000 7f385a000640 1 -- 172.21.15.90:0/2879931922 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f384c0037a0 con 0x7f385c104260 2024-04-24T09:21:03.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.410+0000 7f385a000640 1 --2- 172.21.15.90:0/2879931922 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f38400a3bb0 0x7f38400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:03.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.410+0000 7f385a000640 1 -- 172.21.15.90:0/2879931922 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f384c0d0840 con 0x7f385c104260 2024-04-24T09:21:03.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.410+0000 7f3853e00640 1 --2- 172.21.15.90:0/2879931922 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f38400a3bb0 0x7f38400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:03.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.411+0000 7f3853e00640 1 --2- 172.21.15.90:0/2879931922 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f38400a3bb0 0x7f38400a6070 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f385c19ba10 tx=0x7f3844009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:03.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.414+0000 7f385a000640 1 -- 172.21.15.90:0/2879931922 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f384c09d1e0 con 0x7f385c104260 2024-04-24T09:21:03.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.657+0000 7f3863400640 1 -- 172.21.15.90:0/2879931922 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f385c02d620 con 0x7f385c104260 2024-04-24T09:21:03.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.658+0000 7f385a000640 1 -- 172.21.15.90:0/2879931922 <== mon.2 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11928 (secure 0 0 0) 0x7f384c09cae0 con 0x7f385c104260 2024-04-24T09:21:03.658 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:21:03.658 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/227251251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":227251251},{"type":"v1","addr":"172.21.15.90:6837","nonce":227251251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/1904617971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":1904617971},{"type":"v1","addr":"172.21.15.90:6839","nonce":1904617971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T09:19:09.430875+0000","modified":"2024-04-24T09:19:17.259940+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24547,"mds_2":24545,"mds_3":24526,"mds_4":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6833/219377141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":219377141},{"type":"v1","addr":"172.21.15.114:6833","nonce":219377141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"j","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.90:6835/2396995634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2396995634},{"type":"v1","addr":"172.21.15.90:6835","nonce":2396995634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/779642417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":779642417},{"type":"v1","addr":"172.21.15.149:6833","nonce":779642417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6835/2565340800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":2565340800},{"type":"v1","addr":"172.21.15.114:6835","nonce":2565340800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6837/875324280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":875324280},{"type":"v1","addr":"172.21.15.114:6837","nonce":875324280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/3157375624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3157375624},{"type":"v1","addr":"172.21.15.149:6835","nonce":3157375624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.149:6837/2250310587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2250310587},{"type":"v1","addr":"172.21.15.149:6837","nonce":2250310587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.90:6841/397593398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":397593398},{"type":"v1","addr":"172.21.15.90:6841","nonce":397593398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.149:6839/652734263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":652734263},{"type":"v1","addr":"172.21.15.149:6839","nonce":652734263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6839/1025403710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":1025403710},{"type":"v1","addr":"172.21.15.114:6839","nonce":1025403710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24545,24557,24446,24526,24547]},"id":1}]} 2024-04-24T09:21:03.659 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 27 2024-04-24T09:21:03.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.660+0000 7f3863400640 1 -- 172.21.15.90:0/2879931922 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f38400a3bb0 msgr2=0x7f38400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:03.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.660+0000 7f3863400640 1 --2- 172.21.15.90:0/2879931922 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f38400a3bb0 0x7f38400a6070 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f385c19ba10 tx=0x7f3844009260 comp rx=0 tx=0).stop 2024-04-24T09:21:03.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.660+0000 7f3863400640 1 -- 172.21.15.90:0/2879931922 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f385c104260 msgr2=0x7f385c19a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:03.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.661+0000 7f3863400640 1 --2- 172.21.15.90:0/2879931922 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f385c104260 0x7f385c19a500 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f384c00e9c0 tx=0x7f384c00ee80 comp rx=0 tx=0).stop 2024-04-24T09:21:03.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.661+0000 7f3863400640 1 -- 172.21.15.90:0/2879931922 shutdown_connections 2024-04-24T09:21:03.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.661+0000 7f3863400640 1 --2- 172.21.15.90:0/2879931922 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f38400a3bb0 0x7f38400a6070 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:03.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.661+0000 7f3863400640 1 --2- 172.21.15.90:0/2879931922 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f385c105e10 0x7f385c1a1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:03.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.661+0000 7f3863400640 1 --2- 172.21.15.90:0/2879931922 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f385c105450 0x7f385c19aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:03.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.661+0000 7f3863400640 1 --2- 172.21.15.90:0/2879931922 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f385c104260 0x7f385c19a500 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:03.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.661+0000 7f3863400640 1 -- 172.21.15.90:0/2879931922 >> 172.21.15.90:0/2879931922 conn(0x7f385c0ff7f0 msgr2=0x7f385c1086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:03.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.661+0000 7f3863400640 1 -- 172.21.15.90:0/2879931922 shutdown_connections 2024-04-24T09:21:03.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.661+0000 7f3863400640 1 -- 172.21.15.90:0/2879931922 wait complete. 2024-04-24T09:21:03.682 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T09:21:03.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.879+0000 7fb6e9200640 1 -- 172.21.15.90:0/999812525 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6e4108660 msgr2=0x7fb6e4108a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:03.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.879+0000 7fb6e9200640 1 --2- 172.21.15.90:0/999812525 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6e4108660 0x7fb6e4108a80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4009900 tx=0x7fb6d402a410 comp rx=0 tx=0).stop 2024-04-24T09:21:03.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.879+0000 7fb6e9200640 1 -- 172.21.15.90:0/999812525 shutdown_connections 2024-04-24T09:21:03.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.879+0000 7fb6e9200640 1 --2- 172.21.15.90:0/999812525 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb6e410a210 0x7fb6e4110aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:03.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.879+0000 7fb6e9200640 1 --2- 172.21.15.90:0/999812525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6e4109850 0x7fb6e4109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:03.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.879+0000 7fb6e9200640 1 --2- 172.21.15.90:0/999812525 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6e4108660 0x7fb6e4108a80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:03.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.879+0000 7fb6e9200640 1 -- 172.21.15.90:0/999812525 >> 172.21.15.90:0/999812525 conn(0x7fb6e4077620 msgr2=0x7fb6e4079a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:03.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.879+0000 7fb6e9200640 1 -- 172.21.15.90:0/999812525 shutdown_connections 2024-04-24T09:21:03.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.879+0000 7fb6e9200640 1 -- 172.21.15.90:0/999812525 wait complete. 2024-04-24T09:21:03.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.880+0000 7fb6e9200640 1 Processor -- start 2024-04-24T09:21:03.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.880+0000 7fb6e9200640 1 -- start start 2024-04-24T09:21:03.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.881+0000 7fb6e9200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb6e4108660 0x7fb6e419e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:03.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.881+0000 7fb6e9200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6e4109850 0x7fb6e419eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:03.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.881+0000 7fb6e9200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6e410a210 0x7fb6e41a5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:03.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.881+0000 7fb6e9200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e4114300 con 0x7fb6e410a210 2024-04-24T09:21:03.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.881+0000 7fb6e9200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e4114180 con 0x7fb6e4108660 2024-04-24T09:21:03.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.881+0000 7fb6e9200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e4114480 con 0x7fb6e4109850 2024-04-24T09:21:03.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.881+0000 7fb6e2800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6e410a210 0x7fb6e41a5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:03.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.881+0000 7fb6d9400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6e4109850 0x7fb6e419eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:03.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.881+0000 7fb6e2800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6e410a210 0x7fb6e41a5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45424/0 (socket says 172.21.15.90:45424) 2024-04-24T09:21:03.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.881+0000 7fb6e2800640 1 -- 172.21.15.90:0/3336379529 learned_addr learned my addr 172.21.15.90:0/3336379529 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:03.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.881+0000 7fb6e1e00640 1 --2- 172.21.15.90:0/3336379529 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb6e4108660 0x7fb6e419e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:03.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.882+0000 7fb6e2800640 1 -- 172.21.15.90:0/3336379529 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6e4109850 msgr2=0x7fb6e419eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:03.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.882+0000 7fb6e2800640 1 --2- 172.21.15.90:0/3336379529 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6e4109850 0x7fb6e419eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:03.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.882+0000 7fb6e2800640 1 -- 172.21.15.90:0/3336379529 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb6e4108660 msgr2=0x7fb6e419e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:03.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.882+0000 7fb6e2800640 1 --2- 172.21.15.90:0/3336379529 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb6e4108660 0x7fb6e419e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:03.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.882+0000 7fb6e2800640 1 -- 172.21.15.90:0/3336379529 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6e41a65c0 con 0x7fb6e410a210 2024-04-24T09:21:03.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.882+0000 7fb6e2800640 1 --2- 172.21.15.90:0/3336379529 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6e410a210 0x7fb6e41a5f50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc00efd0 tx=0x7fb6dc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:03.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.882+0000 7fb6db400640 1 -- 172.21.15.90:0/3336379529 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc003b70 con 0x7fb6e410a210 2024-04-24T09:21:03.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.882+0000 7fb6e9200640 1 -- 172.21.15.90:0/3336379529 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6e41a68b0 con 0x7fb6e410a210 2024-04-24T09:21:03.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.882+0000 7fb6db400640 1 -- 172.21.15.90:0/3336379529 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6dc003d10 con 0x7fb6e410a210 2024-04-24T09:21:03.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.882+0000 7fb6db400640 1 -- 172.21.15.90:0/3336379529 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc0109f0 con 0x7fb6e410a210 2024-04-24T09:21:03.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.882+0000 7fb6e9200640 1 -- 172.21.15.90:0/3336379529 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6e41a6db0 con 0x7fb6e410a210 2024-04-24T09:21:03.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.884+0000 7fb6e9200640 1 -- 172.21.15.90:0/3336379529 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6e410fa00 con 0x7fb6e410a210 2024-04-24T09:21:03.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.886+0000 7fb6db400640 1 -- 172.21.15.90:0/3336379529 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb6dc004020 con 0x7fb6e410a210 2024-04-24T09:21:03.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.887+0000 7fb6db400640 1 --2- 172.21.15.90:0/3336379529 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fb6c80a3bb0 0x7fb6c80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:03.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.887+0000 7fb6db400640 1 -- 172.21.15.90:0/3336379529 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fb6dc0d0890 con 0x7fb6e410a210 2024-04-24T09:21:03.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.887+0000 7fb6e1e00640 1 --2- 172.21.15.90:0/3336379529 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fb6c80a3bb0 0x7fb6c80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:03.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.888+0000 7fb6e1e00640 1 --2- 172.21.15.90:0/3336379529 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fb6c80a3bb0 0x7fb6c80a6070 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb6d402aa00 tx=0x7fb6d40315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:03.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:03.891+0000 7fb6db400640 1 -- 172.21.15.90:0/3336379529 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb6dc09d230 con 0x7fb6e410a210 2024-04-24T09:21:03.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:03 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/2565198844' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:21:03.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:03 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/3206770712' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:21:03.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:03 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/2879931922' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:21:04.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.093+0000 7fb6e9200640 1 -- 172.21.15.90:0/3336379529 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb6e4062c00 con 0x7fb6e410a210 2024-04-24T09:21:04.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.096+0000 7fb6db400640 1 -- 172.21.15.90:0/3336379529 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22652 (secure 0 0 0) 0x7fb6dc09cb30 con 0x7fb6e410a210 2024-04-24T09:21:04.097 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:21:04.097 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":88,"fsid":"30ee9be2-021a-11ef-bc93-c7b262605968","created":"2024-04-24T09:09:49.496019+0000","modified":"2024-04-24T09:19:58.609640+0000","last_up_change":"2024-04-24T09:18:10.455632+0000","last_in_change":"2024-04-24T09:17:50.600366+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T09:13:45.548774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T09:19:07.185580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T09:19:08.374818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13a59b9f-b46e-4ec4-90af-bb7a3d5c753b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6803","nonce":2864288066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6805","nonce":2864288066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6809","nonce":2864288066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6807","nonce":2864288066}]},"public_addr":"172.21.15.90:6803/2864288066","cluster_addr":"172.21.15.90:6805/2864288066","heartbeat_back_addr":"172.21.15.90:6809/2864288066","heartbeat_front_addr":"172.21.15.90:6807/2864288066","state":["exists","up"]},{"osd":1,"uuid":"1567ce8f-24e6-42cf-bf64-f6f0c534a9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6801","nonce":344222414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6803","nonce":344222414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6807","nonce":344222414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6805","nonce":344222414}]},"public_addr":"172.21.15.114:6801/344222414","cluster_addr":"172.21.15.114:6803/344222414","heartbeat_back_addr":"172.21.15.114:6807/344222414","heartbeat_front_addr":"172.21.15.114:6805/344222414","state":["exists","up"]},{"osd":2,"uuid":"f861b304-9c7d-4629-8522-37f33a747945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6801","nonce":3257067419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6803","nonce":3257067419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6807","nonce":3257067419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6805","nonce":3257067419}]},"public_addr":"172.21.15.149:6801/3257067419","cluster_addr":"172.21.15.149:6803/3257067419","heartbeat_back_addr":"172.21.15.149:6807/3257067419","heartbeat_front_addr":"172.21.15.149:6805/3257067419","state":["exists","up"]},{"osd":3,"uuid":"9377ab6e-5996-45fb-93a9-475a2e5f17a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6811","nonce":3068524970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6813","nonce":3068524970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6817","nonce":3068524970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6815","nonce":3068524970}]},"public_addr":"172.21.15.90:6811/3068524970","cluster_addr":"172.21.15.90:6813/3068524970","heartbeat_back_addr":"172.21.15.90:6817/3068524970","heartbeat_front_addr":"172.21.15.90:6815/3068524970","state":["exists","up"]},{"osd":4,"uuid":"e2ddd19e-e641-4d61-b342-02b5f12b1c4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6809","nonce":2782015522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6811","nonce":2782015522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6815","nonce":2782015522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6813","nonce":2782015522}]},"public_addr":"172.21.15.114:6809/2782015522","cluster_addr":"172.21.15.114:6811/2782015522","heartbeat_back_addr":"172.21.15.114:6815/2782015522","heartbeat_front_addr":"172.21.15.114:6813/2782015522","state":["exists","up"]},{"osd":5,"uuid":"b6ff9d10-5210-4b46-a192-6dcbf9d1f156","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6809","nonce":4266460479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6811","nonce":4266460479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6815","nonce":4266460479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6813","nonce":4266460479}]},"public_addr":"172.21.15.149:6809/4266460479","cluster_addr":"172.21.15.149:6811/4266460479","heartbeat_back_addr":"172.21.15.149:6815/4266460479","heartbeat_front_addr":"172.21.15.149:6813/4266460479","state":["exists","up"]},{"osd":6,"uuid":"4287b5a1-f953-421c-bf62-5e390bee834d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6819","nonce":1121461276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6821","nonce":1121461276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6825","nonce":1121461276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6823","nonce":1121461276}]},"public_addr":"172.21.15.90:6819/1121461276","cluster_addr":"172.21.15.90:6821/1121461276","heartbeat_back_addr":"172.21.15.90:6825/1121461276","heartbeat_front_addr":"172.21.15.90:6823/1121461276","state":["exists","up"]},{"osd":7,"uuid":"a54e507d-d6ba-4bf2-9a7f-a7b95c3859bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6817","nonce":3222546280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6819","nonce":3222546280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6823","nonce":3222546280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6821","nonce":3222546280}]},"public_addr":"172.21.15.114:6817/3222546280","cluster_addr":"172.21.15.114:6819/3222546280","heartbeat_back_addr":"172.21.15.114:6823/3222546280","heartbeat_front_addr":"172.21.15.114:6821/3222546280","state":["exists","up"]},{"osd":8,"uuid":"9950b010-5865-4109-ae37-582b1dbd2f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6817","nonce":157615149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6819","nonce":157615149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6823","nonce":157615149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6821","nonce":157615149}]},"public_addr":"172.21.15.149:6817/157615149","cluster_addr":"172.21.15.149:6819/157615149","heartbeat_back_addr":"172.21.15.149:6823/157615149","heartbeat_front_addr":"172.21.15.149:6821/157615149","state":["exists","up"]},{"osd":9,"uuid":"31e2635c-e234-421f-9757-592fd673c936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6827","nonce":1307318076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6829","nonce":1307318076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6833","nonce":1307318076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6831","nonce":1307318076}]},"public_addr":"172.21.15.90:6827/1307318076","cluster_addr":"172.21.15.90:6829/1307318076","heartbeat_back_addr":"172.21.15.90:6833/1307318076","heartbeat_front_addr":"172.21.15.90:6831/1307318076","state":["exists","up"]},{"osd":10,"uuid":"46a3b2a6-e6e5-4578-b4bb-f3e998b9bc95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6825","nonce":1719802140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6827","nonce":1719802140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6831","nonce":1719802140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6829","nonce":1719802140}]},"public_addr":"172.21.15.114:6825/1719802140","cluster_addr":"172.21.15.114:6827/1719802140","heartbeat_back_addr":"172.21.15.114:6831/1719802140","heartbeat_front_addr":"172.21.15.114:6829/1719802140","state":["exists","up"]},{"osd":11,"uuid":"ccd4456b-e947-46cf-a779-148ffa289b58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6825","nonce":1737275166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6827","nonce":1737275166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6831","nonce":1737275166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6829","nonce":1737275166}]},"public_addr":"172.21.15.149:6825/1737275166","cluster_addr":"172.21.15.149:6827/1737275166","heartbeat_back_addr":"172.21.15.149:6831/1737275166","heartbeat_front_addr":"172.21.15.149:6829/1737275166","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:12:53.015078+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:13:18.047053+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:13:41.935891+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:14:15.095388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:14:42.512355+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:15:08.113775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:15:43.828550+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:16:10.901909+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:16:36.422962+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:17:13.924446+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:17:42.814401+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:18:08.672793+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/3090852580":"2024-04-25T09:10:57.377655+0000","172.21.15.90:0/687197394":"2024-04-25T09:10:57.377655+0000","172.21.15.90:6801/1809038230":"2024-04-25T09:10:57.377655+0000","172.21.15.90:6800/1809038230":"2024-04-25T09:10:57.377655+0000","172.21.15.90:0/2964149071":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/3617388113":"2024-04-25T09:10:22.599674+0000","172.21.15.90:6801/3089140842":"2024-04-25T09:10:22.599674+0000","172.21.15.90:6800/3089140842":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/2840416149":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/305432710":"2024-04-25T09:10:57.377655+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T09:21:04.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.099+0000 7fb6e9200640 1 -- 172.21.15.90:0/3336379529 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fb6c80a3bb0 msgr2=0x7fb6c80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:04.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.099+0000 7fb6e9200640 1 --2- 172.21.15.90:0/3336379529 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fb6c80a3bb0 0x7fb6c80a6070 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb6d402aa00 tx=0x7fb6d40315b0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.099+0000 7fb6e9200640 1 -- 172.21.15.90:0/3336379529 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6e410a210 msgr2=0x7fb6e41a5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:04.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.099+0000 7fb6e9200640 1 --2- 172.21.15.90:0/3336379529 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6e410a210 0x7fb6e41a5f50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc00efd0 tx=0x7fb6dc00c6a0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.099+0000 7fb6e9200640 1 -- 172.21.15.90:0/3336379529 shutdown_connections 2024-04-24T09:21:04.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.099+0000 7fb6e9200640 1 --2- 172.21.15.90:0/3336379529 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fb6c80a3bb0 0x7fb6c80a6070 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.099+0000 7fb6e9200640 1 --2- 172.21.15.90:0/3336379529 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6e410a210 0x7fb6e41a5f50 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.099+0000 7fb6e9200640 1 --2- 172.21.15.90:0/3336379529 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6e4109850 0x7fb6e419eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.099+0000 7fb6e9200640 1 --2- 172.21.15.90:0/3336379529 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb6e4108660 0x7fb6e419e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.100+0000 7fb6e9200640 1 -- 172.21.15.90:0/3336379529 >> 172.21.15.90:0/3336379529 conn(0x7fb6e4077620 msgr2=0x7fb6e410cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:04.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.100+0000 7fb6e9200640 1 -- 172.21.15.90:0/3336379529 shutdown_connections 2024-04-24T09:21:04.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.100+0000 7fb6e9200640 1 -- 172.21.15.90:0/3336379529 wait complete. 2024-04-24T09:21:04.101 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:03 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/2565198844' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:21:04.101 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:03 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/3206770712' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:21:04.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:03 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/2879931922' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:21:04.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:03 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/2565198844' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:21:04.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:03 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/3206770712' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:21:04.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:03 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/2879931922' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:21:04.119 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:21:04.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.322+0000 7f2aaf200640 1 -- 172.21.15.90:0/517677531 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ab0105270 msgr2=0x7f2ab01056f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:04.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.322+0000 7f2aaf200640 1 --2- 172.21.15.90:0/517677531 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ab0105270 0x7f2ab01056f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2a98009900 tx=0x7f2a9802a3f0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.323+0000 7f2aaf200640 1 -- 172.21.15.90:0/517677531 shutdown_connections 2024-04-24T09:21:04.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.323+0000 7f2aaf200640 1 --2- 172.21.15.90:0/517677531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ab0105c30 0x7f2ab010c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.323+0000 7f2aaf200640 1 --2- 172.21.15.90:0/517677531 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ab0105270 0x7f2ab01056f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.323+0000 7f2aaf200640 1 --2- 172.21.15.90:0/517677531 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ab0104080 0x7f2ab01044a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.323+0000 7f2aaf200640 1 -- 172.21.15.90:0/517677531 >> 172.21.15.90:0/517677531 conn(0x7f2ab00ff610 msgr2=0x7f2ab0101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:04.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.323+0000 7f2aaf200640 1 -- 172.21.15.90:0/517677531 shutdown_connections 2024-04-24T09:21:04.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.323+0000 7f2aaf200640 1 -- 172.21.15.90:0/517677531 wait complete. 2024-04-24T09:21:04.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.324+0000 7f2aaf200640 1 Processor -- start 2024-04-24T09:21:04.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.324+0000 7f2aaf200640 1 -- start start 2024-04-24T09:21:04.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.324+0000 7f2aaf200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ab0104080 0x7f2ab019a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:04.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.324+0000 7f2aaf200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ab0105270 0x7f2ab019a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:04.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.325+0000 7f2aaf200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ab0105c30 0x7f2ab01a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:04.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.325+0000 7f2aaf200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab010fd10 con 0x7f2ab0105c30 2024-04-24T09:21:04.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.325+0000 7f2aaf200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab010fb90 con 0x7f2ab0105270 2024-04-24T09:21:04.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.325+0000 7f2aaf200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab010fe90 con 0x7f2ab0104080 2024-04-24T09:21:04.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.325+0000 7f2aae800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ab0105c30 0x7f2ab01a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:04.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.325+0000 7f2aade00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ab0104080 0x7f2ab019a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:04.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.325+0000 7f2aae800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ab0105c30 0x7f2ab01a1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45442/0 (socket says 172.21.15.90:45442) 2024-04-24T09:21:04.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.325+0000 7f2aad400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ab0105270 0x7f2ab019a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:04.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.325+0000 7f2aae800640 1 -- 172.21.15.90:0/2299423204 learned_addr learned my addr 172.21.15.90:0/2299423204 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:04.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.325+0000 7f2aae800640 1 -- 172.21.15.90:0/2299423204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ab0104080 msgr2=0x7f2ab019a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:04.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.325+0000 7f2aae800640 1 --2- 172.21.15.90:0/2299423204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ab0104080 0x7f2ab019a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.325+0000 7f2aae800640 1 -- 172.21.15.90:0/2299423204 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ab0105270 msgr2=0x7f2ab019a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:04.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.325+0000 7f2aae800640 1 --2- 172.21.15.90:0/2299423204 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ab0105270 0x7f2ab019a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.325+0000 7f2aae800640 1 -- 172.21.15.90:0/2299423204 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ab01a1f70 con 0x7f2ab0105c30 2024-04-24T09:21:04.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.325+0000 7f2aade00640 1 --2- 172.21.15.90:0/2299423204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ab0104080 0x7f2ab019a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:21:04.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.326+0000 7f2aae800640 1 --2- 172.21.15.90:0/2299423204 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ab0105c30 0x7f2ab01a1900 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2aa800d950 tx=0x7f2aa800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:04.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.326+0000 7f2aa6a00640 1 -- 172.21.15.90:0/2299423204 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa8017030 con 0x7f2ab0105c30 2024-04-24T09:21:04.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.326+0000 7f2aa6a00640 1 -- 172.21.15.90:0/2299423204 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2aa8004020 con 0x7f2ab0105c30 2024-04-24T09:21:04.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.326+0000 7f2aaf200640 1 -- 172.21.15.90:0/2299423204 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ab01a2260 con 0x7f2ab0105c30 2024-04-24T09:21:04.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.326+0000 7f2aa6a00640 1 -- 172.21.15.90:0/2299423204 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa8003c00 con 0x7f2ab0105c30 2024-04-24T09:21:04.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.326+0000 7f2aaf200640 1 -- 172.21.15.90:0/2299423204 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ab01a2730 con 0x7f2ab0105c30 2024-04-24T09:21:04.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.327+0000 7f2aaf200640 1 -- 172.21.15.90:0/2299423204 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ab010b3a0 con 0x7f2ab0105c30 2024-04-24T09:21:04.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.330+0000 7f2aa6a00640 1 -- 172.21.15.90:0/2299423204 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2aa801a070 con 0x7f2ab0105c30 2024-04-24T09:21:04.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.332+0000 7f2aa6a00640 1 --2- 172.21.15.90:0/2299423204 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f2a940a3bb0 0x7f2a940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:04.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.332+0000 7f2aa6a00640 1 -- 172.21.15.90:0/2299423204 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f2aa80cff50 con 0x7f2ab0105c30 2024-04-24T09:21:04.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.332+0000 7f2aade00640 1 --2- 172.21.15.90:0/2299423204 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f2a940a3bb0 0x7f2a940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:04.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.333+0000 7f2aade00640 1 --2- 172.21.15.90:0/2299423204 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f2a940a3bb0 0x7f2a940a6070 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2ab01050d0 tx=0x7f2aa0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:04.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.334+0000 7f2aa6a00640 1 -- 172.21.15.90:0/2299423204 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2aa8021070 con 0x7f2ab0105c30 2024-04-24T09:21:04.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.575+0000 7f2aaf200640 1 -- 172.21.15.90:0/2299423204 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ab002d620 con 0x7f2ab0105c30 2024-04-24T09:21:04.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.576+0000 7f2aa6a00640 1 -- 172.21.15.90:0/2299423204 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11928 (secure 0 0 0) 0x7f2aa809c930 con 0x7f2ab0105c30 2024-04-24T09:21:04.577 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:21:04.577 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/227251251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":227251251},{"type":"v1","addr":"172.21.15.90:6837","nonce":227251251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/1904617971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":1904617971},{"type":"v1","addr":"172.21.15.90:6839","nonce":1904617971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T09:19:09.430875+0000","modified":"2024-04-24T09:19:17.259940+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24547,"mds_2":24545,"mds_3":24526,"mds_4":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6833/219377141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":219377141},{"type":"v1","addr":"172.21.15.114:6833","nonce":219377141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"j","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.90:6835/2396995634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2396995634},{"type":"v1","addr":"172.21.15.90:6835","nonce":2396995634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/779642417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":779642417},{"type":"v1","addr":"172.21.15.149:6833","nonce":779642417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6835/2565340800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":2565340800},{"type":"v1","addr":"172.21.15.114:6835","nonce":2565340800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6837/875324280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":875324280},{"type":"v1","addr":"172.21.15.114:6837","nonce":875324280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/3157375624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3157375624},{"type":"v1","addr":"172.21.15.149:6835","nonce":3157375624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.149:6837/2250310587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2250310587},{"type":"v1","addr":"172.21.15.149:6837","nonce":2250310587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.90:6841/397593398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":397593398},{"type":"v1","addr":"172.21.15.90:6841","nonce":397593398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.149:6839/652734263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":652734263},{"type":"v1","addr":"172.21.15.149:6839","nonce":652734263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6839/1025403710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":1025403710},{"type":"v1","addr":"172.21.15.114:6839","nonce":1025403710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24545,24557,24446,24526,24547]},"id":1}]} 2024-04-24T09:21:04.577 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 27 2024-04-24T09:21:04.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.579+0000 7f2aaf200640 1 -- 172.21.15.90:0/2299423204 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f2a940a3bb0 msgr2=0x7f2a940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:04.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.579+0000 7f2aaf200640 1 --2- 172.21.15.90:0/2299423204 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f2a940a3bb0 0x7f2a940a6070 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2ab01050d0 tx=0x7f2aa0009500 comp rx=0 tx=0).stop 2024-04-24T09:21:04.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.579+0000 7f2aaf200640 1 -- 172.21.15.90:0/2299423204 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ab0105c30 msgr2=0x7f2ab01a1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:04.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.579+0000 7f2aaf200640 1 --2- 172.21.15.90:0/2299423204 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ab0105c30 0x7f2ab01a1900 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2aa800d950 tx=0x7f2aa800de10 comp rx=0 tx=0).stop 2024-04-24T09:21:04.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.579+0000 7f2aaf200640 1 -- 172.21.15.90:0/2299423204 shutdown_connections 2024-04-24T09:21:04.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.579+0000 7f2aaf200640 1 --2- 172.21.15.90:0/2299423204 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f2a940a3bb0 0x7f2a940a6070 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.579+0000 7f2aaf200640 1 --2- 172.21.15.90:0/2299423204 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ab0105c30 0x7f2ab01a1900 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.579+0000 7f2aaf200640 1 --2- 172.21.15.90:0/2299423204 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2ab0105270 0x7f2ab019a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.579+0000 7f2aaf200640 1 --2- 172.21.15.90:0/2299423204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2ab0104080 0x7f2ab019a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.579+0000 7f2aaf200640 1 -- 172.21.15.90:0/2299423204 >> 172.21.15.90:0/2299423204 conn(0x7f2ab00ff610 msgr2=0x7f2ab0108570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:04.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.579+0000 7f2aaf200640 1 -- 172.21.15.90:0/2299423204 shutdown_connections 2024-04-24T09:21:04.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.579+0000 7f2aaf200640 1 -- 172.21.15.90:0/2299423204 wait complete. 2024-04-24T09:21:04.598 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-24T09:19:09.430875+0000', 'modified': '2024-04-24T09:19:17.259940+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24557, 'mds_1': 24547, 'mds_2': 24545, 'mds_3': 24526, 'mds_4': 24446}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24428': {'gid': 24428, 'name': 'k', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.114:6833/219377141', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6832', 'nonce': 219377141}, {'type': 'v1', 'addr': '172.21.15.114:6833', 'nonce': 219377141}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24446': {'gid': 24446, 'name': 'j', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.90:6835/2396995634', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6834', 'nonce': 2396995634}, {'type': 'v1', 'addr': '172.21.15.90:6835', 'nonce': 2396995634}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.149:6833/779642417', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6832', 'nonce': 779642417}, {'type': 'v1', 'addr': '172.21.15.149:6833', 'nonce': 779642417}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24476': {'gid': 24476, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.114:6835/2565340800', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6834', 'nonce': 2565340800}, {'type': 'v1', 'addr': '172.21.15.114:6835', 'nonce': 2565340800}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24500': {'gid': 24500, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.114:6837/875324280', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6836', 'nonce': 875324280}, {'type': 'v1', 'addr': '172.21.15.114:6837', 'nonce': 875324280}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24514': {'gid': 24514, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.149:6835/3157375624', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6834', 'nonce': 3157375624}, {'type': 'v1', 'addr': '172.21.15.149:6835', 'nonce': 3157375624}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24526': {'gid': 24526, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.149:6837/2250310587', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6836', 'nonce': 2250310587}, {'type': 'v1', 'addr': '172.21.15.149:6837', 'nonce': 2250310587}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24545': {'gid': 24545, 'name': 'a', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.90:6841/397593398', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6840', 'nonce': 397593398}, {'type': 'v1', 'addr': '172.21.15.90:6841', 'nonce': 397593398}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24547': {'gid': 24547, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.149:6839/652734263', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6838', 'nonce': 652734263}, {'type': 'v1', 'addr': '172.21.15.149:6839', 'nonce': 652734263}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24557': {'gid': 24557, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.114:6839/1025403710', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6838', 'nonce': 1025403710}, {'type': 'v1', 'addr': '172.21.15.114:6839', 'nonce': 1025403710}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24557, 'qdb_cluster': [24545, 24557, 24446, 24526, 24547]} 2024-04-24T09:21:04.598 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-24T09:21:04.599 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.j status 2024-04-24T09:21:04.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.798+0000 7f32f7e00640 1 -- 172.21.15.90:0/2162924851 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32f8108540 msgr2=0x7f32f8108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:04.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.798+0000 7f32f7e00640 1 --2- 172.21.15.90:0/2162924851 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32f8108540 0x7f32f8108960 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f32e8009900 tx=0x7f32e802a3f0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.798+0000 7f32f7e00640 1 -- 172.21.15.90:0/2162924851 shutdown_connections 2024-04-24T09:21:04.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.798+0000 7f32f7e00640 1 --2- 172.21.15.90:0/2162924851 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32f810a0f0 0x7f32f8110980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.798+0000 7f32f7e00640 1 --2- 172.21.15.90:0/2162924851 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f32f8109730 0x7f32f8109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.798+0000 7f32f7e00640 1 --2- 172.21.15.90:0/2162924851 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32f8108540 0x7f32f8108960 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.798+0000 7f32f7e00640 1 -- 172.21.15.90:0/2162924851 >> 172.21.15.90:0/2162924851 conn(0x7f32f80774d0 msgr2=0x7f32f8079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:04.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.798+0000 7f32f7e00640 1 -- 172.21.15.90:0/2162924851 shutdown_connections 2024-04-24T09:21:04.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.798+0000 7f32f7e00640 1 -- 172.21.15.90:0/2162924851 wait complete. 2024-04-24T09:21:04.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.799+0000 7f32f7e00640 1 Processor -- start 2024-04-24T09:21:04.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.799+0000 7f32f7e00640 1 -- start start 2024-04-24T09:21:04.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.800+0000 7f32f7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32f8108540 0x7f32f819e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:04.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.800+0000 7f32f7e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32f8109730 0x7f32f819ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:04.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.800+0000 7f32f7e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f32f810a0f0 0x7f32f81a5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:04.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.800+0000 7f32f7e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f32f81141d0 con 0x7f32f8109730 2024-04-24T09:21:04.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.800+0000 7f32f7e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f32f8114050 con 0x7f32f810a0f0 2024-04-24T09:21:04.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.800+0000 7f32f7e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f32f8114350 con 0x7f32f8108540 2024-04-24T09:21:04.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.800+0000 7f32efe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32f8109730 0x7f32f819ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:04.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.800+0000 7f32f6a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32f8108540 0x7f32f819e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:04.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.800+0000 7f32efe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32f8109730 0x7f32f819ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45460/0 (socket says 172.21.15.90:45460) 2024-04-24T09:21:04.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.800+0000 7f32efe00640 1 -- 172.21.15.90:0/1410216587 learned_addr learned my addr 172.21.15.90:0/1410216587 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:04.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.800+0000 7f32f7400640 1 --2- 172.21.15.90:0/1410216587 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f32f810a0f0 0x7f32f81a5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:04.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.800+0000 7f32efe00640 1 -- 172.21.15.90:0/1410216587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32f8108540 msgr2=0x7f32f819e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:04.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.800+0000 7f32efe00640 1 --2- 172.21.15.90:0/1410216587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32f8108540 0x7f32f819e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.800+0000 7f32efe00640 1 -- 172.21.15.90:0/1410216587 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f32f810a0f0 msgr2=0x7f32f81a5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:04.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.800+0000 7f32efe00640 1 --2- 172.21.15.90:0/1410216587 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f32f810a0f0 0x7f32f81a5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.800+0000 7f32efe00640 1 -- 172.21.15.90:0/1410216587 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32f81a6420 con 0x7f32f8109730 2024-04-24T09:21:04.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.801+0000 7f32efe00640 1 --2- 172.21.15.90:0/1410216587 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32f8109730 0x7f32f819ed40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f32e000e9f0 tx=0x7f32e000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:04.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.801+0000 7f32fca00640 1 -- 172.21.15.90:0/1410216587 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e000cd80 con 0x7f32f8109730 2024-04-24T09:21:04.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.801+0000 7f32f7e00640 1 -- 172.21.15.90:0/1410216587 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32f81a6710 con 0x7f32f8109730 2024-04-24T09:21:04.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.801+0000 7f32fca00640 1 -- 172.21.15.90:0/1410216587 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32e0010910 con 0x7f32f8109730 2024-04-24T09:21:04.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.801+0000 7f32f7e00640 1 -- 172.21.15.90:0/1410216587 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32f81a6cb0 con 0x7f32f8109730 2024-04-24T09:21:04.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.801+0000 7f32fca00640 1 -- 172.21.15.90:0/1410216587 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e0018910 con 0x7f32f8109730 2024-04-24T09:21:04.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.804+0000 7f32fca00640 1 -- 172.21.15.90:0/1410216587 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f32e0010ac0 con 0x7f32f8109730 2024-04-24T09:21:04.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.804+0000 7f3300d9b740 1 Processor -- start 2024-04-24T09:21:04.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.804+0000 7f3300d9b740 1 -- start start 2024-04-24T09:21:04.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.805+0000 7f3300d9b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5596313f6260 0x5596313f6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:04.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.805+0000 7f3300d9b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596313f6c50 0x5596313fb750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:04.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.805+0000 7f3300d9b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596313fbdc0 0x5596313fe250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:04.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.805+0000 7f3300d9b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55963135bb10 con 0x5596313fbdc0 2024-04-24T09:21:04.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.805+0000 7f3300d9b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5596313022b0 con 0x5596313f6c50 2024-04-24T09:21:04.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.805+0000 7f3300d9b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559631357ad0 con 0x5596313f6260 2024-04-24T09:21:04.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.805+0000 7f32efe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596313fbdc0 0x5596313fe250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:04.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.805+0000 7f32f6a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596313f6c50 0x5596313fb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:04.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.805+0000 7f32efe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596313fbdc0 0x5596313fe250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45468/0 (socket says 172.21.15.90:45468) 2024-04-24T09:21:04.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.805+0000 7f32f7400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5596313f6260 0x5596313f6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:04.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.805+0000 7f32efe00640 1 -- 172.21.15.90:0/1327374209 learned_addr learned my addr 172.21.15.90:0/1327374209 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:04.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.805+0000 7f32f6a00640 1 -- 172.21.15.90:0/1327374209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5596313f6260 msgr2=0x5596313f6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:04.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.805+0000 7f32f6a00640 1 --2- 172.21.15.90:0/1327374209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5596313f6260 0x5596313f6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.805+0000 7f32f6a00640 1 -- 172.21.15.90:0/1327374209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596313fbdc0 msgr2=0x5596313fe250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:04.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.805+0000 7f32f6a00640 1 --2- 172.21.15.90:0/1327374209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596313fbdc0 0x5596313fe250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.806+0000 7f32f6a00640 1 -- 172.21.15.90:0/1327374209 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55963106e2d0 con 0x5596313f6c50 2024-04-24T09:21:04.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.806+0000 7f32f6a00640 1 --2- 172.21.15.90:0/1327374209 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596313f6c50 0x5596313fb750 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f32e8009230 tx=0x7f32e8005e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b48657fc6dde6ada server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:04.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.806+0000 7f32fca00640 1 --2- 172.21.15.90:0/1410216587 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f32dc0a3bb0 0x7f32dc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:04.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.806+0000 7f32e7e00640 1 -- 172.21.15.90:0/1327374209 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e8040070 con 0x5596313f6c50 2024-04-24T09:21:04.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.806+0000 7f32fca00640 1 -- 172.21.15.90:0/1410216587 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f32e00d0760 con 0x7f32f8109730 2024-04-24T09:21:04.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.806+0000 7f32f7400640 1 --2- 172.21.15.90:0/1410216587 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f32dc0a3bb0 0x7f32dc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:04.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.806+0000 7f32e7e00640 1 -- 172.21.15.90:0/1327374209 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32e8004030 con 0x5596313f6c50 2024-04-24T09:21:04.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.806+0000 7f32e7e00640 1 -- 172.21.15.90:0/1327374209 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e8035040 con 0x5596313f6c50 2024-04-24T09:21:04.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.807+0000 7f3300d9b740 1 -- 172.21.15.90:0/1327374209 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596313f6c50 msgr2=0x5596313fb750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:04.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.807+0000 7f32f7400640 1 --2- 172.21.15.90:0/1410216587 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f32dc0a3bb0 0x7f32dc0a6070 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x5596313f74b0 tx=0x7f32f0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:04.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.807+0000 7f3300d9b740 1 --2- 172.21.15.90:0/1327374209 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596313f6c50 0x5596313fb750 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f32e8009230 tx=0x7f32e8005e20 comp rx=0 tx=0).stop 2024-04-24T09:21:04.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.807+0000 7f3300d9b740 1 -- 172.21.15.90:0/1327374209 shutdown_connections 2024-04-24T09:21:04.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.807+0000 7f3300d9b740 1 --2- 172.21.15.90:0/1327374209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596313fbdc0 0x5596313fe250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.807+0000 7f3300d9b740 1 --2- 172.21.15.90:0/1327374209 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596313f6c50 0x5596313fb750 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.807+0000 7f3300d9b740 1 --2- 172.21.15.90:0/1327374209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5596313f6260 0x5596313f6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.807+0000 7f3300d9b740 1 -- 172.21.15.90:0/1327374209 >> 172.21.15.90:0/1327374209 conn(0x5596313f18d0 msgr2=0x5596313f1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:04.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.807+0000 7f3300d9b740 1 -- 172.21.15.90:0/1327374209 shutdown_connections 2024-04-24T09:21:04.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.807+0000 7f3300d9b740 1 -- 172.21.15.90:0/1327374209 wait complete. 2024-04-24T09:21:04.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.808+0000 7f3300d9b740 1 -- start start 2024-04-24T09:21:04.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.808+0000 7f3300d9b740 1 Processor -- start 2024-04-24T09:21:04.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.808+0000 7f3300d9b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596313f6260 0x559631400670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:04.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.808+0000 7f3300d9b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5596313f6c50 0x559631400bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:04.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.808+0000 7f3300d9b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596313fbdc0 0x559631407bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:04.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.808+0000 7f3300d9b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5596313fed10 con 0x5596313fbdc0 2024-04-24T09:21:04.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.808+0000 7f3300d9b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x559631315590 con 0x5596313f6260 2024-04-24T09:21:04.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.808+0000 7f3300d9b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5596313fee90 con 0x5596313f6c50 2024-04-24T09:21:04.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.808+0000 7f3300d9b740 10 client.0 _ll_register_callbacks cb 0x5596312f83a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:21:04.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.808+0000 7f32efe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5596313f6c50 0x559631400bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:04.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.808+0000 7f32efe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5596313f6c50 0x559631400bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:38114/0 (socket says 172.21.15.90:38114) 2024-04-24T09:21:04.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.808+0000 7f32efe00640 1 -- 172.21.15.90:0/2898211216 learned_addr learned my addr 172.21.15.90:0/2898211216 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:04.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.808+0000 7f32f6a00640 1 --2- 172.21.15.90:0/2898211216 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596313f6260 0x559631400670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:04.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.809+0000 7f32f7400640 1 --2- 172.21.15.90:0/2898211216 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596313fbdc0 0x559631407bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:04.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.809+0000 7f32f7400640 1 -- 172.21.15.90:0/2898211216 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5596313f6c50 msgr2=0x559631400bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:04.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.809+0000 7f32f7400640 1 --2- 172.21.15.90:0/2898211216 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5596313f6c50 0x559631400bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.809+0000 7f32f7400640 1 -- 172.21.15.90:0/2898211216 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596313f6260 msgr2=0x559631400670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:04.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.809+0000 7f32f7400640 1 --2- 172.21.15.90:0/2898211216 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596313f6260 0x559631400670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.809+0000 7f32f7400640 1 -- 172.21.15.90:0/2898211216 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596311386b0 con 0x5596313fbdc0 2024-04-24T09:21:04.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.809+0000 7f32efe00640 1 --2- 172.21.15.90:0/2898211216 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5596313f6c50 0x559631400bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:21:04.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.809+0000 7f32f7400640 1 --2- 172.21.15.90:0/2898211216 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596313fbdc0 0x559631407bf0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f32f0005fd0 tx=0x7f32f0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=152c833f6607c63a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:04.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.809+0000 7f32e4c00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-24T09:21:04.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.810+0000 7f32e4c00640 1 -- 172.21.15.90:0/2898211216 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f000f870 con 0x5596313fbdc0 2024-04-24T09:21:04.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.810+0000 7f32e4c00640 1 -- 172.21.15.90:0/2898211216 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f32f000fa10 con 0x5596313fbdc0 2024-04-24T09:21:04.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.810+0000 7f3300d9b740 1 -- 172.21.15.90:0/2898211216 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32e8004030 con 0x5596313fbdc0 2024-04-24T09:21:04.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.810+0000 7f32e4c00640 1 -- 172.21.15.90:0/2898211216 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f32f000fc10 con 0x5596313fbdc0 2024-04-24T09:21:04.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.810+0000 7f3300d9b740 10 client.14880 fetch_fsmap learned FSMap version 27 2024-04-24T09:21:04.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.810+0000 7f3300d9b740 1 -- 172.21.15.90:0/2898211216 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55963140b080 con 0x5596313fbdc0 2024-04-24T09:21:04.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.810+0000 7f32e4c00640 1 -- 172.21.15.90:0/2898211216 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32e8004030 con 0x5596313fbdc0 2024-04-24T09:21:04.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.811+0000 7f32e4c00640 1 -- 172.21.15.90:0/2898211216 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f32f0026070 con 0x5596313fbdc0 2024-04-24T09:21:04.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.811+0000 7f3300d9b740 10 client.14880 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T09:21:04.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.811+0000 7f3300d9b740 10 client.14880 resolve_mds: resolved name 'j' to daemon mds.j 2024-04-24T09:21:04.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.811+0000 7f3300d9b740 20 client.14880 populate_metadata read hostname 'smithi090' 2024-04-24T09:21:04.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.811+0000 7f3300d9b740 1 --2- 172.21.15.90:0/2898211216 >> [v2:172.21.15.90:6834/2396995634,v1:172.21.15.90:6835/2396995634] conn(0x55963140d730 0x55963140eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:04.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.811+0000 7f3300d9b740 4 client.14880 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:21:04.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.811+0000 7f3300d9b740 1 -- 172.21.15.90:0/2898211216 --> [v2:172.21.15.90:6834/2396995634,v1:172.21.15.90:6835/2396995634] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559631369530 con 0x55963140d730 2024-04-24T09:21:04.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.811+0000 7f32f6a00640 1 --2- 172.21.15.90:0/2898211216 >> [v2:172.21.15.90:6834/2396995634,v1:172.21.15.90:6835/2396995634] conn(0x55963140d730 0x55963140eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:04.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.812+0000 7f32f6a00640 1 --2- 172.21.15.90:0/2898211216 >> [v2:172.21.15.90:6834/2396995634,v1:172.21.15.90:6835/2396995634] conn(0x55963140d730 0x55963140eba0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=756f3b7bdd99668c server_cookie=e653e494823bced2 in_seq=0 out_seq=0 2024-04-24T09:21:04.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.812+0000 7f32e4c00640 10 client.14880 ms_handle_connect on v2:172.21.15.90:6834/2396995634 2024-04-24T09:21:04.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.812+0000 7f32e4c00640 1 -- 172.21.15.90:0/2898211216 <== mds.4 v2:172.21.15.90:6834/2396995634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559631369530 con 0x55963140d730 2024-04-24T09:21:04.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.812+0000 7f32e4c00640 10 client.14880 handle_command_reply: tid=0 2024-04-24T09:21:04.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.812+0000 7f3300d9b740 1 client.14880 shutdown 2024-04-24T09:21:04.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.813+0000 7f3300d9b740 1 -- 172.21.15.90:0/2898211216 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596313fbdc0 msgr2=0x559631407bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:04.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.813+0000 7f3300d9b740 1 --2- 172.21.15.90:0/2898211216 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596313fbdc0 0x559631407bf0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f32f0005fd0 tx=0x7f32f0004ab0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.813+0000 7f3300d9b740 1 -- 172.21.15.90:0/2898211216 shutdown_connections 2024-04-24T09:21:04.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.813+0000 7f3300d9b740 1 --2- 172.21.15.90:0/2898211216 >> [v2:172.21.15.90:6834/2396995634,v1:172.21.15.90:6835/2396995634] conn(0x55963140d730 0x55963140eba0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.813+0000 7f3300d9b740 1 --2- 172.21.15.90:0/2898211216 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596313fbdc0 0x559631407bf0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.813+0000 7f3300d9b740 1 --2- 172.21.15.90:0/2898211216 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5596313f6c50 0x559631400bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.813+0000 7f32e4c00640 0 client.14880 ms_handle_reset on v2:172.21.15.90:6834/2396995634 2024-04-24T09:21:04.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.813+0000 7f3300d9b740 1 --2- 172.21.15.90:0/2898211216 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596313f6260 0x559631400670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.813+0000 7f3300d9b740 1 -- 172.21.15.90:0/2898211216 >> 172.21.15.90:0/2898211216 conn(0x5596313f18d0 msgr2=0x5596313f0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:04.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.813+0000 7f3300d9b740 1 -- 172.21.15.90:0/2898211216 shutdown_connections 2024-04-24T09:21:04.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.813+0000 7f3300d9b740 1 -- 172.21.15.90:0/2898211216 wait complete. 2024-04-24T09:21:04.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.813+0000 7f3300d9b740 20 client.14880 trim_cache size 0 max 16384 2024-04-24T09:21:04.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.833+0000 7f3300d9b740 1 Processor -- start 2024-04-24T09:21:04.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.833+0000 7f3300d9b740 1 -- start start 2024-04-24T09:21:04.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.833+0000 7f3300d9b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5596313f6260 0x5596313f10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:04.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.833+0000 7f3300d9b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596313f6c50 0x559631407bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:04.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.833+0000 7f3300d9b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596313fbdc0 0x5596313fe1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:04.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.833+0000 7f3300d9b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55963140cee0 con 0x5596313f6c50 2024-04-24T09:21:04.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.833+0000 7f3300d9b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55963140d5b0 con 0x5596313fbdc0 2024-04-24T09:21:04.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.833+0000 7f3300d9b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55963140dc80 con 0x5596313f6260 2024-04-24T09:21:04.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.833+0000 7f32efe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596313f6c50 0x559631407bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:04.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.833+0000 7f32efe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596313f6c50 0x559631407bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45484/0 (socket says 172.21.15.90:45484) 2024-04-24T09:21:04.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.833+0000 7f32efe00640 1 -- 172.21.15.90:0/2978521033 learned_addr learned my addr 172.21.15.90:0/2978521033 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:04.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.833+0000 7f32f6a00640 1 --2- 172.21.15.90:0/2978521033 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5596313f6260 0x5596313f10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:04.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.834+0000 7f32f7400640 1 --2- 172.21.15.90:0/2978521033 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596313fbdc0 0x5596313fe1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:04.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.834+0000 7f32efe00640 1 -- 172.21.15.90:0/2978521033 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5596313f6260 msgr2=0x5596313f10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:04.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.834+0000 7f32efe00640 1 --2- 172.21.15.90:0/2978521033 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5596313f6260 0x5596313f10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.834+0000 7f32efe00640 1 -- 172.21.15.90:0/2978521033 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596313fbdc0 msgr2=0x5596313fe1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:04.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.834+0000 7f32efe00640 1 --2- 172.21.15.90:0/2978521033 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596313fbdc0 0x5596313fe1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.834+0000 7f32efe00640 1 -- 172.21.15.90:0/2978521033 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596313f1700 con 0x5596313f6c50 2024-04-24T09:21:04.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.834+0000 7f32f7400640 1 --2- 172.21.15.90:0/2978521033 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596313fbdc0 0x5596313fe1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:21:04.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.834+0000 7f32efe00640 1 --2- 172.21.15.90:0/2978521033 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596313f6c50 0x559631407bb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f32e0098f50 tx=0x7f32e00d6530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80e48415b1a77e78 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:04.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.834+0000 7f32e7400640 1 -- 172.21.15.90:0/2978521033 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e009c450 con 0x5596313f6c50 2024-04-24T09:21:04.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.834+0000 7f32e7400640 1 -- 172.21.15.90:0/2978521033 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32e00bd0b0 con 0x5596313f6c50 2024-04-24T09:21:04.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.834+0000 7f32e7400640 1 -- 172.21.15.90:0/2978521033 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e00d0e60 con 0x5596313f6c50 2024-04-24T09:21:04.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.835+0000 7f3300d9b740 1 -- 172.21.15.90:0/2978521033 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596313f6c50 msgr2=0x559631407bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:04.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.835+0000 7f3300d9b740 1 --2- 172.21.15.90:0/2978521033 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596313f6c50 0x559631407bb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f32e0098f50 tx=0x7f32e00d6530 comp rx=0 tx=0).stop 2024-04-24T09:21:04.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.835+0000 7f3300d9b740 1 -- 172.21.15.90:0/2978521033 shutdown_connections 2024-04-24T09:21:04.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.835+0000 7f3300d9b740 1 --2- 172.21.15.90:0/2978521033 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596313fbdc0 0x5596313fe1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.835+0000 7f3300d9b740 1 --2- 172.21.15.90:0/2978521033 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596313f6c50 0x559631407bb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.835+0000 7f3300d9b740 1 --2- 172.21.15.90:0/2978521033 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5596313f6260 0x5596313f10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.835+0000 7f3300d9b740 1 -- 172.21.15.90:0/2978521033 >> 172.21.15.90:0/2978521033 conn(0x5596313f18d0 msgr2=0x5596313f8150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:04.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.835+0000 7f3300d9b740 1 -- 172.21.15.90:0/2978521033 shutdown_connections 2024-04-24T09:21:04.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.835+0000 7f3300d9b740 1 -- 172.21.15.90:0/2978521033 wait complete. 2024-04-24T09:21:04.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.835+0000 7f3300d9b740 1 -- start start 2024-04-24T09:21:04.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.836+0000 7f3300d9b740 1 Processor -- start 2024-04-24T09:21:04.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.836+0000 7f3300d9b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596313f6260 0x559631403d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:04.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.836+0000 7f3300d9b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5596313f6c50 0x559631404280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:04.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.836+0000 7f3300d9b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596313fbdc0 0x559631406d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:04.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.836+0000 7f3300d9b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55963140ea20 con 0x5596313f6260 2024-04-24T09:21:04.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.836+0000 7f3300d9b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55963140e350 con 0x5596313fbdc0 2024-04-24T09:21:04.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.836+0000 7f3300d9b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55963140f0f0 con 0x5596313f6c50 2024-04-24T09:21:04.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.836+0000 7f3300d9b740 10 client.0 _ll_register_callbacks cb 0x7f32f0003e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:21:04.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.836+0000 7f32f6a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596313f6260 0x559631403d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:04.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.836+0000 7f32efe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5596313f6c50 0x559631404280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:04.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.836+0000 7f32efe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5596313f6c50 0x559631404280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:38138/0 (socket says 172.21.15.90:38138) 2024-04-24T09:21:04.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.836+0000 7f32efe00640 1 -- 172.21.15.90:0/1561540170 learned_addr learned my addr 172.21.15.90:0/1561540170 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:04.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.836+0000 7f32f7400640 1 --2- 172.21.15.90:0/1561540170 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596313fbdc0 0x559631406d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:04.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.837+0000 7f32efe00640 1 -- 172.21.15.90:0/1561540170 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596313fbdc0 msgr2=0x559631406d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:04.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.837+0000 7f32efe00640 1 --2- 172.21.15.90:0/1561540170 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596313fbdc0 0x559631406d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.837+0000 7f32efe00640 1 -- 172.21.15.90:0/1561540170 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596313f6260 msgr2=0x559631403d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:04.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.837+0000 7f32efe00640 1 --2- 172.21.15.90:0/1561540170 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596313f6260 0x559631403d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.837+0000 7f32efe00640 1 -- 172.21.15.90:0/1561540170 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559631407300 con 0x5596313f6c50 2024-04-24T09:21:04.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.837+0000 7f32f6a00640 1 --2- 172.21.15.90:0/1561540170 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596313f6260 0x559631403d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:21:04.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.837+0000 7f32efe00640 1 --2- 172.21.15.90:0/1561540170 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5596313f6c50 0x559631404280 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f32e0099470 tx=0x7f32e00ce370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d33d1d6cd5bcb1ad server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:04.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.837+0000 7f32e4c00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-04-24T09:21:04.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.837+0000 7f32e4c00640 1 -- 172.21.15.90:0/1561540170 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e009c450 con 0x5596313f6c50 2024-04-24T09:21:04.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.837+0000 7f32e4c00640 1 -- 172.21.15.90:0/1561540170 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f32e0010430 con 0x5596313f6c50 2024-04-24T09:21:04.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.837+0000 7f3300d9b740 1 -- 172.21.15.90:0/1561540170 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32e00bd0b0 con 0x5596313f6c50 2024-04-24T09:21:04.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.837+0000 7f32e4c00640 1 -- 172.21.15.90:0/1561540170 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f32e009caf0 con 0x5596313f6c50 2024-04-24T09:21:04.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.837+0000 7f3300d9b740 10 client.24997 fetch_fsmap learned FSMap version 27 2024-04-24T09:21:04.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.837+0000 7f3300d9b740 1 -- 172.21.15.90:0/1561540170 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559631473bd0 con 0x5596313f6c50 2024-04-24T09:21:04.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.838+0000 7f32e4c00640 1 -- 172.21.15.90:0/1561540170 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32e00bd0b0 con 0x5596313f6c50 2024-04-24T09:21:04.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.838+0000 7f32e4c00640 1 -- 172.21.15.90:0/1561540170 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f32e009b9b0 con 0x5596313f6c50 2024-04-24T09:21:04.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.838+0000 7f3300d9b740 10 client.24997 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T09:21:04.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.838+0000 7f3300d9b740 10 client.24997 resolve_mds: resolved name 'j' to daemon mds.j 2024-04-24T09:21:04.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.838+0000 7f3300d9b740 20 client.24997 populate_metadata read hostname 'smithi090' 2024-04-24T09:21:04.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.838+0000 7f3300d9b740 1 --2- 172.21.15.90:0/1561540170 >> [v2:172.21.15.90:6834/2396995634,v1:172.21.15.90:6835/2396995634] conn(0x5596314782f0 0x559631474e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:04.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.838+0000 7f3300d9b740 4 client.24997 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T09:21:04.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.838+0000 7f3300d9b740 1 -- 172.21.15.90:0/1561540170 --> [v2:172.21.15.90:6834/2396995634,v1:172.21.15.90:6835/2396995634] -- command(tid 0: {"prefix": "status"}) -- 0x5596313633a0 con 0x5596314782f0 2024-04-24T09:21:04.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.838+0000 7f32f6a00640 1 --2- 172.21.15.90:0/1561540170 >> [v2:172.21.15.90:6834/2396995634,v1:172.21.15.90:6835/2396995634] conn(0x5596314782f0 0x559631474e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:04.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.839+0000 7f32f6a00640 1 --2- 172.21.15.90:0/1561540170 >> [v2:172.21.15.90:6834/2396995634,v1:172.21.15.90:6835/2396995634] conn(0x5596314782f0 0x559631474e00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c5c6befa5f307d9f server_cookie=f2d3acef82b256f2 in_seq=0 out_seq=0 2024-04-24T09:21:04.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.839+0000 7f32e4c00640 10 client.24997 ms_handle_connect on v2:172.21.15.90:6834/2396995634 2024-04-24T09:21:04.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.839+0000 7f32e4c00640 1 -- 172.21.15.90:0/1561540170 <== mds.4 v2:172.21.15.90:6834/2396995634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5596313633a0 con 0x5596314782f0 2024-04-24T09:21:04.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.839+0000 7f32e4c00640 10 client.24997 handle_command_reply: tid=0 2024-04-24T09:21:04.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.839+0000 7f3300d9b740 1 client.24997 shutdown 2024-04-24T09:21:04.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.839+0000 7f3300d9b740 1 -- 172.21.15.90:0/1561540170 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5596313f6c50 msgr2=0x559631404280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:04.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.839+0000 7f3300d9b740 1 --2- 172.21.15.90:0/1561540170 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5596313f6c50 0x559631404280 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f32e0099470 tx=0x7f32e00ce370 comp rx=0 tx=0).stop 2024-04-24T09:21:04.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.839+0000 7f3300d9b740 1 -- 172.21.15.90:0/1561540170 shutdown_connections 2024-04-24T09:21:04.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.839+0000 7f3300d9b740 1 --2- 172.21.15.90:0/1561540170 >> [v2:172.21.15.90:6834/2396995634,v1:172.21.15.90:6835/2396995634] conn(0x5596314782f0 0x559631474e00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.839+0000 7f3300d9b740 1 --2- 172.21.15.90:0/1561540170 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596313fbdc0 0x559631406d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.839+0000 7f3300d9b740 1 --2- 172.21.15.90:0/1561540170 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5596313f6c50 0x559631404280 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.839+0000 7f3300d9b740 1 --2- 172.21.15.90:0/1561540170 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596313f6260 0x559631403d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.839+0000 7f3300d9b740 1 -- 172.21.15.90:0/1561540170 >> 172.21.15.90:0/1561540170 conn(0x5596313f18d0 msgr2=0x559631400670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:04.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.839+0000 7f32e4c00640 0 client.24997 ms_handle_reset on v2:172.21.15.90:6834/2396995634 2024-04-24T09:21:04.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.839+0000 7f3300d9b740 1 -- 172.21.15.90:0/1561540170 shutdown_connections 2024-04-24T09:21:04.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.839+0000 7f3300d9b740 1 -- 172.21.15.90:0/1561540170 wait complete. 2024-04-24T09:21:04.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.840+0000 7f3300d9b740 20 client.24997 trim_cache size 0 max 16384 2024-04-24T09:21:04.844 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T09:21:04.844 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "30ee9be2-021a-11ef-bc93-c7b262605968", 2024-04-24T09:21:04.844 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 4, 2024-04-24T09:21:04.844 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 24446, 2024-04-24T09:21:04.844 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-04-24T09:21:04.844 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-04-24T09:21:04.844 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-04-24T09:21:04.844 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 108.583589144, 2024-04-24T09:21:04.844 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 27, 2024-04-24T09:21:04.844 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 88, 2024-04-24T09:21:04.844 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 86, 2024-04-24T09:21:04.845 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 151.10908776100001 2024-04-24T09:21:04.845 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T09:21:04.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.840+0000 7f32f7e00640 1 -- 172.21.15.90:0/1410216587 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f32dc0a3bb0 msgr2=0x7f32dc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:04.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.840+0000 7f32f7e00640 1 --2- 172.21.15.90:0/1410216587 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f32dc0a3bb0 0x7f32dc0a6070 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x5596313f74b0 tx=0x7f32f0009500 comp rx=0 tx=0).stop 2024-04-24T09:21:04.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.840+0000 7f32f7e00640 1 -- 172.21.15.90:0/1410216587 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32f8109730 msgr2=0x7f32f819ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:04.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.840+0000 7f32f7e00640 1 --2- 172.21.15.90:0/1410216587 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32f8109730 0x7f32f819ed40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f32e000e9f0 tx=0x7f32e000eeb0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.840+0000 7f32f7e00640 1 -- 172.21.15.90:0/1410216587 shutdown_connections 2024-04-24T09:21:04.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.840+0000 7f32f7e00640 1 --2- 172.21.15.90:0/1410216587 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f32dc0a3bb0 0x7f32dc0a6070 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.840+0000 7f32f7e00640 1 --2- 172.21.15.90:0/1410216587 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f32f810a0f0 0x7f32f81a5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.840+0000 7f32f7e00640 1 --2- 172.21.15.90:0/1410216587 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32f8109730 0x7f32f819ed40 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.840+0000 7f32f7e00640 1 --2- 172.21.15.90:0/1410216587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32f8108540 0x7f32f819e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:04.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.840+0000 7f32f7e00640 1 -- 172.21.15.90:0/1410216587 >> 172.21.15.90:0/1410216587 conn(0x7f32f80774d0 msgr2=0x7f32f810ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:04.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.840+0000 7f32f7e00640 1 -- 172.21.15.90:0/1410216587 shutdown_connections 2024-04-24T09:21:04.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:04.840+0000 7f32f7e00640 1 -- 172.21.15.90:0/1410216587 wait complete. 2024-04-24T09:21:04.851 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:04 smithi090 ceph-mon[27825]: pgmap v381: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:21:04.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:04 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/3336379529' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:21:04.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:04 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/2299423204' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:21:04.859 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-24T09:21:05.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.048+0000 7fa556800640 1 -- 172.21.15.90:0/868173358 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa550108540 msgr2=0x7fa550108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:05.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.048+0000 7fa556800640 1 --2- 172.21.15.90:0/868173358 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa550108540 0x7fa550108960 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa544009860 tx=0x7fa54402a340 comp rx=0 tx=0).stop 2024-04-24T09:21:05.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.048+0000 7fa556800640 1 -- 172.21.15.90:0/868173358 shutdown_connections 2024-04-24T09:21:05.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.048+0000 7fa556800640 1 --2- 172.21.15.90:0/868173358 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa55010a0f0 0x7fa550110980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.048+0000 7fa556800640 1 --2- 172.21.15.90:0/868173358 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa550109730 0x7fa550109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.048+0000 7fa556800640 1 --2- 172.21.15.90:0/868173358 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa550108540 0x7fa550108960 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.048+0000 7fa556800640 1 -- 172.21.15.90:0/868173358 >> 172.21.15.90:0/868173358 conn(0x7fa5500774d0 msgr2=0x7fa550079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:05.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.048+0000 7fa556800640 1 -- 172.21.15.90:0/868173358 shutdown_connections 2024-04-24T09:21:05.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.048+0000 7fa556800640 1 -- 172.21.15.90:0/868173358 wait complete. 2024-04-24T09:21:05.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.049+0000 7fa556800640 1 Processor -- start 2024-04-24T09:21:05.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.049+0000 7fa556800640 1 -- start start 2024-04-24T09:21:05.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.050+0000 7fa556800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa550108540 0x7fa55019e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.050+0000 7fa556800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa550109730 0x7fa55019ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.050+0000 7fa556800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa55010a0f0 0x7fa5501a5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.050+0000 7fa556800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa5501141d0 con 0x7fa550109730 2024-04-24T09:21:05.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.050+0000 7fa556800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa550114050 con 0x7fa550108540 2024-04-24T09:21:05.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.050+0000 7fa556800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa550114350 con 0x7fa55010a0f0 2024-04-24T09:21:05.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.050+0000 7fa554a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa550109730 0x7fa55019ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.050+0000 7fa555400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa550108540 0x7fa55019e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.050+0000 7fa554a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa550109730 0x7fa55019ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45498/0 (socket says 172.21.15.90:45498) 2024-04-24T09:21:05.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.050+0000 7fa555400640 1 --2- 172.21.15.90:0/2297517043 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa550108540 0x7fa55019e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.90:38830/0 (socket says 172.21.15.90:38830) 2024-04-24T09:21:05.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.050+0000 7fa554a00640 1 -- 172.21.15.90:0/2297517043 learned_addr learned my addr 172.21.15.90:0/2297517043 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:05.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.050+0000 7fa555e00640 1 --2- 172.21.15.90:0/2297517043 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa55010a0f0 0x7fa5501a5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.050+0000 7fa555400640 1 -- 172.21.15.90:0/2297517043 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa55010a0f0 msgr2=0x7fa5501a5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:05.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.050+0000 7fa555400640 1 --2- 172.21.15.90:0/2297517043 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa55010a0f0 0x7fa5501a5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.050+0000 7fa555400640 1 -- 172.21.15.90:0/2297517043 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa550109730 msgr2=0x7fa55019ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:05.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.050+0000 7fa555400640 1 --2- 172.21.15.90:0/2297517043 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa550109730 0x7fa55019ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.051+0000 7fa555400640 1 -- 172.21.15.90:0/2297517043 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5501a6420 con 0x7fa550108540 2024-04-24T09:21:05.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.051+0000 7fa555400640 1 --2- 172.21.15.90:0/2297517043 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa550108540 0x7fa55019e800 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa54402a860 tx=0x7fa544033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:05.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.051+0000 7fa54e000640 1 -- 172.21.15.90:0/2297517043 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa544037040 con 0x7fa550108540 2024-04-24T09:21:05.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.051+0000 7fa54e000640 1 -- 172.21.15.90:0/2297517043 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa544030850 con 0x7fa550108540 2024-04-24T09:21:05.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.051+0000 7fa556800640 1 -- 172.21.15.90:0/2297517043 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5501a66b0 con 0x7fa550108540 2024-04-24T09:21:05.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.051+0000 7fa54e000640 1 -- 172.21.15.90:0/2297517043 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa544005650 con 0x7fa550108540 2024-04-24T09:21:05.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.051+0000 7fa556800640 1 -- 172.21.15.90:0/2297517043 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5501a6b70 con 0x7fa550108540 2024-04-24T09:21:05.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.055+0000 7fa55ac25740 1 Processor -- start 2024-04-24T09:21:05.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.055+0000 7fa55ac25740 1 -- start start 2024-04-24T09:21:05.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.056+0000 7fa55ac25740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610229c4260 0x5610229c4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.056+0000 7fa55ac25740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610229c4c50 0x5610229c9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.056+0000 7fa55ac25740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610229c9dc0 0x5610229cc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.056+0000 7fa55ac25740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561022929b10 con 0x5610229c4260 2024-04-24T09:21:05.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.056+0000 7fa55ac25740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5610228d02b0 con 0x5610229c9dc0 2024-04-24T09:21:05.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.056+0000 7fa55ac25740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561022925ad0 con 0x5610229c4c50 2024-04-24T09:21:05.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.056+0000 7fa54e000640 1 -- 172.21.15.90:0/2297517043 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa544004050 con 0x7fa550108540 2024-04-24T09:21:05.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.056+0000 7fa555e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610229c4260 0x5610229c4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.056+0000 7fa555e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610229c4260 0x5610229c4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45512/0 (socket says 172.21.15.90:45512) 2024-04-24T09:21:05.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.056+0000 7fa555e00640 1 -- 172.21.15.90:0/3936394581 learned_addr learned my addr 172.21.15.90:0/3936394581 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:05.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.056+0000 7fa554a00640 1 --2- 172.21.15.90:0/3936394581 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610229c9dc0 0x5610229cc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.056+0000 7fa555400640 1 --2- 172.21.15.90:0/3936394581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610229c4c50 0x5610229c9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.056+0000 7fa555e00640 1 -- 172.21.15.90:0/3936394581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610229c4c50 msgr2=0x5610229c9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:05.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.056+0000 7fa555e00640 1 --2- 172.21.15.90:0/3936394581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610229c4c50 0x5610229c9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.056+0000 7fa555e00640 1 -- 172.21.15.90:0/3936394581 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610229c9dc0 msgr2=0x5610229cc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:05.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.056+0000 7fa555e00640 1 --2- 172.21.15.90:0/3936394581 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610229c9dc0 0x5610229cc250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.056+0000 7fa555e00640 1 -- 172.21.15.90:0/3936394581 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56102263c2d0 con 0x5610229c4260 2024-04-24T09:21:05.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.057+0000 7fa555e00640 1 --2- 172.21.15.90:0/3936394581 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610229c4260 0x5610229c4680 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa54800e480 tx=0x7fa5480095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f53a74c5cccf096 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:05.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.057+0000 7fa53e600640 1 -- 172.21.15.90:0/3936394581 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa548003970 con 0x5610229c4260 2024-04-24T09:21:05.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.057+0000 7fa53e600640 1 -- 172.21.15.90:0/3936394581 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa54800d7d0 con 0x5610229c4260 2024-04-24T09:21:05.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.057+0000 7fa53e600640 1 -- 172.21.15.90:0/3936394581 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5480044a0 con 0x5610229c4260 2024-04-24T09:21:05.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.057+0000 7fa54e000640 1 --2- 172.21.15.90:0/2297517043 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa5380a3bb0 0x7fa5380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.058+0000 7fa54e000640 1 -- 172.21.15.90:0/2297517043 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fa5440f0280 con 0x7fa550108540 2024-04-24T09:21:05.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.058+0000 7fa55ac25740 1 -- 172.21.15.90:0/3936394581 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610229c4260 msgr2=0x5610229c4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:05.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.058+0000 7fa55ac25740 1 --2- 172.21.15.90:0/3936394581 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610229c4260 0x5610229c4680 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa54800e480 tx=0x7fa5480095c0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.058+0000 7fa555400640 1 --2- 172.21.15.90:0/2297517043 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa5380a3bb0 0x7fa5380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.058+0000 7fa55ac25740 1 -- 172.21.15.90:0/3936394581 shutdown_connections 2024-04-24T09:21:05.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.058+0000 7fa55ac25740 1 --2- 172.21.15.90:0/3936394581 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610229c9dc0 0x5610229cc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.058+0000 7fa55ac25740 1 --2- 172.21.15.90:0/3936394581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610229c4c50 0x5610229c9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.058+0000 7fa55ac25740 1 --2- 172.21.15.90:0/3936394581 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610229c4260 0x5610229c4680 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.058+0000 7fa55ac25740 1 -- 172.21.15.90:0/3936394581 >> 172.21.15.90:0/3936394581 conn(0x5610229bf8d0 msgr2=0x5610229bfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:05.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.058+0000 7fa55ac25740 1 -- 172.21.15.90:0/3936394581 shutdown_connections 2024-04-24T09:21:05.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.058+0000 7fa55ac25740 1 -- 172.21.15.90:0/3936394581 wait complete. 2024-04-24T09:21:05.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.058+0000 7fa555400640 1 --2- 172.21.15.90:0/2297517043 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa5380a3bb0 0x7fa5380a6070 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa54402a930 tx=0x7fa544005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:05.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.059+0000 7fa55ac25740 1 -- start start 2024-04-24T09:21:05.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.059+0000 7fa55ac25740 1 Processor -- start 2024-04-24T09:21:05.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.059+0000 7fa55ac25740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610229c4260 0x5610229ce660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.059+0000 7fa55ac25740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610229c4c50 0x5610229ceba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.059+0000 7fa55ac25740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610229c9dc0 0x5610229d5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.059+0000 7fa55ac25740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5610229ccd70 con 0x5610229c9dc0 2024-04-24T09:21:05.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.059+0000 7fa55ac25740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5610228e3590 con 0x5610229c4c50 2024-04-24T09:21:05.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.059+0000 7fa55ac25740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5610229ccef0 con 0x5610229c4260 2024-04-24T09:21:05.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.059+0000 7fa55ac25740 10 client.0 _ll_register_callbacks cb 0x5610228c63a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:21:05.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.059+0000 7fa554a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610229c4c50 0x5610229ceba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.059+0000 7fa555e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610229c4260 0x5610229ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.060+0000 7fa554a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610229c4c50 0x5610229ceba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.90:38834/0 (socket says 172.21.15.90:38834) 2024-04-24T09:21:05.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.060+0000 7fa554a00640 1 -- 172.21.15.90:0/3523734054 learned_addr learned my addr 172.21.15.90:0/3523734054 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:05.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.060+0000 7fa555e00640 1 --2- 172.21.15.90:0/3523734054 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610229c9dc0 0x5610229d5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.060+0000 7fa554a00640 1 -- 172.21.15.90:0/3523734054 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610229c4260 msgr2=0x5610229ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:05.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.060+0000 7fa554a00640 1 --2- 172.21.15.90:0/3523734054 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610229c4260 0x5610229ce660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.060+0000 7fa554a00640 1 -- 172.21.15.90:0/3523734054 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610229c9dc0 msgr2=0x5610229d5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:05.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.060+0000 7fa554a00640 1 --2- 172.21.15.90:0/3523734054 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610229c9dc0 0x5610229d5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.060+0000 7fa554a00640 1 -- 172.21.15.90:0/3523734054 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610227066b0 con 0x5610229c4c50 2024-04-24T09:21:05.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.060+0000 7fa554a00640 1 --2- 172.21.15.90:0/3523734054 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610229c4c50 0x5610229ceba0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa54000ba90 tx=0x7fa54000bf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e91fdcfcb14ed7c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:05.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.060+0000 7fa532a00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-04-24T09:21:05.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.061+0000 7fa532a00640 1 -- 172.21.15.90:0/3523734054 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa540011440 con 0x5610229c4c50 2024-04-24T09:21:05.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.061+0000 7fa532a00640 1 -- 172.21.15.90:0/3523734054 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fa5400115e0 con 0x5610229c4c50 2024-04-24T09:21:05.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.061+0000 7fa55ac25740 1 -- 172.21.15.90:0/3523734054 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa54800d7d0 con 0x5610229c4c50 2024-04-24T09:21:05.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.061+0000 7fa532a00640 1 -- 172.21.15.90:0/3523734054 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa5400117e0 con 0x5610229c4c50 2024-04-24T09:21:05.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.061+0000 7fa55ac25740 10 client.24983 fetch_fsmap learned FSMap version 27 2024-04-24T09:21:05.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.061+0000 7fa55ac25740 1 -- 172.21.15.90:0/3523734054 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5610229d9010 con 0x5610229c4c50 2024-04-24T09:21:05.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.061+0000 7fa532a00640 1 -- 172.21.15.90:0/3523734054 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa54800d7d0 con 0x5610229c4c50 2024-04-24T09:21:05.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.061+0000 7fa532a00640 1 -- 172.21.15.90:0/3523734054 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa54001bbf0 con 0x5610229c4c50 2024-04-24T09:21:05.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.061+0000 7fa55ac25740 10 client.24983 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T09:21:05.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.062+0000 7fa55ac25740 10 client.24983 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-24T09:21:05.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.062+0000 7fa55ac25740 20 client.24983 populate_metadata read hostname 'smithi090' 2024-04-24T09:21:05.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.062+0000 7fa55ac25740 1 --2- 172.21.15.90:0/3523734054 >> [v2:172.21.15.149:6836/2250310587,v1:172.21.15.149:6837/2250310587] conn(0x5610229db690 0x5610229dcb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.062+0000 7fa55ac25740 4 client.24983 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:21:05.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.062+0000 7fa55ac25740 1 -- 172.21.15.90:0/3523734054 --> [v2:172.21.15.149:6836/2250310587,v1:172.21.15.149:6837/2250310587] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561022937530 con 0x5610229db690 2024-04-24T09:21:05.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.062+0000 7fa555e00640 1 --2- 172.21.15.90:0/3523734054 >> [v2:172.21.15.149:6836/2250310587,v1:172.21.15.149:6837/2250310587] conn(0x5610229db690 0x5610229dcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.063+0000 7fa532a00640 10 client.24983 ms_handle_connect on v2:172.21.15.149:6836/2250310587 2024-04-24T09:21:05.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.063+0000 7fa555e00640 1 --2- 172.21.15.90:0/3523734054 >> [v2:172.21.15.149:6836/2250310587,v1:172.21.15.149:6837/2250310587] conn(0x5610229db690 0x5610229dcb00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f2fcfdd7330a3087 server_cookie=bf5dae43c143fd24 in_seq=0 out_seq=0 2024-04-24T09:21:05.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.064+0000 7fa532a00640 1 -- 172.21.15.90:0/3523734054 <== mds.3 v2:172.21.15.149:6836/2250310587 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561022937530 con 0x5610229db690 2024-04-24T09:21:05.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.064+0000 7fa532a00640 10 client.24983 handle_command_reply: tid=0 2024-04-24T09:21:05.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.064+0000 7fa55ac25740 1 client.24983 shutdown 2024-04-24T09:21:05.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.064+0000 7fa55ac25740 1 -- 172.21.15.90:0/3523734054 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610229c4c50 msgr2=0x5610229ceba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:05.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.064+0000 7fa55ac25740 1 --2- 172.21.15.90:0/3523734054 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610229c4c50 0x5610229ceba0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa54000ba90 tx=0x7fa54000bf50 comp rx=0 tx=0).stop 2024-04-24T09:21:05.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.064+0000 7fa55ac25740 1 -- 172.21.15.90:0/3523734054 shutdown_connections 2024-04-24T09:21:05.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.064+0000 7fa55ac25740 1 --2- 172.21.15.90:0/3523734054 >> [v2:172.21.15.149:6836/2250310587,v1:172.21.15.149:6837/2250310587] conn(0x5610229db690 0x5610229dcb00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.064+0000 7fa55ac25740 1 --2- 172.21.15.90:0/3523734054 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610229c9dc0 0x5610229d5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.064+0000 7fa55ac25740 1 --2- 172.21.15.90:0/3523734054 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610229c4c50 0x5610229ceba0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.064+0000 7fa55ac25740 1 --2- 172.21.15.90:0/3523734054 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610229c4260 0x5610229ce660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.064+0000 7fa55ac25740 1 -- 172.21.15.90:0/3523734054 >> 172.21.15.90:0/3523734054 conn(0x5610229bf8d0 msgr2=0x5610229be970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:05.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.064+0000 7fa532a00640 0 client.24983 ms_handle_reset on v2:172.21.15.149:6836/2250310587 2024-04-24T09:21:05.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.064+0000 7fa55ac25740 1 -- 172.21.15.90:0/3523734054 shutdown_connections 2024-04-24T09:21:05.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.064+0000 7fa55ac25740 1 -- 172.21.15.90:0/3523734054 wait complete. 2024-04-24T09:21:05.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.064+0000 7fa55ac25740 20 client.24983 trim_cache size 0 max 16384 2024-04-24T09:21:05.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.084+0000 7fa55ac25740 1 Processor -- start 2024-04-24T09:21:05.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.084+0000 7fa55ac25740 1 -- start start 2024-04-24T09:21:05.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.084+0000 7fa55ac25740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610229c4260 0x5610229be0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.084+0000 7fa55ac25740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610229c4c50 0x5610229d5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.084+0000 7fa55ac25740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610229c9dc0 0x5610229be620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.084+0000 7fa55ac25740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5610229dae40 con 0x5610229c9dc0 2024-04-24T09:21:05.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.084+0000 7fa55ac25740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5610229db510 con 0x5610229c4260 2024-04-24T09:21:05.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.084+0000 7fa55ac25740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5610229dbbe0 con 0x5610229c4c50 2024-04-24T09:21:05.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.084+0000 7fa555e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610229c9dc0 0x5610229be620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.084+0000 7fa554a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610229c4c50 0x5610229d5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.084+0000 7fa555e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610229c4260 0x5610229be0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.084+0000 7fa555e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610229c4260 0x5610229be0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.90:38848/0 (socket says 172.21.15.90:38848) 2024-04-24T09:21:05.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.084+0000 7fa555e00640 1 -- 172.21.15.90:0/3535246411 learned_addr learned my addr 172.21.15.90:0/3535246411 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:05.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.084+0000 7fa554a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610229c4c50 0x5610229d5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:38178/0 (socket says 172.21.15.90:38178) 2024-04-24T09:21:05.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.085+0000 7fa555e00640 1 -- 172.21.15.90:0/3535246411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610229c4c50 msgr2=0x5610229d5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:05.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.085+0000 7fa555e00640 1 --2- 172.21.15.90:0/3535246411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610229c4c50 0x5610229d5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.085+0000 7fa555e00640 1 -- 172.21.15.90:0/3535246411 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610229c9dc0 msgr2=0x5610229be620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:05.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.085+0000 7fa555e00640 1 --2- 172.21.15.90:0/3535246411 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610229c9dc0 0x5610229be620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.085+0000 7fa555e00640 1 -- 172.21.15.90:0/3535246411 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610229bb230 con 0x5610229c4260 2024-04-24T09:21:05.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.085+0000 7fa555e00640 1 --2- 172.21.15.90:0/3535246411 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610229c4260 0x5610229be0e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa54800dfd0 tx=0x7fa548005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b2f0cc72f24882d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:05.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.085+0000 7fa53dc00640 1 -- 172.21.15.90:0/3535246411 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa548038070 con 0x5610229c4260 2024-04-24T09:21:05.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.085+0000 7fa53dc00640 1 -- 172.21.15.90:0/3535246411 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5480312c0 con 0x5610229c4260 2024-04-24T09:21:05.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.085+0000 7fa53dc00640 1 -- 172.21.15.90:0/3535246411 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5480051c0 con 0x5610229c4260 2024-04-24T09:21:05.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.086+0000 7fa55ac25740 1 -- 172.21.15.90:0/3535246411 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610229c4260 msgr2=0x5610229be0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:05.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.086+0000 7fa55ac25740 1 --2- 172.21.15.90:0/3535246411 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610229c4260 0x5610229be0e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa54800dfd0 tx=0x7fa548005fd0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.086+0000 7fa55ac25740 1 -- 172.21.15.90:0/3535246411 shutdown_connections 2024-04-24T09:21:05.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.086+0000 7fa55ac25740 1 --2- 172.21.15.90:0/3535246411 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610229c9dc0 0x5610229be620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.086+0000 7fa55ac25740 1 --2- 172.21.15.90:0/3535246411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610229c4c50 0x5610229d5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.086+0000 7fa55ac25740 1 --2- 172.21.15.90:0/3535246411 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610229c4260 0x5610229be0e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.086+0000 7fa55ac25740 1 -- 172.21.15.90:0/3535246411 >> 172.21.15.90:0/3535246411 conn(0x5610229bf8d0 msgr2=0x5610229c6010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:05.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.086+0000 7fa55ac25740 1 -- 172.21.15.90:0/3535246411 shutdown_connections 2024-04-24T09:21:05.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.086+0000 7fa55ac25740 1 -- 172.21.15.90:0/3535246411 wait complete. 2024-04-24T09:21:05.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.086+0000 7fa55ac25740 1 -- start start 2024-04-24T09:21:05.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.087+0000 7fa55ac25740 1 Processor -- start 2024-04-24T09:21:05.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.087+0000 7fa55ac25740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610229c4260 0x5610229d1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.087+0000 7fa55ac25740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610229c4c50 0x5610229d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.087+0000 7fa55ac25740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610229c9dc0 0x5610229d4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.087+0000 7fa55ac25740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5610229dc980 con 0x5610229c9dc0 2024-04-24T09:21:05.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.087+0000 7fa55ac25740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5610229dc2b0 con 0x5610229c4c50 2024-04-24T09:21:05.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.087+0000 7fa55ac25740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5610229dd050 con 0x5610229c4260 2024-04-24T09:21:05.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.087+0000 7fa55ac25740 10 client.0 _ll_register_callbacks cb 0x7fa540004530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:21:05.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.087+0000 7fa555e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610229c9dc0 0x5610229d4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.087+0000 7fa555e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610229c9dc0 0x5610229d4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45540/0 (socket says 172.21.15.90:45540) 2024-04-24T09:21:05.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.087+0000 7fa555e00640 1 -- 172.21.15.90:0/2385492097 learned_addr learned my addr 172.21.15.90:0/2385492097 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:05.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.087+0000 7fa555e00640 1 --2- 172.21.15.90:0/2385492097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610229c4260 0x5610229d1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.087+0000 7fa554a00640 1 --2- 172.21.15.90:0/2385492097 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610229c4c50 0x5610229d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.088+0000 7fa555e00640 1 -- 172.21.15.90:0/2385492097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610229c4260 msgr2=0x5610229d1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:05.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.088+0000 7fa555e00640 1 --2- 172.21.15.90:0/2385492097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610229c4260 0x5610229d1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.088+0000 7fa555e00640 1 -- 172.21.15.90:0/2385492097 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610229c4c50 msgr2=0x5610229d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:05.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.088+0000 7fa555e00640 1 --2- 172.21.15.90:0/2385492097 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610229c4c50 0x5610229d2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.088+0000 7fa555e00640 1 -- 172.21.15.90:0/2385492097 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610229d5270 con 0x5610229c9dc0 2024-04-24T09:21:05.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.088+0000 7fa555e00640 1 --2- 172.21.15.90:0/2385492097 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610229c9dc0 0x5610229d4c30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x5610229d2bb0 tx=0x7fa548027040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=462eb5bab08045ff server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:05.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.088+0000 7fa532a00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-24T09:21:05.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.088+0000 7fa532a00640 1 -- 172.21.15.90:0/2385492097 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa548038040 con 0x5610229c9dc0 2024-04-24T09:21:05.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.088+0000 7fa532a00640 1 -- 172.21.15.90:0/2385492097 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fa548031470 con 0x5610229c9dc0 2024-04-24T09:21:05.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.088+0000 7fa55ac25740 1 -- 172.21.15.90:0/2385492097 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5480312c0 con 0x5610229c9dc0 2024-04-24T09:21:05.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.088+0000 7fa532a00640 1 -- 172.21.15.90:0/2385492097 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa548031670 con 0x5610229c9dc0 2024-04-24T09:21:05.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.088+0000 7fa55ac25740 10 client.14904 fetch_fsmap learned FSMap version 27 2024-04-24T09:21:05.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.089+0000 7fa55ac25740 1 -- 172.21.15.90:0/2385492097 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561022a41a50 con 0x5610229c9dc0 2024-04-24T09:21:05.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.089+0000 7fa532a00640 1 -- 172.21.15.90:0/2385492097 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5480312c0 con 0x5610229c9dc0 2024-04-24T09:21:05.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.089+0000 7fa532a00640 1 -- 172.21.15.90:0/2385492097 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa54803e040 con 0x5610229c9dc0 2024-04-24T09:21:05.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.089+0000 7fa55ac25740 10 client.14904 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T09:21:05.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.089+0000 7fa55ac25740 10 client.14904 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-24T09:21:05.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.089+0000 7fa55ac25740 20 client.14904 populate_metadata read hostname 'smithi090' 2024-04-24T09:21:05.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.089+0000 7fa55ac25740 1 --2- 172.21.15.90:0/2385492097 >> [v2:172.21.15.149:6836/2250310587,v1:172.21.15.149:6837/2250310587] conn(0x561022a45f20 0x561022a44c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.089+0000 7fa55ac25740 4 client.14904 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T09:21:05.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.089+0000 7fa55ac25740 1 -- 172.21.15.90:0/2385492097 --> [v2:172.21.15.149:6836/2250310587,v1:172.21.15.149:6837/2250310587] -- command(tid 0: {"prefix": "status"}) -- 0x5610229313a0 con 0x561022a45f20 2024-04-24T09:21:05.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.089+0000 7fa554a00640 1 --2- 172.21.15.90:0/2385492097 >> [v2:172.21.15.149:6836/2250310587,v1:172.21.15.149:6837/2250310587] conn(0x561022a45f20 0x561022a44c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.090+0000 7fa554a00640 1 --2- 172.21.15.90:0/2385492097 >> [v2:172.21.15.149:6836/2250310587,v1:172.21.15.149:6837/2250310587] conn(0x561022a45f20 0x561022a44c30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=da68e706508e8be7 server_cookie=cd91f8121e264e8e in_seq=0 out_seq=0 2024-04-24T09:21:05.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.090+0000 7fa532a00640 10 client.14904 ms_handle_connect on v2:172.21.15.149:6836/2250310587 2024-04-24T09:21:05.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.090+0000 7fa532a00640 1 -- 172.21.15.90:0/2385492097 <== mds.3 v2:172.21.15.149:6836/2250310587 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x5610229313a0 con 0x561022a45f20 2024-04-24T09:21:05.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.090+0000 7fa532a00640 10 client.14904 handle_command_reply: tid=0 2024-04-24T09:21:05.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.090+0000 7fa55ac25740 1 client.14904 shutdown 2024-04-24T09:21:05.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.090+0000 7fa55ac25740 1 -- 172.21.15.90:0/2385492097 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610229c9dc0 msgr2=0x5610229d4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:05.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.090+0000 7fa55ac25740 1 --2- 172.21.15.90:0/2385492097 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610229c9dc0 0x5610229d4c30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x5610229d2bb0 tx=0x7fa548027040 comp rx=0 tx=0).stop 2024-04-24T09:21:05.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.091+0000 7fa55ac25740 1 -- 172.21.15.90:0/2385492097 shutdown_connections 2024-04-24T09:21:05.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.091+0000 7fa55ac25740 1 --2- 172.21.15.90:0/2385492097 >> [v2:172.21.15.149:6836/2250310587,v1:172.21.15.149:6837/2250310587] conn(0x561022a45f20 0x561022a44c30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.091+0000 7fa55ac25740 1 --2- 172.21.15.90:0/2385492097 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610229c9dc0 0x5610229d4c30 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.091+0000 7fa55ac25740 1 --2- 172.21.15.90:0/2385492097 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610229c4c50 0x5610229d2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.091+0000 7fa55ac25740 1 --2- 172.21.15.90:0/2385492097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610229c4260 0x5610229d1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.091+0000 7fa532a00640 0 client.14904 ms_handle_reset on v2:172.21.15.149:6836/2250310587 2024-04-24T09:21:05.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.091+0000 7fa55ac25740 1 -- 172.21.15.90:0/2385492097 >> 172.21.15.90:0/2385492097 conn(0x5610229bf8d0 msgr2=0x5610229c6010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:05.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.091+0000 7fa55ac25740 1 -- 172.21.15.90:0/2385492097 shutdown_connections 2024-04-24T09:21:05.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.091+0000 7fa55ac25740 1 -- 172.21.15.90:0/2385492097 wait complete. 2024-04-24T09:21:05.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.091+0000 7fa55ac25740 20 client.14904 trim_cache size 0 max 16384 2024-04-24T09:21:05.096 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T09:21:05.096 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "30ee9be2-021a-11ef-bc93-c7b262605968", 2024-04-24T09:21:05.096 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 3, 2024-04-24T09:21:05.096 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 24526, 2024-04-24T09:21:05.097 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-04-24T09:21:05.097 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-04-24T09:21:05.097 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-04-24T09:21:05.097 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 109.796433432, 2024-04-24T09:21:05.097 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 27, 2024-04-24T09:21:05.097 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 88, 2024-04-24T09:21:05.097 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 86, 2024-04-24T09:21:05.097 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 138.333306924 2024-04-24T09:21:05.097 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T09:21:05.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.091+0000 7fa556800640 1 -- 172.21.15.90:0/2297517043 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa5380a3bb0 msgr2=0x7fa5380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:05.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.091+0000 7fa556800640 1 --2- 172.21.15.90:0/2297517043 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa5380a3bb0 0x7fa5380a6070 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa54402a930 tx=0x7fa544005f90 comp rx=0 tx=0).stop 2024-04-24T09:21:05.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.091+0000 7fa556800640 1 -- 172.21.15.90:0/2297517043 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa550108540 msgr2=0x7fa55019e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:05.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.091+0000 7fa556800640 1 --2- 172.21.15.90:0/2297517043 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa550108540 0x7fa55019e800 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa54402a860 tx=0x7fa544033d90 comp rx=0 tx=0).stop 2024-04-24T09:21:05.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.092+0000 7fa556800640 1 -- 172.21.15.90:0/2297517043 shutdown_connections 2024-04-24T09:21:05.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.092+0000 7fa556800640 1 --2- 172.21.15.90:0/2297517043 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa5380a3bb0 0x7fa5380a6070 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.092+0000 7fa556800640 1 --2- 172.21.15.90:0/2297517043 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa55010a0f0 0x7fa5501a5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.092+0000 7fa556800640 1 --2- 172.21.15.90:0/2297517043 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa550109730 0x7fa55019ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.092+0000 7fa556800640 1 --2- 172.21.15.90:0/2297517043 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa550108540 0x7fa55019e800 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.092+0000 7fa556800640 1 -- 172.21.15.90:0/2297517043 >> 172.21.15.90:0/2297517043 conn(0x7fa5500774d0 msgr2=0x7fa55010ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:05.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.092+0000 7fa556800640 1 -- 172.21.15.90:0/2297517043 shutdown_connections 2024-04-24T09:21:05.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.092+0000 7fa556800640 1 -- 172.21.15.90:0/2297517043 wait complete. 2024-04-24T09:21:05.110 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-24T09:21:05.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:04 smithi149 ceph-mon[32592]: pgmap v381: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:21:05.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:04 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/3336379529' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:21:05.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:04 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/2299423204' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:21:05.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:04 smithi114 ceph-mon[32501]: pgmap v381: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:21:05.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:04 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/3336379529' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:21:05.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:04 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/2299423204' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:21:05.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.313+0000 7f230e400640 1 -- 172.21.15.90:0/2314751685 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2308105cf0 msgr2=0x7f230810c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:05.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.313+0000 7f230e400640 1 --2- 172.21.15.90:0/2314751685 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2308105cf0 0x7f230810c580 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2300009ca0 tx=0x7f230002a8b0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.313+0000 7f230e400640 1 -- 172.21.15.90:0/2314751685 shutdown_connections 2024-04-24T09:21:05.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.313+0000 7f230e400640 1 --2- 172.21.15.90:0/2314751685 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2308105cf0 0x7f230810c580 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.313+0000 7f230e400640 1 --2- 172.21.15.90:0/2314751685 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2308105330 0x7f23081057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.313+0000 7f230e400640 1 --2- 172.21.15.90:0/2314751685 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2308104140 0x7f2308104560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.313+0000 7f230e400640 1 -- 172.21.15.90:0/2314751685 >> 172.21.15.90:0/2314751685 conn(0x7f23080ff6b0 msgr2=0x7f2308101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:05.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.313+0000 7f230e400640 1 -- 172.21.15.90:0/2314751685 shutdown_connections 2024-04-24T09:21:05.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.313+0000 7f230e400640 1 -- 172.21.15.90:0/2314751685 wait complete. 2024-04-24T09:21:05.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.314+0000 7f230e400640 1 Processor -- start 2024-04-24T09:21:05.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.314+0000 7f230e400640 1 -- start start 2024-04-24T09:21:05.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.314+0000 7f230e400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2308104140 0x7f230819a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.314+0000 7f230e400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2308105330 0x7f230819a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.314+0000 7f230e400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2308105cf0 0x7f23081a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.314+0000 7f230e400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f230810fdd0 con 0x7f2308104140 2024-04-24T09:21:05.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.314+0000 7f230e400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f230810fc50 con 0x7f2308105330 2024-04-24T09:21:05.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.314+0000 7f230e400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f230810ff50 con 0x7f2308105cf0 2024-04-24T09:21:05.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.314+0000 7f230d000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2308104140 0x7f230819a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.314+0000 7f230da00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2308105cf0 0x7f23081a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.314+0000 7f230d000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2308104140 0x7f230819a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45550/0 (socket says 172.21.15.90:45550) 2024-04-24T09:21:05.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.314+0000 7f230d000640 1 -- 172.21.15.90:0/504036796 learned_addr learned my addr 172.21.15.90:0/504036796 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:05.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.314+0000 7f2307e00640 1 --2- 172.21.15.90:0/504036796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2308105330 0x7f230819a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.315+0000 7f230d000640 1 -- 172.21.15.90:0/504036796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2308105cf0 msgr2=0x7f23081a19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:05.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.315+0000 7f230d000640 1 --2- 172.21.15.90:0/504036796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2308105cf0 0x7f23081a19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.315+0000 7f230d000640 1 -- 172.21.15.90:0/504036796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2308105330 msgr2=0x7f230819a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:05.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.315+0000 7f230d000640 1 --2- 172.21.15.90:0/504036796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2308105330 0x7f230819a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.315+0000 7f230d000640 1 -- 172.21.15.90:0/504036796 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23081a2010 con 0x7f2308104140 2024-04-24T09:21:05.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.315+0000 7f230d000640 1 --2- 172.21.15.90:0/504036796 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2308104140 0x7f230819a3f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f22fc00e9c0 tx=0x7f22fc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:05.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.315+0000 7f2305600640 1 -- 172.21.15.90:0/504036796 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22fc00cd80 con 0x7f2308104140 2024-04-24T09:21:05.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.315+0000 7f230e400640 1 -- 172.21.15.90:0/504036796 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23081a2300 con 0x7f2308104140 2024-04-24T09:21:05.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.315+0000 7f2305600640 1 -- 172.21.15.90:0/504036796 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22fc0035f0 con 0x7f2308104140 2024-04-24T09:21:05.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.315+0000 7f2305600640 1 -- 172.21.15.90:0/504036796 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22fc010690 con 0x7f2308104140 2024-04-24T09:21:05.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.315+0000 7f230e400640 1 -- 172.21.15.90:0/504036796 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23081a2800 con 0x7f2308104140 2024-04-24T09:21:05.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.319+0000 7f23129b1740 1 Processor -- start 2024-04-24T09:21:05.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.319+0000 7f2305600640 1 -- 172.21.15.90:0/504036796 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f22fc0037a0 con 0x7f2308104140 2024-04-24T09:21:05.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.319+0000 7f23129b1740 1 -- start start 2024-04-24T09:21:05.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.319+0000 7f23129b1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558ef3b45260 0x558ef3b45680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.319+0000 7f23129b1740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558ef3b45c50 0x558ef3b4a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.319+0000 7f23129b1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ef3b4adc0 0x558ef3b4d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.319+0000 7f23129b1740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x558ef3aaab10 con 0x558ef3b45c50 2024-04-24T09:21:05.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.319+0000 7f23129b1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x558ef3a512b0 con 0x558ef3b45260 2024-04-24T09:21:05.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.319+0000 7f23129b1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x558ef3aa6ad0 con 0x558ef3b4adc0 2024-04-24T09:21:05.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.319+0000 7f2307e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ef3b4adc0 0x558ef3b4d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.319+0000 7f230d000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558ef3b45c50 0x558ef3b4a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.319+0000 7f2307e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ef3b4adc0 0x558ef3b4d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:38208/0 (socket says 172.21.15.90:38208) 2024-04-24T09:21:05.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.319+0000 7f2307e00640 1 -- 172.21.15.90:0/3149248999 learned_addr learned my addr 172.21.15.90:0/3149248999 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:05.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.320+0000 7f230da00640 1 --2- 172.21.15.90:0/3149248999 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558ef3b45260 0x558ef3b45680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.320+0000 7f2307e00640 1 -- 172.21.15.90:0/3149248999 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558ef3b45260 msgr2=0x558ef3b45680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:05.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.320+0000 7f2307e00640 1 --2- 172.21.15.90:0/3149248999 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558ef3b45260 0x558ef3b45680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.320+0000 7f2307e00640 1 -- 172.21.15.90:0/3149248999 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558ef3b45c50 msgr2=0x558ef3b4a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:05.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.320+0000 7f2307e00640 1 --2- 172.21.15.90:0/3149248999 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558ef3b45c50 0x558ef3b4a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.320+0000 7f2307e00640 1 -- 172.21.15.90:0/3149248999 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ef37bd2d0 con 0x558ef3b4adc0 2024-04-24T09:21:05.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.320+0000 7f2307e00640 1 --2- 172.21.15.90:0/3149248999 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ef3b4adc0 0x558ef3b4d250 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f22f4009c90 tx=0x7f22f4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c748a02744f59d2d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:05.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.320+0000 7f2305600640 1 --2- 172.21.15.90:0/504036796 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f22f00a3bb0 0x7f22f00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.320+0000 7f22fa000640 1 -- 172.21.15.90:0/3149248999 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f401e070 con 0x558ef3b4adc0 2024-04-24T09:21:05.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.320+0000 7f22fa000640 1 -- 172.21.15.90:0/3149248999 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22f4019b10 con 0x558ef3b4adc0 2024-04-24T09:21:05.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.320+0000 7f22fa000640 1 -- 172.21.15.90:0/3149248999 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f400eae0 con 0x558ef3b4adc0 2024-04-24T09:21:05.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.321+0000 7f2305600640 1 -- 172.21.15.90:0/504036796 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f22fc0d0760 con 0x7f2308104140 2024-04-24T09:21:05.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.321+0000 7f230da00640 1 --2- 172.21.15.90:0/504036796 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f22f00a3bb0 0x7f22f00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.321+0000 7f23129b1740 1 -- 172.21.15.90:0/3149248999 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ef3b4adc0 msgr2=0x558ef3b4d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:05.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.321+0000 7f23129b1740 1 --2- 172.21.15.90:0/3149248999 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ef3b4adc0 0x558ef3b4d250 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f22f4009c90 tx=0x7f22f4011810 comp rx=0 tx=0).stop 2024-04-24T09:21:05.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.321+0000 7f230da00640 1 --2- 172.21.15.90:0/504036796 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f22f00a3bb0 0x7f22f00a6070 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2300004af0 tx=0x7f2300004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:05.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.321+0000 7f23129b1740 1 -- 172.21.15.90:0/3149248999 shutdown_connections 2024-04-24T09:21:05.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.321+0000 7f23129b1740 1 --2- 172.21.15.90:0/3149248999 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ef3b4adc0 0x558ef3b4d250 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.321+0000 7f23129b1740 1 --2- 172.21.15.90:0/3149248999 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558ef3b45c50 0x558ef3b4a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.321+0000 7f23129b1740 1 --2- 172.21.15.90:0/3149248999 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558ef3b45260 0x558ef3b45680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.321+0000 7f23129b1740 1 -- 172.21.15.90:0/3149248999 >> 172.21.15.90:0/3149248999 conn(0x558ef3b408d0 msgr2=0x558ef3b40cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:05.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.321+0000 7f23129b1740 1 -- 172.21.15.90:0/3149248999 shutdown_connections 2024-04-24T09:21:05.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.322+0000 7f23129b1740 1 -- 172.21.15.90:0/3149248999 wait complete. 2024-04-24T09:21:05.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.322+0000 7f23129b1740 1 -- start start 2024-04-24T09:21:05.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.322+0000 7f23129b1740 1 Processor -- start 2024-04-24T09:21:05.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.322+0000 7f23129b1740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558ef3b45260 0x558ef3b4f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.322+0000 7f23129b1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ef3b45c50 0x558ef3b4fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.322+0000 7f23129b1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558ef3b4adc0 0x558ef3b56bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.322+0000 7f23129b1740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x558ef3b4dd70 con 0x558ef3b45260 2024-04-24T09:21:05.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.323+0000 7f23129b1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x558ef3a64590 con 0x558ef3b4adc0 2024-04-24T09:21:05.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.323+0000 7f23129b1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x558ef3b4def0 con 0x558ef3b45c50 2024-04-24T09:21:05.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.323+0000 7f23129b1740 10 client.0 _ll_register_callbacks cb 0x558ef3a473a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:21:05.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.323+0000 7f230d000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558ef3b45260 0x558ef3b4f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.323+0000 7f230d000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558ef3b45260 0x558ef3b4f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45560/0 (socket says 172.21.15.90:45560) 2024-04-24T09:21:05.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.323+0000 7f230d000640 1 -- 172.21.15.90:0/3209450836 learned_addr learned my addr 172.21.15.90:0/3209450836 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:05.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.323+0000 7f2307e00640 1 --2- 172.21.15.90:0/3209450836 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ef3b45c50 0x558ef3b4fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.323+0000 7f230da00640 1 --2- 172.21.15.90:0/3209450836 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558ef3b4adc0 0x558ef3b56bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.323+0000 7f230d000640 1 -- 172.21.15.90:0/3209450836 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ef3b45c50 msgr2=0x558ef3b4fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:05.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.323+0000 7f230d000640 1 --2- 172.21.15.90:0/3209450836 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ef3b45c50 0x558ef3b4fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.323+0000 7f230d000640 1 -- 172.21.15.90:0/3209450836 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558ef3b4adc0 msgr2=0x558ef3b56bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:05.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.323+0000 7f230d000640 1 --2- 172.21.15.90:0/3209450836 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558ef3b4adc0 0x558ef3b56bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.323+0000 7f230d000640 1 -- 172.21.15.90:0/3209450836 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ef38876b0 con 0x558ef3b45260 2024-04-24T09:21:05.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.323+0000 7f230d000640 1 --2- 172.21.15.90:0/3209450836 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558ef3b45260 0x558ef3b4f650 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f22fc0a5180 tx=0x7f22fc098680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90e12b559a45a75e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:05.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.323+0000 7f22eea00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-24T09:21:05.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.324+0000 7f22eea00640 1 -- 172.21.15.90:0/3209450836 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22fc0d6020 con 0x558ef3b45260 2024-04-24T09:21:05.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.324+0000 7f22eea00640 1 -- 172.21.15.90:0/3209450836 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f22fc003b10 con 0x558ef3b45260 2024-04-24T09:21:05.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.324+0000 7f23129b1740 1 -- 172.21.15.90:0/3209450836 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22f4019b10 con 0x558ef3b45260 2024-04-24T09:21:05.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.324+0000 7f22eea00640 1 -- 172.21.15.90:0/3209450836 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f22fc09c580 con 0x558ef3b45260 2024-04-24T09:21:05.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.324+0000 7f23129b1740 10 client.14916 fetch_fsmap learned FSMap version 27 2024-04-24T09:21:05.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.324+0000 7f23129b1740 1 -- 172.21.15.90:0/3209450836 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558ef3b59fb0 con 0x558ef3b45260 2024-04-24T09:21:05.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.324+0000 7f22eea00640 1 -- 172.21.15.90:0/3209450836 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22f4019b10 con 0x558ef3b45260 2024-04-24T09:21:05.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.324+0000 7f22eea00640 1 -- 172.21.15.90:0/3209450836 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f22fc0a52b0 con 0x558ef3b45260 2024-04-24T09:21:05.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.324+0000 7f23129b1740 10 client.14916 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T09:21:05.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.324+0000 7f23129b1740 10 client.14916 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T09:21:05.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.324+0000 7f23129b1740 20 client.14916 populate_metadata read hostname 'smithi090' 2024-04-24T09:21:05.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.325+0000 7f23129b1740 1 --2- 172.21.15.90:0/3209450836 >> [v2:172.21.15.90:6840/397593398,v1:172.21.15.90:6841/397593398] conn(0x558ef3b5c670 0x558ef3b5dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.325+0000 7f23129b1740 4 client.14916 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:21:05.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.325+0000 7f23129b1740 1 -- 172.21.15.90:0/3209450836 --> [v2:172.21.15.90:6840/397593398,v1:172.21.15.90:6841/397593398] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ef3ab8530 con 0x558ef3b5c670 2024-04-24T09:21:05.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.325+0000 7f2307e00640 1 --2- 172.21.15.90:0/3209450836 >> [v2:172.21.15.90:6840/397593398,v1:172.21.15.90:6841/397593398] conn(0x558ef3b5c670 0x558ef3b5dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.325+0000 7f2307e00640 1 --2- 172.21.15.90:0/3209450836 >> [v2:172.21.15.90:6840/397593398,v1:172.21.15.90:6841/397593398] conn(0x558ef3b5c670 0x558ef3b5dae0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=85cd296c29829202 server_cookie=2267ef5d21886e79 in_seq=0 out_seq=0 2024-04-24T09:21:05.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.325+0000 7f22eea00640 10 client.14916 ms_handle_connect on v2:172.21.15.90:6840/397593398 2024-04-24T09:21:05.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.326+0000 7f22eea00640 1 -- 172.21.15.90:0/3209450836 <== mds.2 v2:172.21.15.90:6840/397593398 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558ef3ab8530 con 0x558ef3b5c670 2024-04-24T09:21:05.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.326+0000 7f22eea00640 10 client.14916 handle_command_reply: tid=0 2024-04-24T09:21:05.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.326+0000 7f23129b1740 1 client.14916 shutdown 2024-04-24T09:21:05.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.327+0000 7f23129b1740 1 -- 172.21.15.90:0/3209450836 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558ef3b45260 msgr2=0x558ef3b4f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:05.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.327+0000 7f23129b1740 1 --2- 172.21.15.90:0/3209450836 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558ef3b45260 0x558ef3b4f650 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f22fc0a5180 tx=0x7f22fc098680 comp rx=0 tx=0).stop 2024-04-24T09:21:05.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.327+0000 7f23129b1740 1 -- 172.21.15.90:0/3209450836 shutdown_connections 2024-04-24T09:21:05.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.327+0000 7f23129b1740 1 --2- 172.21.15.90:0/3209450836 >> [v2:172.21.15.90:6840/397593398,v1:172.21.15.90:6841/397593398] conn(0x558ef3b5c670 0x558ef3b5dae0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.327+0000 7f23129b1740 1 --2- 172.21.15.90:0/3209450836 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558ef3b4adc0 0x558ef3b56bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.327+0000 7f23129b1740 1 --2- 172.21.15.90:0/3209450836 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ef3b45c50 0x558ef3b4fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.327+0000 7f23129b1740 1 --2- 172.21.15.90:0/3209450836 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558ef3b45260 0x558ef3b4f650 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.327+0000 7f22eea00640 0 client.14916 ms_handle_reset on v2:172.21.15.90:6840/397593398 2024-04-24T09:21:05.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.327+0000 7f23129b1740 1 -- 172.21.15.90:0/3209450836 >> 172.21.15.90:0/3209450836 conn(0x558ef3b408d0 msgr2=0x558ef3b3f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:05.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.327+0000 7f23129b1740 1 -- 172.21.15.90:0/3209450836 shutdown_connections 2024-04-24T09:21:05.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.327+0000 7f23129b1740 1 -- 172.21.15.90:0/3209450836 wait complete. 2024-04-24T09:21:05.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.327+0000 7f23129b1740 20 client.14916 trim_cache size 0 max 16384 2024-04-24T09:21:05.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.346+0000 7f23129b1740 1 Processor -- start 2024-04-24T09:21:05.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.346+0000 7f23129b1740 1 -- start start 2024-04-24T09:21:05.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.347+0000 7f23129b1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558ef3b45260 0x558ef3b4d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.347+0000 7f23129b1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ef3b45c50 0x558ef3b56b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.347+0000 7f23129b1740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558ef3b4adc0 0x558ef3b4a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.347+0000 7f23129b1740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x558ef3b5be20 con 0x558ef3b4adc0 2024-04-24T09:21:05.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.347+0000 7f23129b1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x558ef3b5c4f0 con 0x558ef3b45260 2024-04-24T09:21:05.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.347+0000 7f23129b1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x558ef3b5cbc0 con 0x558ef3b45c50 2024-04-24T09:21:05.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.347+0000 7f230da00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558ef3b4adc0 0x558ef3b4a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.347+0000 7f2307e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ef3b45c50 0x558ef3b56b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.347+0000 7f230da00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558ef3b4adc0 0x558ef3b4a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45564/0 (socket says 172.21.15.90:45564) 2024-04-24T09:21:05.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.347+0000 7f230da00640 1 -- 172.21.15.90:0/3732504353 learned_addr learned my addr 172.21.15.90:0/3732504353 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:05.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.347+0000 7f230d000640 1 --2- 172.21.15.90:0/3732504353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558ef3b45260 0x558ef3b4d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.347+0000 7f2307e00640 1 -- 172.21.15.90:0/3732504353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558ef3b45260 msgr2=0x558ef3b4d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:05.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.347+0000 7f2307e00640 1 --2- 172.21.15.90:0/3732504353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558ef3b45260 0x558ef3b4d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.347+0000 7f2307e00640 1 -- 172.21.15.90:0/3732504353 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558ef3b4adc0 msgr2=0x558ef3b4a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:05.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.347+0000 7f2307e00640 1 --2- 172.21.15.90:0/3732504353 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558ef3b4adc0 0x558ef3b4a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.347+0000 7f2307e00640 1 -- 172.21.15.90:0/3732504353 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ef3b46030 con 0x558ef3b45c50 2024-04-24T09:21:05.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.348+0000 7f230da00640 1 --2- 172.21.15.90:0/3732504353 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558ef3b4adc0 0x558ef3b4a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:21:05.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.348+0000 7f2307e00640 1 --2- 172.21.15.90:0/3732504353 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ef3b45c50 0x558ef3b56b60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f22f4011d10 tx=0x7f22f4009f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c57a5d76b33602ce server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:05.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.348+0000 7f22f9600640 1 -- 172.21.15.90:0/3732504353 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f401e040 con 0x558ef3b45c50 2024-04-24T09:21:05.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.348+0000 7f22f9600640 1 -- 172.21.15.90:0/3732504353 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22f4030020 con 0x558ef3b45c50 2024-04-24T09:21:05.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.348+0000 7f22f9600640 1 -- 172.21.15.90:0/3732504353 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f4019a30 con 0x558ef3b45c50 2024-04-24T09:21:05.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.349+0000 7f23129b1740 1 -- 172.21.15.90:0/3732504353 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ef3b45c50 msgr2=0x558ef3b56b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:05.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.349+0000 7f23129b1740 1 --2- 172.21.15.90:0/3732504353 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ef3b45c50 0x558ef3b56b60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f22f4011d10 tx=0x7f22f4009f00 comp rx=0 tx=0).stop 2024-04-24T09:21:05.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.349+0000 7f23129b1740 1 -- 172.21.15.90:0/3732504353 shutdown_connections 2024-04-24T09:21:05.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.349+0000 7f23129b1740 1 --2- 172.21.15.90:0/3732504353 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558ef3b4adc0 0x558ef3b4a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.349+0000 7f23129b1740 1 --2- 172.21.15.90:0/3732504353 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ef3b45c50 0x558ef3b56b60 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.349+0000 7f23129b1740 1 --2- 172.21.15.90:0/3732504353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558ef3b45260 0x558ef3b4d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.349+0000 7f23129b1740 1 -- 172.21.15.90:0/3732504353 >> 172.21.15.90:0/3732504353 conn(0x558ef3b408d0 msgr2=0x558ef3b46340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:05.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.349+0000 7f23129b1740 1 -- 172.21.15.90:0/3732504353 shutdown_connections 2024-04-24T09:21:05.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.349+0000 7f23129b1740 1 -- 172.21.15.90:0/3732504353 wait complete. 2024-04-24T09:21:05.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.349+0000 7f23129b1740 1 -- start start 2024-04-24T09:21:05.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.349+0000 7f23129b1740 1 Processor -- start 2024-04-24T09:21:05.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.350+0000 7f23129b1740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558ef3b45260 0x558ef3b52d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.350+0000 7f23129b1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ef3b45c50 0x558ef3b53260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.350+0000 7f23129b1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558ef3b4adc0 0x558ef3b55b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.350+0000 7f23129b1740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x558ef3b5d960 con 0x558ef3b45260 2024-04-24T09:21:05.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.350+0000 7f23129b1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x558ef3b5d290 con 0x558ef3b4adc0 2024-04-24T09:21:05.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.350+0000 7f23129b1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x558ef3b5e030 con 0x558ef3b45c50 2024-04-24T09:21:05.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.350+0000 7f23129b1740 10 client.0 _ll_register_callbacks cb 0x7f22fc0096e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:21:05.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.350+0000 7f230d000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558ef3b45260 0x558ef3b52d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.350+0000 7f230d000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558ef3b45260 0x558ef3b52d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45568/0 (socket says 172.21.15.90:45568) 2024-04-24T09:21:05.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.350+0000 7f230d000640 1 -- 172.21.15.90:0/4195521161 learned_addr learned my addr 172.21.15.90:0/4195521161 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:05.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.350+0000 7f2307e00640 1 --2- 172.21.15.90:0/4195521161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ef3b45c50 0x558ef3b53260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.350+0000 7f230da00640 1 --2- 172.21.15.90:0/4195521161 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558ef3b4adc0 0x558ef3b55b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.351+0000 7f230d000640 1 -- 172.21.15.90:0/4195521161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ef3b45c50 msgr2=0x558ef3b53260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:05.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.351+0000 7f230d000640 1 --2- 172.21.15.90:0/4195521161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ef3b45c50 0x558ef3b53260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.351+0000 7f230d000640 1 -- 172.21.15.90:0/4195521161 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558ef3b4adc0 msgr2=0x558ef3b55b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:05.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.351+0000 7f230d000640 1 --2- 172.21.15.90:0/4195521161 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558ef3b4adc0 0x558ef3b55b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.351+0000 7f230d000640 1 -- 172.21.15.90:0/4195521161 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ef3b56170 con 0x558ef3b45260 2024-04-24T09:21:05.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.351+0000 7f230d000640 1 --2- 172.21.15.90:0/4195521161 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558ef3b45260 0x558ef3b52d20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f22fc098ea0 tx=0x7f22fc003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31a45ad9f400873c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:05.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.351+0000 7f22eea00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-24T09:21:05.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.351+0000 7f22eea00640 1 -- 172.21.15.90:0/4195521161 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22fc09c3a0 con 0x558ef3b45260 2024-04-24T09:21:05.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.351+0000 7f22eea00640 1 -- 172.21.15.90:0/4195521161 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f22fc09bf10 con 0x558ef3b45260 2024-04-24T09:21:05.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.351+0000 7f23129b1740 1 -- 172.21.15.90:0/4195521161 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22f4030020 con 0x558ef3b45260 2024-04-24T09:21:05.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.351+0000 7f22eea00640 1 -- 172.21.15.90:0/4195521161 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f22fc0a5420 con 0x558ef3b45260 2024-04-24T09:21:05.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.351+0000 7f23129b1740 10 client.14922 fetch_fsmap learned FSMap version 27 2024-04-24T09:21:05.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.351+0000 7f23129b1740 1 -- 172.21.15.90:0/4195521161 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558ef3bc2960 con 0x558ef3b45260 2024-04-24T09:21:05.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.351+0000 7f22eea00640 1 -- 172.21.15.90:0/4195521161 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22f4030020 con 0x558ef3b45260 2024-04-24T09:21:05.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.352+0000 7f22eea00640 1 -- 172.21.15.90:0/4195521161 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f22fc098820 con 0x558ef3b45260 2024-04-24T09:21:05.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.352+0000 7f23129b1740 10 client.14922 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T09:21:05.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.352+0000 7f23129b1740 10 client.14922 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T09:21:05.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.352+0000 7f23129b1740 20 client.14922 populate_metadata read hostname 'smithi090' 2024-04-24T09:21:05.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.352+0000 7f23129b1740 1 --2- 172.21.15.90:0/4195521161 >> [v2:172.21.15.90:6840/397593398,v1:172.21.15.90:6841/397593398] conn(0x558ef3bc7550 0x558ef3bc3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.352+0000 7f23129b1740 4 client.14922 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T09:21:05.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.352+0000 7f23129b1740 1 -- 172.21.15.90:0/4195521161 --> [v2:172.21.15.90:6840/397593398,v1:172.21.15.90:6841/397593398] -- command(tid 0: {"prefix": "status"}) -- 0x558ef3ab23a0 con 0x558ef3bc7550 2024-04-24T09:21:05.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.352+0000 7f2307e00640 1 --2- 172.21.15.90:0/4195521161 >> [v2:172.21.15.90:6840/397593398,v1:172.21.15.90:6841/397593398] conn(0x558ef3bc7550 0x558ef3bc3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.352+0000 7f2307e00640 1 --2- 172.21.15.90:0/4195521161 >> [v2:172.21.15.90:6840/397593398,v1:172.21.15.90:6841/397593398] conn(0x558ef3bc7550 0x558ef3bc3990 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=99568b2f9c4034b5 server_cookie=f51e613b668d5b44 in_seq=0 out_seq=0 2024-04-24T09:21:05.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.352+0000 7f22eea00640 10 client.14922 ms_handle_connect on v2:172.21.15.90:6840/397593398 2024-04-24T09:21:05.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.353+0000 7f22eea00640 1 -- 172.21.15.90:0/4195521161 <== mds.2 v2:172.21.15.90:6840/397593398 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x558ef3ab23a0 con 0x558ef3bc7550 2024-04-24T09:21:05.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.353+0000 7f22eea00640 10 client.14922 handle_command_reply: tid=0 2024-04-24T09:21:05.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.353+0000 7f23129b1740 1 client.14922 shutdown 2024-04-24T09:21:05.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.353+0000 7f23129b1740 1 -- 172.21.15.90:0/4195521161 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558ef3b45260 msgr2=0x558ef3b52d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:05.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.353+0000 7f23129b1740 1 --2- 172.21.15.90:0/4195521161 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558ef3b45260 0x558ef3b52d20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f22fc098ea0 tx=0x7f22fc003f10 comp rx=0 tx=0).stop 2024-04-24T09:21:05.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.353+0000 7f23129b1740 1 -- 172.21.15.90:0/4195521161 shutdown_connections 2024-04-24T09:21:05.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.353+0000 7f23129b1740 1 --2- 172.21.15.90:0/4195521161 >> [v2:172.21.15.90:6840/397593398,v1:172.21.15.90:6841/397593398] conn(0x558ef3bc7550 0x558ef3bc3990 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.353+0000 7f23129b1740 1 --2- 172.21.15.90:0/4195521161 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558ef3b4adc0 0x558ef3b55b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.353+0000 7f23129b1740 1 --2- 172.21.15.90:0/4195521161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ef3b45c50 0x558ef3b53260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.353+0000 7f23129b1740 1 --2- 172.21.15.90:0/4195521161 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558ef3b45260 0x558ef3b52d20 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.353+0000 7f22eea00640 0 client.14922 ms_handle_reset on v2:172.21.15.90:6840/397593398 2024-04-24T09:21:05.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.353+0000 7f23129b1740 1 -- 172.21.15.90:0/4195521161 >> 172.21.15.90:0/4195521161 conn(0x558ef3b408d0 msgr2=0x558ef3b47050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:05.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.353+0000 7f23129b1740 1 -- 172.21.15.90:0/4195521161 shutdown_connections 2024-04-24T09:21:05.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.353+0000 7f23129b1740 1 -- 172.21.15.90:0/4195521161 wait complete. 2024-04-24T09:21:05.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.353+0000 7f23129b1740 20 client.14922 trim_cache size 0 max 16384 2024-04-24T09:21:05.359 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T09:21:05.359 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "30ee9be2-021a-11ef-bc93-c7b262605968", 2024-04-24T09:21:05.359 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 2, 2024-04-24T09:21:05.359 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 24545, 2024-04-24T09:21:05.359 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-04-24T09:21:05.360 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-04-24T09:21:05.360 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-04-24T09:21:05.360 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 111.06799121500001, 2024-04-24T09:21:05.360 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 27, 2024-04-24T09:21:05.360 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 88, 2024-04-24T09:21:05.360 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 86, 2024-04-24T09:21:05.360 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 124.428823575 2024-04-24T09:21:05.360 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T09:21:05.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.354+0000 7f230e400640 1 -- 172.21.15.90:0/504036796 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f22f00a3bb0 msgr2=0x7f22f00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:05.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.354+0000 7f230e400640 1 --2- 172.21.15.90:0/504036796 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f22f00a3bb0 0x7f22f00a6070 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2300004af0 tx=0x7f2300004a40 comp rx=0 tx=0).stop 2024-04-24T09:21:05.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.354+0000 7f230e400640 1 -- 172.21.15.90:0/504036796 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2308104140 msgr2=0x7f230819a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:05.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.354+0000 7f230e400640 1 --2- 172.21.15.90:0/504036796 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2308104140 0x7f230819a3f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f22fc00e9c0 tx=0x7f22fc00ee80 comp rx=0 tx=0).stop 2024-04-24T09:21:05.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.354+0000 7f230e400640 1 -- 172.21.15.90:0/504036796 shutdown_connections 2024-04-24T09:21:05.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.354+0000 7f230e400640 1 --2- 172.21.15.90:0/504036796 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f22f00a3bb0 0x7f22f00a6070 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.354+0000 7f230e400640 1 --2- 172.21.15.90:0/504036796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2308105cf0 0x7f23081a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.354+0000 7f230e400640 1 --2- 172.21.15.90:0/504036796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2308105330 0x7f230819a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.354+0000 7f230e400640 1 --2- 172.21.15.90:0/504036796 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2308104140 0x7f230819a3f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.354+0000 7f230e400640 1 -- 172.21.15.90:0/504036796 >> 172.21.15.90:0/504036796 conn(0x7f23080ff6b0 msgr2=0x7f2308108630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:05.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.354+0000 7f230e400640 1 -- 172.21.15.90:0/504036796 shutdown_connections 2024-04-24T09:21:05.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.354+0000 7f230e400640 1 -- 172.21.15.90:0/504036796 wait complete. 2024-04-24T09:21:05.371 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-24T09:21:05.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.559+0000 7f0d12200640 1 -- 172.21.15.90:0/1112203602 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d0c104140 msgr2=0x7f0d0c104560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:05.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.559+0000 7f0d12200640 1 --2- 172.21.15.90:0/1112203602 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d0c104140 0x7f0d0c104560 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0d00009900 tx=0x7f0d0002a410 comp rx=0 tx=0).stop 2024-04-24T09:21:05.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.559+0000 7f0d12200640 1 -- 172.21.15.90:0/1112203602 shutdown_connections 2024-04-24T09:21:05.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.559+0000 7f0d12200640 1 --2- 172.21.15.90:0/1112203602 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0d0c105cf0 0x7f0d0c10c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.559+0000 7f0d12200640 1 --2- 172.21.15.90:0/1112203602 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d0c105330 0x7f0d0c1057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.559+0000 7f0d12200640 1 --2- 172.21.15.90:0/1112203602 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d0c104140 0x7f0d0c104560 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.559+0000 7f0d12200640 1 -- 172.21.15.90:0/1112203602 >> 172.21.15.90:0/1112203602 conn(0x7f0d0c0ff6b0 msgr2=0x7f0d0c101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:05.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.560+0000 7f0d12200640 1 -- 172.21.15.90:0/1112203602 shutdown_connections 2024-04-24T09:21:05.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.560+0000 7f0d12200640 1 -- 172.21.15.90:0/1112203602 wait complete. 2024-04-24T09:21:05.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.560+0000 7f0d12200640 1 Processor -- start 2024-04-24T09:21:05.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.561+0000 7f0d12200640 1 -- start start 2024-04-24T09:21:05.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.561+0000 7f0d12200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d0c104140 0x7f0d0c19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.561+0000 7f0d12200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d0c105330 0x7f0d0c19a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.561+0000 7f0d12200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0d0c105cf0 0x7f0d0c1a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.561+0000 7f0d12200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c10fe20 con 0x7f0d0c104140 2024-04-24T09:21:05.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.561+0000 7f0d12200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c10fca0 con 0x7f0d0c105330 2024-04-24T09:21:05.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.561+0000 7f0d12200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c10ffa0 con 0x7f0d0c105cf0 2024-04-24T09:21:05.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.561+0000 7f0d10e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d0c104140 0x7f0d0c19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.561+0000 7f0d0be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d0c105330 0x7f0d0c19a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.561+0000 7f0d10e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d0c104140 0x7f0d0c19a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45592/0 (socket says 172.21.15.90:45592) 2024-04-24T09:21:05.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.561+0000 7f0d10e00640 1 -- 172.21.15.90:0/3702321849 learned_addr learned my addr 172.21.15.90:0/3702321849 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:05.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.561+0000 7f0d11800640 1 --2- 172.21.15.90:0/3702321849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0d0c105cf0 0x7f0d0c1a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.562+0000 7f0d0be00640 1 -- 172.21.15.90:0/3702321849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0d0c105cf0 msgr2=0x7f0d0c1a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:05.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.562+0000 7f0d0be00640 1 --2- 172.21.15.90:0/3702321849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0d0c105cf0 0x7f0d0c1a19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.562+0000 7f0d0be00640 1 -- 172.21.15.90:0/3702321849 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d0c104140 msgr2=0x7f0d0c19a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:05.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.562+0000 7f0d0be00640 1 --2- 172.21.15.90:0/3702321849 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d0c104140 0x7f0d0c19a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.562+0000 7f0d0be00640 1 -- 172.21.15.90:0/3702321849 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d0c1a2040 con 0x7f0d0c105330 2024-04-24T09:21:05.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.562+0000 7f0d10e00640 1 --2- 172.21.15.90:0/3702321849 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d0c104140 0x7f0d0c19a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:21:05.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.562+0000 7f0d0be00640 1 --2- 172.21.15.90:0/3702321849 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d0c105330 0x7f0d0c19a960 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0cf800e9f0 tx=0x7f0cf800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:05.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.562+0000 7f0d09600640 1 -- 172.21.15.90:0/3702321849 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf800ccc0 con 0x7f0d0c105330 2024-04-24T09:21:05.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.562+0000 7f0d12200640 1 -- 172.21.15.90:0/3702321849 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d0c1a2330 con 0x7f0d0c105330 2024-04-24T09:21:05.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.562+0000 7f0d09600640 1 -- 172.21.15.90:0/3702321849 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cf8010910 con 0x7f0d0c105330 2024-04-24T09:21:05.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.562+0000 7f0d09600640 1 -- 172.21.15.90:0/3702321849 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8018960 con 0x7f0d0c105330 2024-04-24T09:21:05.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.562+0000 7f0d12200640 1 -- 172.21.15.90:0/3702321849 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d0c1a28d0 con 0x7f0d0c105330 2024-04-24T09:21:05.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.566+0000 7f0d165f8740 1 Processor -- start 2024-04-24T09:21:05.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.566+0000 7f0d165f8740 1 -- start start 2024-04-24T09:21:05.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.566+0000 7f0d165f8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ba08161260 0x55ba08161680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.566+0000 7f0d165f8740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba08161c50 0x55ba08166750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.566+0000 7f0d165f8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba08166dc0 0x55ba08169250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.566+0000 7f0d165f8740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ba080c6b10 con 0x55ba08161c50 2024-04-24T09:21:05.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.566+0000 7f0d165f8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55ba0806d2b0 con 0x55ba08161260 2024-04-24T09:21:05.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.566+0000 7f0d165f8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ba080c2ad0 con 0x55ba08166dc0 2024-04-24T09:21:05.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.566+0000 7f0d10e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba08161c50 0x55ba08166750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.566+0000 7f0d11800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ba08161260 0x55ba08161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.566+0000 7f0d10e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba08161c50 0x55ba08166750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45602/0 (socket says 172.21.15.90:45602) 2024-04-24T09:21:05.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.566+0000 7f0d10e00640 1 -- 172.21.15.90:0/3151638696 learned_addr learned my addr 172.21.15.90:0/3151638696 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:05.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.566+0000 7f0d09600640 1 -- 172.21.15.90:0/3702321849 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0cf8010ac0 con 0x7f0d0c105330 2024-04-24T09:21:05.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.567+0000 7f0d10e00640 1 -- 172.21.15.90:0/3151638696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba08166dc0 msgr2=0x55ba08169250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:05.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.567+0000 7f0d10e00640 1 --2- 172.21.15.90:0/3151638696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba08166dc0 0x55ba08169250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.567+0000 7f0d10e00640 1 -- 172.21.15.90:0/3151638696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ba08161260 msgr2=0x55ba08161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:05.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.567+0000 7f0d10e00640 1 --2- 172.21.15.90:0/3151638696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ba08161260 0x55ba08161680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.567+0000 7f0d10e00640 1 -- 172.21.15.90:0/3151638696 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba07dd92d0 con 0x55ba08161c50 2024-04-24T09:21:05.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.567+0000 7f0d10e00640 1 --2- 172.21.15.90:0/3151638696 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba08161c50 0x55ba08166750 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0d0002a930 tx=0x7f0d000043f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f33907f5fd92fabb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:05.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.568+0000 7f0cfe000640 1 -- 172.21.15.90:0/3151638696 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d00040070 con 0x55ba08161c50 2024-04-24T09:21:05.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.568+0000 7f0cfe000640 1 -- 172.21.15.90:0/3151638696 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d000047a0 con 0x55ba08161c50 2024-04-24T09:21:05.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.568+0000 7f0cfe000640 1 -- 172.21.15.90:0/3151638696 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d00035040 con 0x55ba08161c50 2024-04-24T09:21:05.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.568+0000 7f0d09600640 1 --2- 172.21.15.90:0/3702321849 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f0cf40a3bb0 0x7f0cf40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.568+0000 7f0d09600640 1 -- 172.21.15.90:0/3702321849 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f0cf80d0780 con 0x7f0d0c105330 2024-04-24T09:21:05.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.568+0000 7f0d11800640 1 --2- 172.21.15.90:0/3702321849 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f0cf40a3bb0 0x7f0cf40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.568+0000 7f0d165f8740 1 -- 172.21.15.90:0/3151638696 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba08161c50 msgr2=0x55ba08166750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:05.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.568+0000 7f0d165f8740 1 --2- 172.21.15.90:0/3151638696 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba08161c50 0x55ba08166750 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0d0002a930 tx=0x7f0d000043f0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.569+0000 7f0d165f8740 1 -- 172.21.15.90:0/3151638696 shutdown_connections 2024-04-24T09:21:05.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.569+0000 7f0d165f8740 1 --2- 172.21.15.90:0/3151638696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba08166dc0 0x55ba08169250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.569+0000 7f0d165f8740 1 --2- 172.21.15.90:0/3151638696 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba08161c50 0x55ba08166750 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.569+0000 7f0d165f8740 1 --2- 172.21.15.90:0/3151638696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ba08161260 0x55ba08161680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.569+0000 7f0d165f8740 1 -- 172.21.15.90:0/3151638696 >> 172.21.15.90:0/3151638696 conn(0x55ba0815c8d0 msgr2=0x55ba0815ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:05.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.569+0000 7f0d165f8740 1 -- 172.21.15.90:0/3151638696 shutdown_connections 2024-04-24T09:21:05.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.569+0000 7f0d11800640 1 --2- 172.21.15.90:0/3702321849 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f0cf40a3bb0 0x7f0cf40a6070 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x55ba081624b0 tx=0x7f0d04009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:05.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.569+0000 7f0d165f8740 1 -- 172.21.15.90:0/3151638696 wait complete. 2024-04-24T09:21:05.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.569+0000 7f0d165f8740 1 -- start start 2024-04-24T09:21:05.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.570+0000 7f0d165f8740 1 Processor -- start 2024-04-24T09:21:05.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.570+0000 7f0d165f8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ba08161260 0x55ba0816b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.570+0000 7f0d165f8740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba08161c50 0x55ba0816bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.570+0000 7f0d165f8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba08166dc0 0x55ba08172c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.570+0000 7f0d165f8740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ba08169d10 con 0x55ba08161c50 2024-04-24T09:21:05.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.570+0000 7f0d165f8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55ba08080590 con 0x55ba08161260 2024-04-24T09:21:05.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.570+0000 7f0d165f8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ba08169e90 con 0x55ba08166dc0 2024-04-24T09:21:05.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.570+0000 7f0d165f8740 10 client.0 _ll_register_callbacks cb 0x55ba080633a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:21:05.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.570+0000 7f0d0be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba08161c50 0x55ba0816bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.570+0000 7f0d0be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba08161c50 0x55ba0816bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45604/0 (socket says 172.21.15.90:45604) 2024-04-24T09:21:05.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.570+0000 7f0d0be00640 1 -- 172.21.15.90:0/1669370376 learned_addr learned my addr 172.21.15.90:0/1669370376 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:05.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.570+0000 7f0d10e00640 1 --2- 172.21.15.90:0/1669370376 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ba08161260 0x55ba0816b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.570+0000 7f0d11800640 1 --2- 172.21.15.90:0/1669370376 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba08166dc0 0x55ba08172c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.571+0000 7f0d0be00640 1 -- 172.21.15.90:0/1669370376 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba08166dc0 msgr2=0x55ba08172c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:05.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.571+0000 7f0d0be00640 1 --2- 172.21.15.90:0/1669370376 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba08166dc0 0x55ba08172c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.571+0000 7f0d0be00640 1 -- 172.21.15.90:0/1669370376 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ba08161260 msgr2=0x55ba0816b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:05.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.571+0000 7f0d0be00640 1 --2- 172.21.15.90:0/1669370376 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ba08161260 0x55ba0816b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.571+0000 7f0d0be00640 1 -- 172.21.15.90:0/1669370376 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba07ea36b0 con 0x55ba08161c50 2024-04-24T09:21:05.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.571+0000 7f0d11800640 1 --2- 172.21.15.90:0/1669370376 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba08166dc0 0x55ba08172c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:21:05.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.571+0000 7f0d0be00640 1 --2- 172.21.15.90:0/1669370376 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba08161c50 0x55ba0816bc00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0cf809ca60 tx=0x7f0cf809c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ac8b06a504a8ff2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:05.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.571+0000 7f0cf2a00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-24T09:21:05.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.571+0000 7f0cf2a00640 1 -- 172.21.15.90:0/1669370376 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf80a5400 con 0x55ba08161c50 2024-04-24T09:21:05.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.571+0000 7f0cf2a00640 1 -- 172.21.15.90:0/1669370376 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f0cf80d0b70 con 0x55ba08161c50 2024-04-24T09:21:05.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.571+0000 7f0d165f8740 1 -- 172.21.15.90:0/1669370376 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d000047a0 con 0x55ba08161c50 2024-04-24T09:21:05.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.571+0000 7f0cf2a00640 1 -- 172.21.15.90:0/1669370376 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0cf80a5690 con 0x55ba08161c50 2024-04-24T09:21:05.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.571+0000 7f0d165f8740 10 client.14934 fetch_fsmap learned FSMap version 27 2024-04-24T09:21:05.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.571+0000 7f0d165f8740 1 -- 172.21.15.90:0/1669370376 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ba08176070 con 0x55ba08161c50 2024-04-24T09:21:05.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.572+0000 7f0cf2a00640 1 -- 172.21.15.90:0/1669370376 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d000047a0 con 0x55ba08161c50 2024-04-24T09:21:05.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.572+0000 7f0cf2a00640 1 -- 172.21.15.90:0/1669370376 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0cf80ba050 con 0x55ba08161c50 2024-04-24T09:21:05.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.572+0000 7f0d165f8740 10 client.14934 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T09:21:05.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.572+0000 7f0d165f8740 10 client.14934 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-24T09:21:05.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.572+0000 7f0d165f8740 20 client.14934 populate_metadata read hostname 'smithi090' 2024-04-24T09:21:05.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.572+0000 7f0d165f8740 1 --2- 172.21.15.90:0/1669370376 >> [v2:172.21.15.149:6838/652734263,v1:172.21.15.149:6839/652734263] conn(0x55ba081785d0 0x55ba08179a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.572+0000 7f0d165f8740 4 client.14934 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:21:05.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.572+0000 7f0d165f8740 1 -- 172.21.15.90:0/1669370376 --> [v2:172.21.15.149:6838/652734263,v1:172.21.15.149:6839/652734263] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba080d4530 con 0x55ba081785d0 2024-04-24T09:21:05.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.572+0000 7f0d10e00640 1 --2- 172.21.15.90:0/1669370376 >> [v2:172.21.15.149:6838/652734263,v1:172.21.15.149:6839/652734263] conn(0x55ba081785d0 0x55ba08179a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.573+0000 7f0d10e00640 1 --2- 172.21.15.90:0/1669370376 >> [v2:172.21.15.149:6838/652734263,v1:172.21.15.149:6839/652734263] conn(0x55ba081785d0 0x55ba08179a40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bc80e79b73ca53ff server_cookie=1e38dfa256e12d20 in_seq=0 out_seq=0 2024-04-24T09:21:05.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.573+0000 7f0cf2a00640 10 client.14934 ms_handle_connect on v2:172.21.15.149:6838/652734263 2024-04-24T09:21:05.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.574+0000 7f0cf2a00640 1 -- 172.21.15.90:0/1669370376 <== mds.1 v2:172.21.15.149:6838/652734263 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ba080d4530 con 0x55ba081785d0 2024-04-24T09:21:05.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.574+0000 7f0cf2a00640 10 client.14934 handle_command_reply: tid=0 2024-04-24T09:21:05.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.574+0000 7f0d165f8740 1 client.14934 shutdown 2024-04-24T09:21:05.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.574+0000 7f0d165f8740 1 -- 172.21.15.90:0/1669370376 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba08161c50 msgr2=0x55ba0816bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:05.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.574+0000 7f0d165f8740 1 --2- 172.21.15.90:0/1669370376 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba08161c50 0x55ba0816bc00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0cf809ca60 tx=0x7f0cf809c700 comp rx=0 tx=0).stop 2024-04-24T09:21:05.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.574+0000 7f0d165f8740 1 -- 172.21.15.90:0/1669370376 shutdown_connections 2024-04-24T09:21:05.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.574+0000 7f0d165f8740 1 --2- 172.21.15.90:0/1669370376 >> [v2:172.21.15.149:6838/652734263,v1:172.21.15.149:6839/652734263] conn(0x55ba081785d0 0x55ba08179a40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.574+0000 7f0d165f8740 1 --2- 172.21.15.90:0/1669370376 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba08166dc0 0x55ba08172c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.574+0000 7f0d165f8740 1 --2- 172.21.15.90:0/1669370376 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba08161c50 0x55ba0816bc00 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.574+0000 7f0d165f8740 1 --2- 172.21.15.90:0/1669370376 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ba08161260 0x55ba0816b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.574+0000 7f0cf2a00640 0 client.14934 ms_handle_reset on v2:172.21.15.149:6838/652734263 2024-04-24T09:21:05.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.574+0000 7f0d165f8740 1 -- 172.21.15.90:0/1669370376 >> 172.21.15.90:0/1669370376 conn(0x55ba0815c8d0 msgr2=0x55ba0815b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:05.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.574+0000 7f0d165f8740 1 -- 172.21.15.90:0/1669370376 shutdown_connections 2024-04-24T09:21:05.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.574+0000 7f0d165f8740 1 -- 172.21.15.90:0/1669370376 wait complete. 2024-04-24T09:21:05.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.574+0000 7f0d165f8740 20 client.14934 trim_cache size 0 max 16384 2024-04-24T09:21:05.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.594+0000 7f0d165f8740 1 Processor -- start 2024-04-24T09:21:05.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.594+0000 7f0d165f8740 1 -- start start 2024-04-24T09:21:05.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.594+0000 7f0d165f8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba08161260 0x55ba08169230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.594+0000 7f0d165f8740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba08161c50 0x55ba08172c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.594+0000 7f0d165f8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ba08166dc0 0x55ba0815fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.594+0000 7f0d165f8740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ba08177d80 con 0x55ba08161c50 2024-04-24T09:21:05.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.594+0000 7f0d165f8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55ba08178450 con 0x55ba08166dc0 2024-04-24T09:21:05.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.594+0000 7f0d165f8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ba08178b20 con 0x55ba08161260 2024-04-24T09:21:05.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.594+0000 7f0d0be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba08161c50 0x55ba08172c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.594+0000 7f0d0be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba08161c50 0x55ba08172c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45618/0 (socket says 172.21.15.90:45618) 2024-04-24T09:21:05.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.594+0000 7f0d0be00640 1 -- 172.21.15.90:0/2763149680 learned_addr learned my addr 172.21.15.90:0/2763149680 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:05.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.595+0000 7f0d10e00640 1 --2- 172.21.15.90:0/2763149680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba08161260 0x55ba08169230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.595+0000 7f0d11800640 1 --2- 172.21.15.90:0/2763149680 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ba08166dc0 0x55ba0815fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.595+0000 7f0d0be00640 1 -- 172.21.15.90:0/2763149680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba08161260 msgr2=0x55ba08169230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:05.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.595+0000 7f0d0be00640 1 --2- 172.21.15.90:0/2763149680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba08161260 0x55ba08169230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.595+0000 7f0d0be00640 1 -- 172.21.15.90:0/2763149680 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ba08166dc0 msgr2=0x55ba0815fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:05.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.595+0000 7f0d0be00640 1 --2- 172.21.15.90:0/2763149680 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ba08166dc0 0x55ba0815fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.595+0000 7f0d0be00640 1 -- 172.21.15.90:0/2763149680 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba0815c700 con 0x55ba08161c50 2024-04-24T09:21:05.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.595+0000 7f0d0be00640 1 --2- 172.21.15.90:0/2763149680 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba08161c50 0x55ba08172c00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8010480 tx=0x7f0cf80a0f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15edb431f9ec0499 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:05.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.595+0000 7f0cfd600640 1 -- 172.21.15.90:0/2763149680 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf809cad0 con 0x55ba08161c50 2024-04-24T09:21:05.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.595+0000 7f0cfd600640 1 -- 172.21.15.90:0/2763149680 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cf80a5200 con 0x55ba08161c50 2024-04-24T09:21:05.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.595+0000 7f0cfd600640 1 -- 172.21.15.90:0/2763149680 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf809b990 con 0x55ba08161c50 2024-04-24T09:21:05.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.596+0000 7f0d165f8740 1 -- 172.21.15.90:0/2763149680 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba08161c50 msgr2=0x55ba08172c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:05.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.596+0000 7f0d165f8740 1 --2- 172.21.15.90:0/2763149680 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba08161c50 0x55ba08172c00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8010480 tx=0x7f0cf80a0f50 comp rx=0 tx=0).stop 2024-04-24T09:21:05.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.596+0000 7f0d165f8740 1 -- 172.21.15.90:0/2763149680 shutdown_connections 2024-04-24T09:21:05.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.596+0000 7f0d165f8740 1 --2- 172.21.15.90:0/2763149680 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ba08166dc0 0x55ba0815fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.596+0000 7f0d165f8740 1 --2- 172.21.15.90:0/2763149680 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba08161c50 0x55ba08172c00 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.596+0000 7f0d165f8740 1 --2- 172.21.15.90:0/2763149680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba08161260 0x55ba08169230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.596+0000 7f0d165f8740 1 -- 172.21.15.90:0/2763149680 >> 172.21.15.90:0/2763149680 conn(0x55ba0815c8d0 msgr2=0x55ba0815f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:05.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.596+0000 7f0d165f8740 1 -- 172.21.15.90:0/2763149680 shutdown_connections 2024-04-24T09:21:05.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.596+0000 7f0d165f8740 1 -- 172.21.15.90:0/2763149680 wait complete. 2024-04-24T09:21:05.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.596+0000 7f0d165f8740 1 -- start start 2024-04-24T09:21:05.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.597+0000 7f0d165f8740 1 Processor -- start 2024-04-24T09:21:05.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.597+0000 7f0d165f8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba08161260 0x55ba0816ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.597+0000 7f0d165f8740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba08161c50 0x55ba0816f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.597+0000 7f0d165f8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ba08166dc0 0x55ba08171ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.597+0000 7f0d165f8740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ba081798c0 con 0x55ba08161c50 2024-04-24T09:21:05.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.597+0000 7f0d165f8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55ba081791f0 con 0x55ba08166dc0 2024-04-24T09:21:05.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.597+0000 7f0d165f8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ba08179f90 con 0x55ba08161260 2024-04-24T09:21:05.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.597+0000 7f0d165f8740 10 client.0 _ll_register_callbacks cb 0x7f0cf809c910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:21:05.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.597+0000 7f0d10e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba08161260 0x55ba0816ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.597+0000 7f0d10e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba08161260 0x55ba0816ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:38308/0 (socket says 172.21.15.90:38308) 2024-04-24T09:21:05.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.597+0000 7f0d10e00640 1 -- 172.21.15.90:0/4070390179 learned_addr learned my addr 172.21.15.90:0/4070390179 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:05.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.597+0000 7f0d0be00640 1 --2- 172.21.15.90:0/4070390179 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba08161c50 0x55ba0816f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.598+0000 7f0d11800640 1 --2- 172.21.15.90:0/4070390179 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ba08166dc0 0x55ba08171ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.598+0000 7f0d10e00640 1 -- 172.21.15.90:0/4070390179 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ba08166dc0 msgr2=0x55ba08171ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:05.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.598+0000 7f0d10e00640 1 --2- 172.21.15.90:0/4070390179 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ba08166dc0 0x55ba08171ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.598+0000 7f0d10e00640 1 -- 172.21.15.90:0/4070390179 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba08161c50 msgr2=0x55ba0816f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:05.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.598+0000 7f0d10e00640 1 --2- 172.21.15.90:0/4070390179 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba08161c50 0x55ba0816f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.598+0000 7f0d10e00640 1 -- 172.21.15.90:0/4070390179 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba081725e0 con 0x55ba08161260 2024-04-24T09:21:05.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.598+0000 7f0d0be00640 1 --2- 172.21.15.90:0/4070390179 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba08161c50 0x55ba0816f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:21:05.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.598+0000 7f0d11800640 1 --2- 172.21.15.90:0/4070390179 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ba08166dc0 0x55ba08171ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:21:05.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.598+0000 7f0d10e00640 1 --2- 172.21.15.90:0/4070390179 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba08161260 0x55ba0816ed90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0d00005ec0 tx=0x7f0d000045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed8fda394844054a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:05.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.598+0000 7f0cf2a00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-04-24T09:21:05.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.598+0000 7f0cf2a00640 1 -- 172.21.15.90:0/4070390179 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d00035040 con 0x55ba08161260 2024-04-24T09:21:05.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.598+0000 7f0d165f8740 1 -- 172.21.15.90:0/4070390179 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cf80a5200 con 0x55ba08161260 2024-04-24T09:21:05.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.598+0000 7f0cf2a00640 1 -- 172.21.15.90:0/4070390179 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f0d00031d70 con 0x55ba08161260 2024-04-24T09:21:05.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.599+0000 7f0cf2a00640 1 -- 172.21.15.90:0/4070390179 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0d00061050 con 0x55ba08161260 2024-04-24T09:21:05.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.599+0000 7f0d165f8740 10 client.25048 fetch_fsmap learned FSMap version 27 2024-04-24T09:21:05.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.599+0000 7f0d165f8740 1 -- 172.21.15.90:0/4070390179 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ba081ded20 con 0x55ba08161260 2024-04-24T09:21:05.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.599+0000 7f0cf2a00640 1 -- 172.21.15.90:0/4070390179 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cf80a5200 con 0x55ba08161260 2024-04-24T09:21:05.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.599+0000 7f0cf2a00640 1 -- 172.21.15.90:0/4070390179 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0d00003680 con 0x55ba08161260 2024-04-24T09:21:05.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.599+0000 7f0d165f8740 10 client.25048 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T09:21:05.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.599+0000 7f0d165f8740 10 client.25048 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-24T09:21:05.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.599+0000 7f0d165f8740 20 client.25048 populate_metadata read hostname 'smithi090' 2024-04-24T09:21:05.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.599+0000 7f0d165f8740 1 --2- 172.21.15.90:0/4070390179 >> [v2:172.21.15.149:6838/652734263,v1:172.21.15.149:6839/652734263] conn(0x55ba081e34b0 0x55ba081e21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.599+0000 7f0d165f8740 4 client.25048 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T09:21:05.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.599+0000 7f0d165f8740 1 -- 172.21.15.90:0/4070390179 --> [v2:172.21.15.149:6838/652734263,v1:172.21.15.149:6839/652734263] -- command(tid 0: {"prefix": "status"}) -- 0x55ba080ce3a0 con 0x55ba081e34b0 2024-04-24T09:21:05.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.600+0000 7f0d0be00640 1 --2- 172.21.15.90:0/4070390179 >> [v2:172.21.15.149:6838/652734263,v1:172.21.15.149:6839/652734263] conn(0x55ba081e34b0 0x55ba081e21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.600+0000 7f0d0be00640 1 --2- 172.21.15.90:0/4070390179 >> [v2:172.21.15.149:6838/652734263,v1:172.21.15.149:6839/652734263] conn(0x55ba081e34b0 0x55ba081e21c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1123f47dda8179a8 server_cookie=bda415f0c8ab7a61 in_seq=0 out_seq=0 2024-04-24T09:21:05.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.600+0000 7f0cf2a00640 10 client.25048 ms_handle_connect on v2:172.21.15.149:6838/652734263 2024-04-24T09:21:05.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.600+0000 7f0cf2a00640 1 -- 172.21.15.90:0/4070390179 <== mds.1 v2:172.21.15.149:6838/652734263 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ba080ce3a0 con 0x55ba081e34b0 2024-04-24T09:21:05.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.600+0000 7f0cf2a00640 10 client.25048 handle_command_reply: tid=0 2024-04-24T09:21:05.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.600+0000 7f0d165f8740 1 client.25048 shutdown 2024-04-24T09:21:05.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.600+0000 7f0d165f8740 1 -- 172.21.15.90:0/4070390179 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba08161260 msgr2=0x55ba0816ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:05.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.600+0000 7f0d165f8740 1 --2- 172.21.15.90:0/4070390179 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba08161260 0x55ba0816ed90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0d00005ec0 tx=0x7f0d000045e0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.601+0000 7f0d165f8740 1 -- 172.21.15.90:0/4070390179 shutdown_connections 2024-04-24T09:21:05.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.601+0000 7f0d165f8740 1 --2- 172.21.15.90:0/4070390179 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ba08166dc0 0x55ba08171ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.601+0000 7f0d165f8740 1 --2- 172.21.15.90:0/4070390179 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba08161c50 0x55ba0816f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.601+0000 7f0d165f8740 1 --2- 172.21.15.90:0/4070390179 >> [v2:172.21.15.149:6838/652734263,v1:172.21.15.149:6839/652734263] conn(0x55ba081e34b0 0x55ba081e21c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.601+0000 7f0d165f8740 1 --2- 172.21.15.90:0/4070390179 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba08161260 0x55ba0816ed90 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.601+0000 7f0d165f8740 1 -- 172.21.15.90:0/4070390179 >> 172.21.15.90:0/4070390179 conn(0x55ba0815c8d0 msgr2=0x55ba0816ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:05.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.601+0000 7f0cf2a00640 0 client.25048 ms_handle_reset on v2:172.21.15.149:6838/652734263 2024-04-24T09:21:05.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.601+0000 7f0d165f8740 1 -- 172.21.15.90:0/4070390179 shutdown_connections 2024-04-24T09:21:05.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.601+0000 7f0d165f8740 1 -- 172.21.15.90:0/4070390179 wait complete. 2024-04-24T09:21:05.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.601+0000 7f0d165f8740 20 client.25048 trim_cache size 0 max 16384 2024-04-24T09:21:05.606 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T09:21:05.606 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "30ee9be2-021a-11ef-bc93-c7b262605968", 2024-04-24T09:21:05.606 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 1, 2024-04-24T09:21:05.606 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 24547, 2024-04-24T09:21:05.606 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-04-24T09:21:05.606 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-04-24T09:21:05.606 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-04-24T09:21:05.606 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 112.33679615299999, 2024-04-24T09:21:05.606 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 27, 2024-04-24T09:21:05.607 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 88, 2024-04-24T09:21:05.607 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 86, 2024-04-24T09:21:05.607 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 133.14755100299999 2024-04-24T09:21:05.607 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T09:21:05.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.601+0000 7f0d12200640 1 -- 172.21.15.90:0/3702321849 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f0cf40a3bb0 msgr2=0x7f0cf40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:05.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.601+0000 7f0d12200640 1 --2- 172.21.15.90:0/3702321849 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f0cf40a3bb0 0x7f0cf40a6070 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x55ba081624b0 tx=0x7f0d04009500 comp rx=0 tx=0).stop 2024-04-24T09:21:05.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.601+0000 7f0d12200640 1 -- 172.21.15.90:0/3702321849 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d0c105330 msgr2=0x7f0d0c19a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:05.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.601+0000 7f0d12200640 1 --2- 172.21.15.90:0/3702321849 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d0c105330 0x7f0d0c19a960 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0cf800e9f0 tx=0x7f0cf800eeb0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.601+0000 7f0d12200640 1 -- 172.21.15.90:0/3702321849 shutdown_connections 2024-04-24T09:21:05.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.602+0000 7f0d12200640 1 --2- 172.21.15.90:0/3702321849 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f0cf40a3bb0 0x7f0cf40a6070 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.602+0000 7f0d12200640 1 --2- 172.21.15.90:0/3702321849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0d0c105cf0 0x7f0d0c1a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.602+0000 7f0d12200640 1 --2- 172.21.15.90:0/3702321849 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d0c105330 0x7f0d0c19a960 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.602+0000 7f0d12200640 1 --2- 172.21.15.90:0/3702321849 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d0c104140 0x7f0d0c19a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.602+0000 7f0d12200640 1 -- 172.21.15.90:0/3702321849 >> 172.21.15.90:0/3702321849 conn(0x7f0d0c0ff6b0 msgr2=0x7f0d0c1085d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:05.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.602+0000 7f0d12200640 1 -- 172.21.15.90:0/3702321849 shutdown_connections 2024-04-24T09:21:05.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.602+0000 7f0d12200640 1 -- 172.21.15.90:0/3702321849 wait complete. 2024-04-24T09:21:05.617 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-24T09:21:05.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.814+0000 7fac0a800640 1 -- 172.21.15.90:0/4164293852 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac040ffb60 msgr2=0x7fac040fff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:05.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.814+0000 7fac0a800640 1 --2- 172.21.15.90:0/4164293852 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac040ffb60 0x7fac040fff60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fabf8009900 tx=0x7fabf802a3f0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.814+0000 7fac0a800640 1 -- 172.21.15.90:0/4164293852 shutdown_connections 2024-04-24T09:21:05.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.814+0000 7fac0a800640 1 --2- 172.21.15.90:0/4164293852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac04107c70 0x7fac0410a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.814+0000 7fac0a800640 1 --2- 172.21.15.90:0/4164293852 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac04100530 0x7fac04107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.814+0000 7fac0a800640 1 --2- 172.21.15.90:0/4164293852 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac040ffb60 0x7fac040fff60 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.814+0000 7fac0a800640 1 -- 172.21.15.90:0/4164293852 >> 172.21.15.90:0/4164293852 conn(0x7fac040fb6c0 msgr2=0x7fac040fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:05.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.814+0000 7fac0a800640 1 -- 172.21.15.90:0/4164293852 shutdown_connections 2024-04-24T09:21:05.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.814+0000 7fac0a800640 1 -- 172.21.15.90:0/4164293852 wait complete. 2024-04-24T09:21:05.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.816+0000 7fac0a800640 1 Processor -- start 2024-04-24T09:21:05.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.816+0000 7fac0a800640 1 -- start start 2024-04-24T09:21:05.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.816+0000 7fac0a800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac040ffb60 0x7fac0419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.816+0000 7fac0a800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac04100530 0x7fac0419a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.816+0000 7fac0a800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac04107c70 0x7fac041a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.816+0000 7fac0a800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fac0410dc00 con 0x7fac04107c70 2024-04-24T09:21:05.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.816+0000 7fac0a800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fac0410da80 con 0x7fac040ffb60 2024-04-24T09:21:05.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.816+0000 7fac0a800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fac0410dd80 con 0x7fac04100530 2024-04-24T09:21:05.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.817+0000 7fac08a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac04100530 0x7fac0419a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.817+0000 7fac09e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac04107c70 0x7fac041a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.817+0000 7fac09400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac040ffb60 0x7fac0419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.817+0000 7fac09400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac040ffb60 0x7fac0419a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.90:58168/0 (socket says 172.21.15.90:58168) 2024-04-24T09:21:05.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.817+0000 7fac09e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac04107c70 0x7fac041a1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40634/0 (socket says 172.21.15.90:40634) 2024-04-24T09:21:05.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.817+0000 7fac09400640 1 -- 172.21.15.90:0/3148251316 learned_addr learned my addr 172.21.15.90:0/3148251316 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:05.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.817+0000 7fac08a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac04100530 0x7fac0419a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:36194/0 (socket says 172.21.15.90:36194) 2024-04-24T09:21:05.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.817+0000 7fac08a00640 1 -- 172.21.15.90:0/3148251316 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac040ffb60 msgr2=0x7fac0419a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:05.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.817+0000 7fac08a00640 1 --2- 172.21.15.90:0/3148251316 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac040ffb60 0x7fac0419a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.817+0000 7fac08a00640 1 -- 172.21.15.90:0/3148251316 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac04107c70 msgr2=0x7fac041a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:05.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.817+0000 7fac08a00640 1 --2- 172.21.15.90:0/3148251316 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac04107c70 0x7fac041a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.817+0000 7fac08a00640 1 -- 172.21.15.90:0/3148251316 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac041a20b0 con 0x7fac04100530 2024-04-24T09:21:05.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.817+0000 7fac09400640 1 --2- 172.21.15.90:0/3148251316 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac040ffb60 0x7fac0419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:21:05.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.817+0000 7fac09e00640 1 --2- 172.21.15.90:0/3148251316 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac04107c70 0x7fac041a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:21:05.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.817+0000 7fac08a00640 1 --2- 172.21.15.90:0/3148251316 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac04100530 0x7fac0419a9d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fabf000e9c0 tx=0x7fabf000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:05.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.818+0000 7fac02000640 1 -- 172.21.15.90:0/3148251316 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf000ccc0 con 0x7fac04100530 2024-04-24T09:21:05.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.818+0000 7fac0a800640 1 -- 172.21.15.90:0/3148251316 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac041a23a0 con 0x7fac04100530 2024-04-24T09:21:05.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.818+0000 7fac02000640 1 -- 172.21.15.90:0/3148251316 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabf00035f0 con 0x7fac04100530 2024-04-24T09:21:05.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.818+0000 7fac02000640 1 -- 172.21.15.90:0/3148251316 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf000ce60 con 0x7fac04100530 2024-04-24T09:21:05.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.818+0000 7fac0a800640 1 -- 172.21.15.90:0/3148251316 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac041a2940 con 0x7fac04100530 2024-04-24T09:21:05.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.821+0000 7fac02000640 1 -- 172.21.15.90:0/3148251316 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fabf00037a0 con 0x7fac04100530 2024-04-24T09:21:05.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.821+0000 7fac0eb84740 1 Processor -- start 2024-04-24T09:21:05.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.821+0000 7fac0eb84740 1 -- start start 2024-04-24T09:21:05.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.822+0000 7fac0eb84740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e38253260 0x555e38253680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.822+0000 7fac0eb84740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e38253c50 0x555e38258750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.822+0000 7fac0eb84740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e38258dc0 0x555e3825b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.822+0000 7fac0eb84740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x555e381b8b10 con 0x555e38253c50 2024-04-24T09:21:05.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.822+0000 7fac0eb84740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x555e3815f2b0 con 0x555e38253260 2024-04-24T09:21:05.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.822+0000 7fac0eb84740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x555e381b4ad0 con 0x555e38258dc0 2024-04-24T09:21:05.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.822+0000 7fac09e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e38253260 0x555e38253680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.822+0000 7fac09e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e38253260 0x555e38253680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.90:58180/0 (socket says 172.21.15.90:58180) 2024-04-24T09:21:05.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.822+0000 7fac09e00640 1 -- 172.21.15.90:0/263298843 learned_addr learned my addr 172.21.15.90:0/263298843 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:05.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.822+0000 7fac09400640 1 --2- 172.21.15.90:0/263298843 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e38253c50 0x555e38258750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.822+0000 7fac08a00640 1 --2- 172.21.15.90:0/263298843 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e38258dc0 0x555e3825b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.822+0000 7fac09e00640 1 -- 172.21.15.90:0/263298843 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e38258dc0 msgr2=0x555e3825b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:05.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.822+0000 7fac09e00640 1 --2- 172.21.15.90:0/263298843 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e38258dc0 0x555e3825b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.823+0000 7fac09e00640 1 -- 172.21.15.90:0/263298843 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e38253c50 msgr2=0x555e38258750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:05.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.823+0000 7fac09e00640 1 --2- 172.21.15.90:0/263298843 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e38253c50 0x555e38258750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.823+0000 7fac09e00640 1 -- 172.21.15.90:0/263298843 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e37ecb2d0 con 0x555e38253260 2024-04-24T09:21:05.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.823+0000 7fac09400640 1 --2- 172.21.15.90:0/263298843 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e38253c50 0x555e38258750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:21:05.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.823+0000 7fac02000640 1 --2- 172.21.15.90:0/3148251316 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fabec0a3bb0 0x7fabec0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.823+0000 7fac02000640 1 -- 172.21.15.90:0/3148251316 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fabf00d0830 con 0x7fac04100530 2024-04-24T09:21:05.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.823+0000 7fac09400640 1 --2- 172.21.15.90:0/3148251316 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fabec0a3bb0 0x7fabec0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.823+0000 7fac09e00640 1 --2- 172.21.15.90:0/263298843 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e38253260 0x555e38253680 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fabfc0117f0 tx=0x7fabfc011cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6521334539084f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:05.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.823+0000 7fabf6600640 1 -- 172.21.15.90:0/263298843 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabfc0112f0 con 0x555e38253260 2024-04-24T09:21:05.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.823+0000 7fabf6600640 1 -- 172.21.15.90:0/263298843 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabfc003e50 con 0x555e38253260 2024-04-24T09:21:05.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.823+0000 7fabf6600640 1 -- 172.21.15.90:0/263298843 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabfc0045f0 con 0x555e38253260 2024-04-24T09:21:05.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.824+0000 7fac0eb84740 1 -- 172.21.15.90:0/263298843 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e38253260 msgr2=0x555e38253680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:05.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.824+0000 7fac0eb84740 1 --2- 172.21.15.90:0/263298843 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e38253260 0x555e38253680 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fabfc0117f0 tx=0x7fabfc011cb0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.824+0000 7fac09400640 1 --2- 172.21.15.90:0/3148251316 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fabec0a3bb0 0x7fabec0a6070 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fabf8000c00 tx=0x7fabf8005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:05.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.824+0000 7fac0eb84740 1 -- 172.21.15.90:0/263298843 shutdown_connections 2024-04-24T09:21:05.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.824+0000 7fac0eb84740 1 --2- 172.21.15.90:0/263298843 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e38258dc0 0x555e3825b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.824+0000 7fac0eb84740 1 --2- 172.21.15.90:0/263298843 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e38253c50 0x555e38258750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.824+0000 7fac0eb84740 1 --2- 172.21.15.90:0/263298843 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e38253260 0x555e38253680 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.824+0000 7fac0eb84740 1 -- 172.21.15.90:0/263298843 >> 172.21.15.90:0/263298843 conn(0x555e3824e8d0 msgr2=0x555e3824ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:05.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.824+0000 7fac0eb84740 1 -- 172.21.15.90:0/263298843 shutdown_connections 2024-04-24T09:21:05.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.824+0000 7fac0eb84740 1 -- 172.21.15.90:0/263298843 wait complete. 2024-04-24T09:21:05.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.825+0000 7fac0eb84740 1 -- start start 2024-04-24T09:21:05.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.825+0000 7fac0eb84740 1 Processor -- start 2024-04-24T09:21:05.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.825+0000 7fac0eb84740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e38253260 0x555e3825d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.825+0000 7fac0eb84740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e38253c50 0x555e3825dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.825+0000 7fac0eb84740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e38258dc0 0x555e38264c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.825+0000 7fac0eb84740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x555e3825bd70 con 0x555e38253260 2024-04-24T09:21:05.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.825+0000 7fac0eb84740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x555e38172590 con 0x555e38253c50 2024-04-24T09:21:05.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.825+0000 7fac0eb84740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x555e3825bef0 con 0x555e38258dc0 2024-04-24T09:21:05.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.825+0000 7fac0eb84740 10 client.0 _ll_register_callbacks cb 0x555e381553a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:21:05.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.825+0000 7fac09400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e38253260 0x555e3825d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.826+0000 7fac09400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e38253260 0x555e3825d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40658/0 (socket says 172.21.15.90:40658) 2024-04-24T09:21:05.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.826+0000 7fac09400640 1 -- 172.21.15.90:0/2884047234 learned_addr learned my addr 172.21.15.90:0/2884047234 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:05.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.826+0000 7fac08a00640 1 --2- 172.21.15.90:0/2884047234 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e38253c50 0x555e3825dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.826+0000 7fac09e00640 1 --2- 172.21.15.90:0/2884047234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e38258dc0 0x555e38264c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.826+0000 7fac08a00640 1 -- 172.21.15.90:0/2884047234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e38258dc0 msgr2=0x555e38264c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:05.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.826+0000 7fac08a00640 1 --2- 172.21.15.90:0/2884047234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e38258dc0 0x555e38264c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.826+0000 7fac08a00640 1 -- 172.21.15.90:0/2884047234 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e38253260 msgr2=0x555e3825d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:05.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.826+0000 7fac08a00640 1 --2- 172.21.15.90:0/2884047234 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e38253260 0x555e3825d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.826+0000 7fac08a00640 1 -- 172.21.15.90:0/2884047234 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e37f956b0 con 0x555e38253c50 2024-04-24T09:21:05.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.826+0000 7fac09e00640 1 --2- 172.21.15.90:0/2884047234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e38258dc0 0x555e38264c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:21:05.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.826+0000 7fac08a00640 1 --2- 172.21.15.90:0/2884047234 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e38253c50 0x555e3825dc80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fabf009ca60 tx=0x7fabf0098d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f06b355a5b6a367 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:05.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.826+0000 7fabeaa00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-04-24T09:21:05.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.826+0000 7fabeaa00640 1 -- 172.21.15.90:0/2884047234 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf009c430 con 0x555e38253c50 2024-04-24T09:21:05.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.826+0000 7fabeaa00640 1 -- 172.21.15.90:0/2884047234 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fabf00ce400 con 0x555e38253c50 2024-04-24T09:21:05.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.827+0000 7fac0eb84740 1 -- 172.21.15.90:0/2884047234 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabfc003e50 con 0x555e38253c50 2024-04-24T09:21:05.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.827+0000 7fabeaa00640 1 -- 172.21.15.90:0/2884047234 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fabf009d1d0 con 0x555e38253c50 2024-04-24T09:21:05.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.827+0000 7fac0eb84740 10 client.25037 fetch_fsmap learned FSMap version 27 2024-04-24T09:21:05.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.827+0000 7fac0eb84740 1 -- 172.21.15.90:0/2884047234 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555e38268030 con 0x555e38253c50 2024-04-24T09:21:05.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.827+0000 7fabeaa00640 1 -- 172.21.15.90:0/2884047234 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabfc003e50 con 0x555e38253c50 2024-04-24T09:21:05.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.827+0000 7fabeaa00640 1 -- 172.21.15.90:0/2884047234 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fabf009bf20 con 0x555e38253c50 2024-04-24T09:21:05.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.827+0000 7fac0eb84740 10 client.25037 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T09:21:05.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.827+0000 7fac0eb84740 10 client.25037 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T09:21:05.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.827+0000 7fac0eb84740 20 client.25037 populate_metadata read hostname 'smithi090' 2024-04-24T09:21:05.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.828+0000 7fac0eb84740 1 --2- 172.21.15.90:0/2884047234 >> [v2:172.21.15.114:6838/1025403710,v1:172.21.15.114:6839/1025403710] conn(0x555e3826a5c0 0x555e3826ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.828+0000 7fac0eb84740 4 client.25037 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:21:05.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.828+0000 7fac0eb84740 1 -- 172.21.15.90:0/2884047234 --> [v2:172.21.15.114:6838/1025403710,v1:172.21.15.114:6839/1025403710] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e381c6530 con 0x555e3826a5c0 2024-04-24T09:21:05.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.828+0000 7fac09400640 1 --2- 172.21.15.90:0/2884047234 >> [v2:172.21.15.114:6838/1025403710,v1:172.21.15.114:6839/1025403710] conn(0x555e3826a5c0 0x555e3826ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.829+0000 7fac09400640 1 --2- 172.21.15.90:0/2884047234 >> [v2:172.21.15.114:6838/1025403710,v1:172.21.15.114:6839/1025403710] conn(0x555e3826a5c0 0x555e3826ba30 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=180759acb503cd4f server_cookie=3506e1e1d994ab05 in_seq=0 out_seq=0 2024-04-24T09:21:05.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.829+0000 7fabeaa00640 10 client.25037 ms_handle_connect on v2:172.21.15.114:6838/1025403710 2024-04-24T09:21:05.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.830+0000 7fabeaa00640 1 -- 172.21.15.90:0/2884047234 <== mds.0 v2:172.21.15.114:6838/1025403710 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555e381c6530 con 0x555e3826a5c0 2024-04-24T09:21:05.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.830+0000 7fabeaa00640 10 client.25037 handle_command_reply: tid=0 2024-04-24T09:21:05.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.830+0000 7fac0eb84740 1 client.25037 shutdown 2024-04-24T09:21:05.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.830+0000 7fac0eb84740 1 -- 172.21.15.90:0/2884047234 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e38253c50 msgr2=0x555e3825dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:05.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.830+0000 7fac0eb84740 1 --2- 172.21.15.90:0/2884047234 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e38253c50 0x555e3825dc80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fabf009ca60 tx=0x7fabf0098d70 comp rx=0 tx=0).stop 2024-04-24T09:21:05.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.830+0000 7fac0eb84740 1 -- 172.21.15.90:0/2884047234 shutdown_connections 2024-04-24T09:21:05.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.830+0000 7fac0eb84740 1 --2- 172.21.15.90:0/2884047234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e38258dc0 0x555e38264c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.830+0000 7fac0eb84740 1 --2- 172.21.15.90:0/2884047234 >> [v2:172.21.15.114:6838/1025403710,v1:172.21.15.114:6839/1025403710] conn(0x555e3826a5c0 0x555e3826ba30 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.830+0000 7fac0eb84740 1 --2- 172.21.15.90:0/2884047234 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e38253c50 0x555e3825dc80 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.830+0000 7fac0eb84740 1 --2- 172.21.15.90:0/2884047234 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e38253260 0x555e3825d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.830+0000 7fabeaa00640 0 client.25037 ms_handle_reset on v2:172.21.15.114:6838/1025403710 2024-04-24T09:21:05.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.830+0000 7fac0eb84740 1 -- 172.21.15.90:0/2884047234 >> 172.21.15.90:0/2884047234 conn(0x555e3824e8d0 msgr2=0x555e3824d970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:05.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.830+0000 7fac0eb84740 1 -- 172.21.15.90:0/2884047234 shutdown_connections 2024-04-24T09:21:05.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.830+0000 7fac0eb84740 1 -- 172.21.15.90:0/2884047234 wait complete. 2024-04-24T09:21:05.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.830+0000 7fac0eb84740 20 client.25037 trim_cache size 0 max 16384 2024-04-24T09:21:05.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.850+0000 7fac0eb84740 1 Processor -- start 2024-04-24T09:21:05.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.850+0000 7fac0eb84740 1 -- start start 2024-04-24T09:21:05.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.851+0000 7fac0eb84740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e38253260 0x555e3825b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.851+0000 7fac0eb84740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e38253c50 0x555e38264c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.851+0000 7fac0eb84740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e38258dc0 0x555e3824d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.851+0000 7fac0eb84740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x555e38269d70 con 0x555e38258dc0 2024-04-24T09:21:05.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.851+0000 7fac0eb84740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x555e3826a440 con 0x555e38253c50 2024-04-24T09:21:05.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.851+0000 7fac0eb84740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x555e3826ab10 con 0x555e38253260 2024-04-24T09:21:05.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.851+0000 7fac09e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e38258dc0 0x555e3824d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.851+0000 7fac09e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e38258dc0 0x555e3824d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40672/0 (socket says 172.21.15.90:40672) 2024-04-24T09:21:05.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.851+0000 7fac09e00640 1 -- 172.21.15.90:0/2588680775 learned_addr learned my addr 172.21.15.90:0/2588680775 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:05.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.851+0000 7fac09400640 1 --2- 172.21.15.90:0/2588680775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e38253260 0x555e3825b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.851+0000 7fac08a00640 1 --2- 172.21.15.90:0/2588680775 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e38253c50 0x555e38264c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.851+0000 7fac09e00640 1 -- 172.21.15.90:0/2588680775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e38253260 msgr2=0x555e3825b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:05.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.851+0000 7fac09e00640 1 --2- 172.21.15.90:0/2588680775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e38253260 0x555e3825b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.851+0000 7fac09e00640 1 -- 172.21.15.90:0/2588680775 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e38253c50 msgr2=0x555e38264c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:05.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.851+0000 7fac09e00640 1 --2- 172.21.15.90:0/2588680775 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e38253c50 0x555e38264c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.852+0000 7fac09e00640 1 -- 172.21.15.90:0/2588680775 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e3826afc0 con 0x555e38258dc0 2024-04-24T09:21:05.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.852+0000 7fac09400640 1 --2- 172.21.15.90:0/2588680775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e38253260 0x555e3825b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:21:05.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.852+0000 7fac09e00640 1 --2- 172.21.15.90:0/2588680775 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e38258dc0 0x555e3824d0e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fabfc019000 tx=0x7fabfc003e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d56cdfdc68802f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:05.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.852+0000 7fabf5c00640 1 -- 172.21.15.90:0/2588680775 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabfc02b070 con 0x555e38258dc0 2024-04-24T09:21:05.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.852+0000 7fabf5c00640 1 -- 172.21.15.90:0/2588680775 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabfc00fe80 con 0x555e38258dc0 2024-04-24T09:21:05.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.852+0000 7fabf5c00640 1 -- 172.21.15.90:0/2588680775 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabfc0058c0 con 0x555e38258dc0 2024-04-24T09:21:05.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.853+0000 7fac0eb84740 1 -- 172.21.15.90:0/2588680775 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e38258dc0 msgr2=0x555e3824d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:05.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.853+0000 7fac0eb84740 1 --2- 172.21.15.90:0/2588680775 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e38258dc0 0x555e3824d0e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fabfc019000 tx=0x7fabfc003e10 comp rx=0 tx=0).stop 2024-04-24T09:21:05.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.853+0000 7fac0eb84740 1 -- 172.21.15.90:0/2588680775 shutdown_connections 2024-04-24T09:21:05.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.853+0000 7fac0eb84740 1 --2- 172.21.15.90:0/2588680775 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e38258dc0 0x555e3824d0e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.853+0000 7fac0eb84740 1 --2- 172.21.15.90:0/2588680775 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e38253c50 0x555e38264c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.853+0000 7fac0eb84740 1 --2- 172.21.15.90:0/2588680775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e38253260 0x555e3825b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.853+0000 7fac0eb84740 1 -- 172.21.15.90:0/2588680775 >> 172.21.15.90:0/2588680775 conn(0x555e3824e8d0 msgr2=0x555e38255120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:05.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.853+0000 7fac0eb84740 1 -- 172.21.15.90:0/2588680775 shutdown_connections 2024-04-24T09:21:05.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.853+0000 7fac0eb84740 1 -- 172.21.15.90:0/2588680775 wait complete. 2024-04-24T09:21:05.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.853+0000 7fac0eb84740 1 -- start start 2024-04-24T09:21:05.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.853+0000 7fac0eb84740 1 Processor -- start 2024-04-24T09:21:05.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.854+0000 7fac0eb84740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e38253260 0x555e38260620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.854+0000 7fac0eb84740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e38253c50 0x555e38260b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.854+0000 7fac0eb84740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e38258dc0 0x555e38263b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.854+0000 7fac0eb84740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x555e3826b8b0 con 0x555e38253c50 2024-04-24T09:21:05.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.854+0000 7fac0eb84740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x555e3826b1e0 con 0x555e38253260 2024-04-24T09:21:05.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.854+0000 7fac0eb84740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x555e3826bf80 con 0x555e38258dc0 2024-04-24T09:21:05.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.854+0000 7fac0eb84740 10 client.0 _ll_register_callbacks cb 0x7fabf0004560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:21:05.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.854+0000 7fac08a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e38253c50 0x555e38260b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.854+0000 7fac08a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e38253c50 0x555e38260b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40676/0 (socket says 172.21.15.90:40676) 2024-04-24T09:21:05.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.854+0000 7fac08a00640 1 -- 172.21.15.90:0/2215219636 learned_addr learned my addr 172.21.15.90:0/2215219636 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:05.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.854+0000 7fac09400640 1 --2- 172.21.15.90:0/2215219636 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e38253260 0x555e38260620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.854+0000 7fac09e00640 1 --2- 172.21.15.90:0/2215219636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e38258dc0 0x555e38263b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.855+0000 7fac09400640 1 -- 172.21.15.90:0/2215219636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e38258dc0 msgr2=0x555e38263b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:05.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.855+0000 7fac09400640 1 --2- 172.21.15.90:0/2215219636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e38258dc0 0x555e38263b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.855+0000 7fac09400640 1 -- 172.21.15.90:0/2215219636 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e38253c50 msgr2=0x555e38260b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:05.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.855+0000 7fac09400640 1 --2- 172.21.15.90:0/2215219636 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e38253c50 0x555e38260b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.855+0000 7fac09400640 1 -- 172.21.15.90:0/2215219636 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e38264170 con 0x555e38253260 2024-04-24T09:21:05.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.855+0000 7fac08a00640 1 --2- 172.21.15.90:0/2215219636 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e38253c50 0x555e38260b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:21:05.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.855+0000 7fac09400640 1 --2- 172.21.15.90:0/2215219636 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e38253260 0x555e38260620 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x555e3826c3c0 tx=0x7fabf8009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc6669f59dd6f5ad server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:05.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.855+0000 7fabeaa00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-04-24T09:21:05.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.855+0000 7fabeaa00640 1 -- 172.21.15.90:0/2215219636 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf8035040 con 0x555e38253260 2024-04-24T09:21:05.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.855+0000 7fac0eb84740 1 -- 172.21.15.90:0/2215219636 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555e381c03a0 con 0x555e38253260 2024-04-24T09:21:05.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.855+0000 7fabeaa00640 1 -- 172.21.15.90:0/2215219636 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fabf8054890 con 0x555e38253260 2024-04-24T09:21:05.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.855+0000 7fabeaa00640 1 -- 172.21.15.90:0/2215219636 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fabf8063050 con 0x555e38253260 2024-04-24T09:21:05.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.855+0000 7fac0eb84740 10 client.25043 fetch_fsmap learned FSMap version 27 2024-04-24T09:21:05.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.855+0000 7fac0eb84740 1 -- 172.21.15.90:0/2215219636 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555e382d0900 con 0x555e38253260 2024-04-24T09:21:05.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.856+0000 7fabeaa00640 1 -- 172.21.15.90:0/2215219636 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x555e381c03a0 con 0x555e38253260 2024-04-24T09:21:05.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.856+0000 7fabeaa00640 1 -- 172.21.15.90:0/2215219636 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fabf8044070 con 0x555e38253260 2024-04-24T09:21:05.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.856+0000 7fac0eb84740 10 client.25043 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T09:21:05.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.856+0000 7fac0eb84740 10 client.25043 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T09:21:05.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.856+0000 7fac0eb84740 20 client.25043 populate_metadata read hostname 'smithi090' 2024-04-24T09:21:05.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.856+0000 7fac0eb84740 1 --2- 172.21.15.90:0/2215219636 >> [v2:172.21.15.114:6838/1025403710,v1:172.21.15.114:6839/1025403710] conn(0x555e382d5480 0x555e382d4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:05.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.856+0000 7fac0eb84740 4 client.25043 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T09:21:05.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.856+0000 7fac0eb84740 1 -- 172.21.15.90:0/2215219636 --> [v2:172.21.15.114:6838/1025403710,v1:172.21.15.114:6839/1025403710] -- command(tid 0: {"prefix": "status"}) -- 0x555e3813c860 con 0x555e382d5480 2024-04-24T09:21:05.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.856+0000 7fac08a00640 1 --2- 172.21.15.90:0/2215219636 >> [v2:172.21.15.114:6838/1025403710,v1:172.21.15.114:6839/1025403710] conn(0x555e382d5480 0x555e382d4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:05.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.857+0000 7fac08a00640 1 --2- 172.21.15.90:0/2215219636 >> [v2:172.21.15.114:6838/1025403710,v1:172.21.15.114:6839/1025403710] conn(0x555e382d5480 0x555e382d4190 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f985c87d968f4a server_cookie=13f03e46b350aba1 in_seq=0 out_seq=0 2024-04-24T09:21:05.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.857+0000 7fabeaa00640 10 client.25043 ms_handle_connect on v2:172.21.15.114:6838/1025403710 2024-04-24T09:21:05.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.857+0000 7fabeaa00640 1 -- 172.21.15.90:0/2215219636 <== mds.0 v2:172.21.15.114:6838/1025403710 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x555e3813c860 con 0x555e382d5480 2024-04-24T09:21:05.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.857+0000 7fabeaa00640 10 client.25043 handle_command_reply: tid=0 2024-04-24T09:21:05.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.857+0000 7fac0eb84740 1 client.25043 shutdown 2024-04-24T09:21:05.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.857+0000 7fac0eb84740 1 -- 172.21.15.90:0/2215219636 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e38253260 msgr2=0x555e38260620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:05.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.857+0000 7fac0eb84740 1 --2- 172.21.15.90:0/2215219636 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e38253260 0x555e38260620 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x555e3826c3c0 tx=0x7fabf8009260 comp rx=0 tx=0).stop 2024-04-24T09:21:05.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.857+0000 7fac0eb84740 1 -- 172.21.15.90:0/2215219636 shutdown_connections 2024-04-24T09:21:05.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.857+0000 7fac0eb84740 1 --2- 172.21.15.90:0/2215219636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e38258dc0 0x555e38263b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.857+0000 7fac0eb84740 1 --2- 172.21.15.90:0/2215219636 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e38253c50 0x555e38260b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.857+0000 7fac0eb84740 1 --2- 172.21.15.90:0/2215219636 >> [v2:172.21.15.114:6838/1025403710,v1:172.21.15.114:6839/1025403710] conn(0x555e382d5480 0x555e382d4190 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.857+0000 7fac0eb84740 1 --2- 172.21.15.90:0/2215219636 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e38253260 0x555e38260620 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.857+0000 7fac0eb84740 1 -- 172.21.15.90:0/2215219636 >> 172.21.15.90:0/2215219636 conn(0x555e3824e8d0 msgr2=0x555e38251670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:05.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.857+0000 7fabeaa00640 0 client.25043 ms_handle_reset on v2:172.21.15.114:6838/1025403710 2024-04-24T09:21:05.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.858+0000 7fac0eb84740 1 -- 172.21.15.90:0/2215219636 shutdown_connections 2024-04-24T09:21:05.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.858+0000 7fac0eb84740 1 -- 172.21.15.90:0/2215219636 wait complete. 2024-04-24T09:21:05.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.858+0000 7fac0eb84740 20 client.25043 trim_cache size 0 max 16384 2024-04-24T09:21:05.862 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T09:21:05.862 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "30ee9be2-021a-11ef-bc93-c7b262605968", 2024-04-24T09:21:05.862 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 0, 2024-04-24T09:21:05.862 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 24557, 2024-04-24T09:21:05.862 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-04-24T09:21:05.862 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-04-24T09:21:05.862 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-04-24T09:21:05.862 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 115.913924408, 2024-04-24T09:21:05.862 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 27, 2024-04-24T09:21:05.863 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 88, 2024-04-24T09:21:05.863 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 86, 2024-04-24T09:21:05.863 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 122.05691967600001 2024-04-24T09:21:05.863 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T09:21:05.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.858+0000 7fac0a800640 1 -- 172.21.15.90:0/3148251316 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fabec0a3bb0 msgr2=0x7fabec0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:05.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.858+0000 7fac0a800640 1 --2- 172.21.15.90:0/3148251316 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fabec0a3bb0 0x7fabec0a6070 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fabf8000c00 tx=0x7fabf8005a20 comp rx=0 tx=0).stop 2024-04-24T09:21:05.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.858+0000 7fac0a800640 1 -- 172.21.15.90:0/3148251316 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac04100530 msgr2=0x7fac0419a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:05.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.858+0000 7fac0a800640 1 --2- 172.21.15.90:0/3148251316 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac04100530 0x7fac0419a9d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fabf000e9c0 tx=0x7fabf000ee80 comp rx=0 tx=0).stop 2024-04-24T09:21:05.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.858+0000 7fac0a800640 1 -- 172.21.15.90:0/3148251316 shutdown_connections 2024-04-24T09:21:05.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.858+0000 7fac0a800640 1 --2- 172.21.15.90:0/3148251316 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fabec0a3bb0 0x7fabec0a6070 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.858+0000 7fac0a800640 1 --2- 172.21.15.90:0/3148251316 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac04107c70 0x7fac041a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.858+0000 7fac0a800640 1 --2- 172.21.15.90:0/3148251316 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac04100530 0x7fac0419a9d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.858+0000 7fac0a800640 1 --2- 172.21.15.90:0/3148251316 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fac040ffb60 0x7fac0419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:05.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.858+0000 7fac0a800640 1 -- 172.21.15.90:0/3148251316 >> 172.21.15.90:0/3148251316 conn(0x7fac040fb6c0 msgr2=0x7fac040fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:05.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.859+0000 7fac0a800640 1 -- 172.21.15.90:0/3148251316 shutdown_connections 2024-04-24T09:21:05.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:05.859+0000 7fac0a800640 1 -- 172.21.15.90:0/3148251316 wait complete. 2024-04-24T09:21:05.877 INFO:tasks.cephadm:Setup complete, yielding 2024-04-24T09:21:05.877 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-24T09:21:05.887 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi090.front.sepia.ceph.com 2024-04-24T09:21:05.887 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-24T09:21:06.331 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:21:06.749 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:06 smithi090 ceph-mon[27825]: pgmap v382: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-24T09:21:07.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:06 smithi149 ceph-mon[32592]: pgmap v382: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-24T09:21:07.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:06 smithi114 ceph-mon[32501]: pgmap v382: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-24T09:21:08.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.664+0000 7f68a1000700 1 -- 172.21.15.90:0/342019628 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f689c106930 msgr2=0x7f689c10d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:08.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.664+0000 7f68a1000700 1 --2- 172.21.15.90:0/342019628 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f689c106930 0x7f689c10d1e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f689400b0d0 tx=0x7f689400b3e0 comp rx=0 tx=0).stop 2024-04-24T09:21:08.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.664+0000 7f68a1000700 1 -- 172.21.15.90:0/342019628 shutdown_connections 2024-04-24T09:21:08.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.664+0000 7f68a1000700 1 --2- 172.21.15.90:0/342019628 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f689c106930 0x7f689c10d1e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:08.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.664+0000 7f68a1000700 1 --2- 172.21.15.90:0/342019628 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f689c104d90 0x7f689c1051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:08.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.664+0000 7f68a1000700 1 --2- 172.21.15.90:0/342019628 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f689c105f90 0x7f689c1063f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:08.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.664+0000 7f68a1000700 1 -- 172.21.15.90:0/342019628 >> 172.21.15.90:0/342019628 conn(0x7f689c100350 msgr2=0x7f689c102770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:08.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.664+0000 7f68a1000700 1 -- 172.21.15.90:0/342019628 shutdown_connections 2024-04-24T09:21:08.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.664+0000 7f68a1000700 1 -- 172.21.15.90:0/342019628 wait complete. 2024-04-24T09:21:08.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.665+0000 7f68a1000700 1 Processor -- start 2024-04-24T09:21:08.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.666+0000 7f68a1000700 1 -- start start 2024-04-24T09:21:08.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.666+0000 7f68a1000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f689c104d90 0x7f689c19aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:08.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.666+0000 7f68a1000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f689c105f90 0x7f689c19b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:08.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.666+0000 7f68a1000700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f689c106930 0x7f689c1a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:08.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.666+0000 7f68a1000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f689c1109b0 con 0x7f689c104d90 2024-04-24T09:21:08.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.666+0000 7f68a1000700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f689c110830 con 0x7f689c106930 2024-04-24T09:21:08.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.666+0000 7f68a1000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f689c110b30 con 0x7f689c105f90 2024-04-24T09:21:08.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.666+0000 7f6899e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f689c104d90 0x7f689c19aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:08.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.666+0000 7f6899400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f689c105f90 0x7f689c19b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:08.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.666+0000 7f6899e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f689c104d90 0x7f689c19aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40692/0 (socket says 172.21.15.90:40692) 2024-04-24T09:21:08.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.666+0000 7f6899e00700 1 -- 172.21.15.90:0/3604193702 learned_addr learned my addr 172.21.15.90:0/3604193702 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:08.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.667+0000 7f689a800700 1 --2- 172.21.15.90:0/3604193702 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f689c106930 0x7f689c1a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:08.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.667+0000 7f6899e00700 1 -- 172.21.15.90:0/3604193702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f689c105f90 msgr2=0x7f689c19b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:08.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.667+0000 7f6899e00700 1 --2- 172.21.15.90:0/3604193702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f689c105f90 0x7f689c19b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:08.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.667+0000 7f6899e00700 1 -- 172.21.15.90:0/3604193702 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f689c106930 msgr2=0x7f689c1a2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:08.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.667+0000 7f6899e00700 1 --2- 172.21.15.90:0/3604193702 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f689c106930 0x7f689c1a2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:08.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.667+0000 7f6899e00700 1 -- 172.21.15.90:0/3604193702 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f689c1a2b30 con 0x7f689c104d90 2024-04-24T09:21:08.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.667+0000 7f6899400700 1 --2- 172.21.15.90:0/3604193702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f689c105f90 0x7f689c19b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:21:08.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.667+0000 7f6899e00700 1 --2- 172.21.15.90:0/3604193702 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f689c104d90 0x7f689c19aed0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f688c00e9a0 tx=0x7f688c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:08.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.667+0000 7f6892a00700 1 -- 172.21.15.90:0/3604193702 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688c004220 con 0x7f689c104d90 2024-04-24T09:21:08.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.667+0000 7f6892a00700 1 -- 172.21.15.90:0/3604193702 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f688c0043c0 con 0x7f689c104d90 2024-04-24T09:21:08.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.667+0000 7f68a1000700 1 -- 172.21.15.90:0/3604193702 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f689c1a2e20 con 0x7f689c104d90 2024-04-24T09:21:08.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.667+0000 7f6892a00700 1 -- 172.21.15.90:0/3604193702 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688c003df0 con 0x7f689c104d90 2024-04-24T09:21:08.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.668+0000 7f68a1000700 1 -- 172.21.15.90:0/3604193702 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f689c1a3270 con 0x7f689c104d90 2024-04-24T09:21:08.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.669+0000 7f68a1000700 1 -- 172.21.15.90:0/3604193702 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f689c02dfe0 con 0x7f689c104d90 2024-04-24T09:21:08.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.671+0000 7f6892a00700 1 -- 172.21.15.90:0/3604193702 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f688c00c810 con 0x7f689c104d90 2024-04-24T09:21:08.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.672+0000 7f6892a00700 1 --2- 172.21.15.90:0/3604193702 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f68800a3c40 0x7f68800a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:08.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.672+0000 7f6892a00700 1 -- 172.21.15.90:0/3604193702 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f688c0d05b0 con 0x7f689c104d90 2024-04-24T09:21:08.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.672+0000 7f6899400700 1 --2- 172.21.15.90:0/3604193702 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f68800a3c40 0x7f68800a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:08.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.673+0000 7f6899400700 1 --2- 172.21.15.90:0/3604193702 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f68800a3c40 0x7f68800a6100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6888011490 tx=0x7f6888009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:08.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.677+0000 7f6892a00700 1 -- 172.21.15.90:0/3604193702 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f688c09cf50 con 0x7f689c104d90 2024-04-24T09:21:08.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:08 smithi090 ceph-mon[27825]: pgmap v383: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:21:08.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.924+0000 7f68a1000700 1 -- 172.21.15.90:0/3604193702 --> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f689c062f30 con 0x7f68800a3c40 2024-04-24T09:21:08.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.927+0000 7f6892a00700 1 -- 172.21.15.90:0/3604193702 <== mgr.14152 v2:172.21.15.90:6800/4261662956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f689c062f30 con 0x7f68800a3c40 2024-04-24T09:21:08.927 INFO:teuthology.orchestra.run.smithi090.stdout:Backend: cephadm 2024-04-24T09:21:08.927 INFO:teuthology.orchestra.run.smithi090.stdout:Available: Yes 2024-04-24T09:21:08.928 INFO:teuthology.orchestra.run.smithi090.stdout:Paused: No 2024-04-24T09:21:08.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.930+0000 7f68a1000700 1 -- 172.21.15.90:0/3604193702 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f68800a3c40 msgr2=0x7f68800a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:08.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.930+0000 7f68a1000700 1 --2- 172.21.15.90:0/3604193702 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f68800a3c40 0x7f68800a6100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6888011490 tx=0x7f6888009260 comp rx=0 tx=0).stop 2024-04-24T09:21:08.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.930+0000 7f68a1000700 1 -- 172.21.15.90:0/3604193702 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f689c104d90 msgr2=0x7f689c19aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:08.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.930+0000 7f68a1000700 1 --2- 172.21.15.90:0/3604193702 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f689c104d90 0x7f689c19aed0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f688c00e9a0 tx=0x7f688c00ecb0 comp rx=0 tx=0).stop 2024-04-24T09:21:08.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.931+0000 7f68a1000700 1 -- 172.21.15.90:0/3604193702 shutdown_connections 2024-04-24T09:21:08.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.931+0000 7f68a1000700 1 --2- 172.21.15.90:0/3604193702 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f689c106930 0x7f689c1a2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:08.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.931+0000 7f68a1000700 1 --2- 172.21.15.90:0/3604193702 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f68800a3c40 0x7f68800a6100 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:08.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.931+0000 7f68a1000700 1 --2- 172.21.15.90:0/3604193702 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f689c104d90 0x7f689c19aed0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:08.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.931+0000 7f68a1000700 1 --2- 172.21.15.90:0/3604193702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f689c105f90 0x7f689c19b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:08.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.931+0000 7f68a1000700 1 -- 172.21.15.90:0/3604193702 >> 172.21.15.90:0/3604193702 conn(0x7f689c100350 msgr2=0x7f689c1091c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:08.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.931+0000 7f68a1000700 1 -- 172.21.15.90:0/3604193702 shutdown_connections 2024-04-24T09:21:08.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:08.931+0000 7f68a1000700 1 -- 172.21.15.90:0/3604193702 wait complete. 2024-04-24T09:21:09.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:08 smithi149 ceph-mon[32592]: pgmap v383: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:21:09.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:08 smithi114 ceph-mon[32501]: pgmap v383: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:21:09.776 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-24T09:21:10.219 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:21:11.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:10 smithi090 ceph-mon[27825]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:21:11.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:10 smithi090 ceph-mon[27825]: pgmap v384: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-24T09:21:11.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:10 smithi149 ceph-mon[32592]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:21:11.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:10 smithi149 ceph-mon[32592]: pgmap v384: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-24T09:21:11.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:10 smithi114 ceph-mon[32501]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:21:11.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:10 smithi114 ceph-mon[32501]: pgmap v384: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-24T09:21:12.915 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:12 smithi090 ceph-mon[27825]: pgmap v385: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T09:21:12.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:12.952+0000 7f86c4a00700 1 -- 172.21.15.90:0/1644393426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86c0109190 msgr2=0x7f86c01095b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:12.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:12.952+0000 7f86c4a00700 1 --2- 172.21.15.90:0/1644393426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86c0109190 0x7f86c01095b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f86b0009990 tx=0x7f86b0009ca0 comp rx=0 tx=0).stop 2024-04-24T09:21:12.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:12.952+0000 7f86c4a00700 1 -- 172.21.15.90:0/1644393426 shutdown_connections 2024-04-24T09:21:12.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:12.952+0000 7f86c4a00700 1 --2- 172.21.15.90:0/1644393426 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86c010ad30 0x7f86c01115e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:12.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:12.952+0000 7f86c4a00700 1 --2- 172.21.15.90:0/1644393426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86c0109190 0x7f86c01095b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:12.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:12.952+0000 7f86c4a00700 1 --2- 172.21.15.90:0/1644393426 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f86c010a390 0x7f86c010a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:12.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:12.952+0000 7f86c4a00700 1 -- 172.21.15.90:0/1644393426 >> 172.21.15.90:0/1644393426 conn(0x7f86c0078160 msgr2=0x7f86c007a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:12.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:12.952+0000 7f86c4a00700 1 -- 172.21.15.90:0/1644393426 shutdown_connections 2024-04-24T09:21:12.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:12.952+0000 7f86c4a00700 1 -- 172.21.15.90:0/1644393426 wait complete. 2024-04-24T09:21:12.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:12.953+0000 7f86c4a00700 1 Processor -- start 2024-04-24T09:21:12.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:12.953+0000 7f86c4a00700 1 -- start start 2024-04-24T09:21:12.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:12.954+0000 7f86c4a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f86c0109190 0x7f86c019f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:12.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:12.954+0000 7f86c4a00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86c010a390 0x7f86c019f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:12.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:12.954+0000 7f86c4a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86c010ad30 0x7f86c01a6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:12.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:12.954+0000 7f86c4a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f86c0114dd0 con 0x7f86c0109190 2024-04-24T09:21:12.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:12.954+0000 7f86c4a00700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f86c0114c50 con 0x7f86c010a390 2024-04-24T09:21:12.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:12.954+0000 7f86c4a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f86c0114f50 con 0x7f86c010ad30 2024-04-24T09:21:12.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:12.954+0000 7f86bde00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f86c0109190 0x7f86c019f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:12.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:12.954+0000 7f86bde00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f86c0109190 0x7f86c019f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40712/0 (socket says 172.21.15.90:40712) 2024-04-24T09:21:12.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:12.954+0000 7f86bde00700 1 -- 172.21.15.90:0/3629457651 learned_addr learned my addr 172.21.15.90:0/3629457651 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:12.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:12.954+0000 7f86b5400700 1 --2- 172.21.15.90:0/3629457651 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86c010a390 0x7f86c019f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:12.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:12.954+0000 7f86be800700 1 --2- 172.21.15.90:0/3629457651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86c010ad30 0x7f86c01a6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:12.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:12.955+0000 7f86bde00700 1 -- 172.21.15.90:0/3629457651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86c010ad30 msgr2=0x7f86c01a6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:12.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:12.955+0000 7f86bde00700 1 --2- 172.21.15.90:0/3629457651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86c010ad30 0x7f86c01a6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:12.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:12.955+0000 7f86bde00700 1 -- 172.21.15.90:0/3629457651 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86c010a390 msgr2=0x7f86c019f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:12.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:12.955+0000 7f86bde00700 1 --2- 172.21.15.90:0/3629457651 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86c010a390 0x7f86c019f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:12.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:12.955+0000 7f86bde00700 1 -- 172.21.15.90:0/3629457651 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86c01a7010 con 0x7f86c0109190 2024-04-24T09:21:12.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:12.955+0000 7f86be800700 1 --2- 172.21.15.90:0/3629457651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86c010ad30 0x7f86c01a6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:21:12.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:12.955+0000 7f86bde00700 1 --2- 172.21.15.90:0/3629457651 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f86c0109190 0x7f86c019f3b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f86b000b820 tx=0x7f86b000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:12.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:12.955+0000 7f86b7400700 1 -- 172.21.15.90:0/3629457651 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b0026070 con 0x7f86c0109190 2024-04-24T09:21:12.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:12.955+0000 7f86b7400700 1 -- 172.21.15.90:0/3629457651 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86b000fda0 con 0x7f86c0109190 2024-04-24T09:21:12.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:12.955+0000 7f86c4a00700 1 -- 172.21.15.90:0/3629457651 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86c01a72a0 con 0x7f86c0109190 2024-04-24T09:21:12.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:12.955+0000 7f86b7400700 1 -- 172.21.15.90:0/3629457651 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b0017650 con 0x7f86c0109190 2024-04-24T09:21:12.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:12.956+0000 7f86c4a00700 1 -- 172.21.15.90:0/3629457651 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86c01a76f0 con 0x7f86c0109190 2024-04-24T09:21:12.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:12.957+0000 7f86c4a00700 1 -- 172.21.15.90:0/3629457651 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86c00697e0 con 0x7f86c0109190 2024-04-24T09:21:12.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:12.959+0000 7f86b7400700 1 -- 172.21.15.90:0/3629457651 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f86b0029070 con 0x7f86c0109190 2024-04-24T09:21:12.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:12.960+0000 7f86b7400700 1 --2- 172.21.15.90:0/3629457651 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f86a40a3c40 0x7f86a40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:12.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:12.960+0000 7f86b7400700 1 -- 172.21.15.90:0/3629457651 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f86b00d13f0 con 0x7f86c0109190 2024-04-24T09:21:12.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:12.960+0000 7f86b5400700 1 --2- 172.21.15.90:0/3629457651 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f86a40a3c40 0x7f86a40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:12.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:12.961+0000 7f86b5400700 1 --2- 172.21.15.90:0/3629457651 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f86a40a3c40 0x7f86a40a6100 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f86a8009c60 tx=0x7f86a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:12.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:12.965+0000 7f86b7400700 1 -- 172.21.15.90:0/3629457651 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f86b005dff0 con 0x7f86c0109190 2024-04-24T09:21:13.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:12 smithi149 ceph-mon[32592]: pgmap v385: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T09:21:13.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:12 smithi114 ceph-mon[32501]: pgmap v385: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T09:21:13.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:13.197+0000 7f86c4a00700 1 -- 172.21.15.90:0/3629457651 --> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f86c0062f30 con 0x7f86a40a3c40 2024-04-24T09:21:13.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:13.215+0000 7f86b7400700 1 -- 172.21.15.90:0/3629457651 <== mgr.14152 v2:172.21.15.90:6800/4261662956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f86c0062f30 con 0x7f86a40a3c40 2024-04-24T09:21:13.215 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T09:21:13.215 INFO:teuthology.orchestra.run.smithi090.stdout:mds.a smithi090 running (2m) 27s ago 2m 16.7M - 19.0.0-3116-g1f81fb39 b98aea5aa06f ec3fa743ec6c 2024-04-24T09:21:13.215 INFO:teuthology.orchestra.run.smithi090.stdout:mds.b smithi114 running (2m) 61s ago 2m 28.6M - 19.0.0-3116-g1f81fb39 b98aea5aa06f a4d4b1357aac 2024-04-24T09:21:13.215 INFO:teuthology.orchestra.run.smithi090.stdout:mds.c smithi149 running (2m) 61s ago 2m 19.2M - 19.0.0-3116-g1f81fb39 b98aea5aa06f b7df5ffb9489 2024-04-24T09:21:13.215 INFO:teuthology.orchestra.run.smithi090.stdout:mds.d smithi090 running (2m) 27s ago 2m 12.4M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 65013c07581f 2024-04-24T09:21:13.216 INFO:teuthology.orchestra.run.smithi090.stdout:mds.e smithi114 running (2m) 61s ago 2m 18.1M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 668c731cf63b 2024-04-24T09:21:13.216 INFO:teuthology.orchestra.run.smithi090.stdout:mds.f smithi149 running (2m) 61s ago 2m 19.5M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 01b2fff4c5a1 2024-04-24T09:21:13.216 INFO:teuthology.orchestra.run.smithi090.stdout:mds.g smithi090 running (2m) 27s ago 2m 12.2M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 8d2f12b5f7de 2024-04-24T09:21:13.216 INFO:teuthology.orchestra.run.smithi090.stdout:mds.h smithi114 running (2m) 61s ago 2m 16.1M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 6c22bab096ce 2024-04-24T09:21:13.216 INFO:teuthology.orchestra.run.smithi090.stdout:mds.i smithi149 running (2m) 61s ago 2m 18.1M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 35e767d0635d 2024-04-24T09:21:13.216 INFO:teuthology.orchestra.run.smithi090.stdout:mds.j smithi090 running (2m) 27s ago 2m 16.2M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 5902404428e6 2024-04-24T09:21:13.216 INFO:teuthology.orchestra.run.smithi090.stdout:mds.k smithi114 running (2m) 61s ago 2m 18.4M - 19.0.0-3116-g1f81fb39 b98aea5aa06f fd4362b3443b 2024-04-24T09:21:13.216 INFO:teuthology.orchestra.run.smithi090.stdout:mds.l smithi149 running (2m) 61s ago 2m 16.2M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 454120a5fc35 2024-04-24T09:21:13.216 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.x smithi090 *:9283,8765 running (11m) 27s ago 11m 515M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 71027b126301 2024-04-24T09:21:13.216 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.y smithi114 *:8443,8765 running (8m) 61s ago 8m 455M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 5b4d3ed09dc7 2024-04-24T09:21:13.216 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.z smithi149 *:8443,8765 running (8m) 61s ago 8m 452M - 19.0.0-3116-g1f81fb39 b98aea5aa06f b35998b28eee 2024-04-24T09:21:13.216 INFO:teuthology.orchestra.run.smithi090.stdout:mon.a smithi090 running (11m) 27s ago 11m 46.9M 2048M 19.0.0-3116-g1f81fb39 b98aea5aa06f b8ed5a8fba8a 2024-04-24T09:21:13.216 INFO:teuthology.orchestra.run.smithi090.stdout:mon.b smithi114 running (9m) 61s ago 9m 41.4M 2048M 19.0.0-3116-g1f81fb39 b98aea5aa06f 5d44667e66fb 2024-04-24T09:21:13.216 INFO:teuthology.orchestra.run.smithi090.stdout:mon.c smithi149 running (9m) 61s ago 9m 41.7M 2048M 19.0.0-3116-g1f81fb39 b98aea5aa06f da0fa284ae48 2024-04-24T09:21:13.216 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (8m) 27s ago 8m 44.3M 4272M 19.0.0-3116-g1f81fb39 b98aea5aa06f f0400b31527d 2024-04-24T09:21:13.217 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi114 running (8m) 61s ago 8m 56.1M 4272M 19.0.0-3116-g1f81fb39 b98aea5aa06f 8ec64866bb21 2024-04-24T09:21:13.217 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi149 running (7m) 61s ago 7m 46.0M 1454M 19.0.0-3116-g1f81fb39 b98aea5aa06f b4c745b0a490 2024-04-24T09:21:13.217 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi090 running (7m) 27s ago 7m 42.0M 4272M 19.0.0-3116-g1f81fb39 b98aea5aa06f 71f3dd397ea8 2024-04-24T09:21:13.217 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi114 running (6m) 61s ago 6m 42.0M 4272M 19.0.0-3116-g1f81fb39 b98aea5aa06f 31e36e901666 2024-04-24T09:21:13.217 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi149 running (6m) 61s ago 6m 57.3M 1454M 19.0.0-3116-g1f81fb39 b98aea5aa06f 4506cdc0ee7d 2024-04-24T09:21:13.217 INFO:teuthology.orchestra.run.smithi090.stdout:osd.6 smithi090 running (5m) 27s ago 5m 41.0M 4272M 19.0.0-3116-g1f81fb39 b98aea5aa06f b5538ae378bb 2024-04-24T09:21:13.217 INFO:teuthology.orchestra.run.smithi090.stdout:osd.7 smithi114 running (5m) 61s ago 5m 55.7M 4272M 19.0.0-3116-g1f81fb39 b98aea5aa06f c8adb77cb155 2024-04-24T09:21:13.217 INFO:teuthology.orchestra.run.smithi090.stdout:osd.8 smithi149 running (4m) 61s ago 4m 52.2M 1454M 19.0.0-3116-g1f81fb39 b98aea5aa06f fe0c490793e9 2024-04-24T09:21:13.217 INFO:teuthology.orchestra.run.smithi090.stdout:osd.9 smithi090 running (4m) 27s ago 4m 50.0M 4272M 19.0.0-3116-g1f81fb39 b98aea5aa06f 244bfccb3ecf 2024-04-24T09:21:13.217 INFO:teuthology.orchestra.run.smithi090.stdout:osd.10 smithi114 running (3m) 61s ago 3m 54.2M 4272M 19.0.0-3116-g1f81fb39 b98aea5aa06f 18bf6d3d68be 2024-04-24T09:21:13.217 INFO:teuthology.orchestra.run.smithi090.stdout:osd.11 smithi149 running (3m) 61s ago 3m 53.1M 1454M 19.0.0-3116-g1f81fb39 b98aea5aa06f f8562d2e7c5f 2024-04-24T09:21:13.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:13.217+0000 7f86c4a00700 1 -- 172.21.15.90:0/3629457651 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f86a40a3c40 msgr2=0x7f86a40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:13.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:13.217+0000 7f86c4a00700 1 --2- 172.21.15.90:0/3629457651 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f86a40a3c40 0x7f86a40a6100 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f86a8009c60 tx=0x7f86a8009260 comp rx=0 tx=0).stop 2024-04-24T09:21:13.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:13.217+0000 7f86c4a00700 1 -- 172.21.15.90:0/3629457651 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f86c0109190 msgr2=0x7f86c019f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:13.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:13.217+0000 7f86c4a00700 1 --2- 172.21.15.90:0/3629457651 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f86c0109190 0x7f86c019f3b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f86b000b820 tx=0x7f86b000f690 comp rx=0 tx=0).stop 2024-04-24T09:21:13.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:13.218+0000 7f86c4a00700 1 -- 172.21.15.90:0/3629457651 shutdown_connections 2024-04-24T09:21:13.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:13.218+0000 7f86c4a00700 1 --2- 172.21.15.90:0/3629457651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86c010ad30 0x7f86c01a6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:13.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:13.218+0000 7f86c4a00700 1 --2- 172.21.15.90:0/3629457651 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f86a40a3c40 0x7f86a40a6100 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:13.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:13.218+0000 7f86c4a00700 1 --2- 172.21.15.90:0/3629457651 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f86c0109190 0x7f86c019f3b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:13.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:13.218+0000 7f86c4a00700 1 --2- 172.21.15.90:0/3629457651 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f86c010a390 0x7f86c019f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:13.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:13.218+0000 7f86c4a00700 1 -- 172.21.15.90:0/3629457651 >> 172.21.15.90:0/3629457651 conn(0x7f86c0078160 msgr2=0x7f86c010d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:13.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:13.218+0000 7f86c4a00700 1 -- 172.21.15.90:0/3629457651 shutdown_connections 2024-04-24T09:21:13.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:13.218+0000 7f86c4a00700 1 -- 172.21.15.90:0/3629457651 wait complete. 2024-04-24T09:21:14.341 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-24T09:21:14.787 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:21:15.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:14 smithi090 ceph-mon[27825]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:21:15.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:14 smithi090 ceph-mon[27825]: pgmap v386: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:21:15.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:14 smithi149 ceph-mon[32592]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:21:15.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:14 smithi149 ceph-mon[32592]: pgmap v386: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:21:15.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:14 smithi114 ceph-mon[32501]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:21:15.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:14 smithi114 ceph-mon[32501]: pgmap v386: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:21:17.040 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:16 smithi090 ceph-mon[27825]: pgmap v387: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-24T09:21:17.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:16 smithi149 ceph-mon[32592]: pgmap v387: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-24T09:21:17.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:16 smithi114 ceph-mon[32501]: pgmap v387: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-24T09:21:17.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:17.771+0000 7f6217a00700 1 -- 172.21.15.90:0/3347914015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6210104700 msgr2=0x7f621010afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:17.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:17.771+0000 7f6217a00700 1 --2- 172.21.15.90:0/3347914015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6210104700 0x7f621010afb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f62080099c0 tx=0x7f6208009cd0 comp rx=0 tx=0).stop 2024-04-24T09:21:17.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:17.771+0000 7f6217a00700 1 -- 172.21.15.90:0/3347914015 shutdown_connections 2024-04-24T09:21:17.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:17.771+0000 7f6217a00700 1 --2- 172.21.15.90:0/3347914015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6210104700 0x7f621010afb0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:17.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:17.771+0000 7f6217a00700 1 --2- 172.21.15.90:0/3347914015 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6210102b60 0x7f6210102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:17.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:17.771+0000 7f6217a00700 1 --2- 172.21.15.90:0/3347914015 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6210103d60 0x7f62101041c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:17.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:17.771+0000 7f6217a00700 1 -- 172.21.15.90:0/3347914015 >> 172.21.15.90:0/3347914015 conn(0x7f62100fe0c0 msgr2=0x7f6210100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:17.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:17.771+0000 7f6217a00700 1 -- 172.21.15.90:0/3347914015 shutdown_connections 2024-04-24T09:21:17.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:17.771+0000 7f6217a00700 1 -- 172.21.15.90:0/3347914015 wait complete. 2024-04-24T09:21:17.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:17.772+0000 7f6217a00700 1 Processor -- start 2024-04-24T09:21:17.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:17.773+0000 7f6217a00700 1 -- start start 2024-04-24T09:21:17.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:17.773+0000 7f6217a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6210102b60 0x7f6210198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:17.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:17.773+0000 7f6217a00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6210103d60 0x7f6210199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:17.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:17.773+0000 7f6217a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6210104700 0x7f62101a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:17.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:17.773+0000 7f6217a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f621010e7a0 con 0x7f6210104700 2024-04-24T09:21:17.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:17.773+0000 7f6217a00700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f621010e620 con 0x7f6210103d60 2024-04-24T09:21:17.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:17.773+0000 7f6217a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f621010e920 con 0x7f6210102b60 2024-04-24T09:21:17.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:17.773+0000 7f6215a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6210104700 0x7f62101a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:17.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:17.773+0000 7f6215a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6210104700 0x7f62101a0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51506/0 (socket says 172.21.15.90:51506) 2024-04-24T09:21:17.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:17.773+0000 7f6215a00700 1 -- 172.21.15.90:0/3594208516 learned_addr learned my addr 172.21.15.90:0/3594208516 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:17.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:17.773+0000 7f6215000700 1 --2- 172.21.15.90:0/3594208516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6210102b60 0x7f6210198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:17.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:17.773+0000 7f620fe00700 1 --2- 172.21.15.90:0/3594208516 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6210103d60 0x7f6210199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:17.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:17.774+0000 7f6215a00700 1 -- 172.21.15.90:0/3594208516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6210102b60 msgr2=0x7f6210198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:17.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:17.774+0000 7f6215a00700 1 --2- 172.21.15.90:0/3594208516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6210102b60 0x7f6210198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:17.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:17.774+0000 7f6215a00700 1 -- 172.21.15.90:0/3594208516 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6210103d60 msgr2=0x7f6210199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:17.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:17.774+0000 7f6215a00700 1 --2- 172.21.15.90:0/3594208516 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6210103d60 0x7f6210199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:17.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:17.774+0000 7f6215a00700 1 -- 172.21.15.90:0/3594208516 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62101a09b0 con 0x7f6210104700 2024-04-24T09:21:17.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:17.774+0000 7f6215a00700 1 --2- 172.21.15.90:0/3594208516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6210104700 0x7f62101a0310 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f620800b580 tx=0x7f620800bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:17.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:17.774+0000 7f620d600700 1 -- 172.21.15.90:0/3594208516 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6208009e60 con 0x7f6210104700 2024-04-24T09:21:17.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:17.774+0000 7f620d600700 1 -- 172.21.15.90:0/3594208516 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6208003e10 con 0x7f6210104700 2024-04-24T09:21:17.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:17.774+0000 7f620d600700 1 -- 172.21.15.90:0/3594208516 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62080173f0 con 0x7f6210104700 2024-04-24T09:21:17.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:17.774+0000 7f6217a00700 1 -- 172.21.15.90:0/3594208516 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62101a0c40 con 0x7f6210104700 2024-04-24T09:21:17.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:17.774+0000 7f6217a00700 1 -- 172.21.15.90:0/3594208516 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62101a1040 con 0x7f6210104700 2024-04-24T09:21:17.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:17.776+0000 7f6217a00700 1 -- 172.21.15.90:0/3594208516 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f621002dfe0 con 0x7f6210104700 2024-04-24T09:21:17.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:17.777+0000 7f620d600700 1 -- 172.21.15.90:0/3594208516 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f62080052d0 con 0x7f6210104700 2024-04-24T09:21:17.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:17.779+0000 7f620d600700 1 --2- 172.21.15.90:0/3594208516 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f61f80a3d40 0x7f61f80a6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:17.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:17.779+0000 7f620d600700 1 -- 172.21.15.90:0/3594208516 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f62080d1300 con 0x7f6210104700 2024-04-24T09:21:17.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:17.779+0000 7f6215000700 1 --2- 172.21.15.90:0/3594208516 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f61f80a3d40 0x7f61f80a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:17.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:17.780+0000 7f6215000700 1 --2- 172.21.15.90:0/3594208516 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f61f80a3d40 0x7f61f80a6200 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6210103bc0 tx=0x7f620400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:17.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:17.784+0000 7f620d600700 1 -- 172.21.15.90:0/3594208516 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6208013070 con 0x7f6210104700 2024-04-24T09:21:18.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:18.027+0000 7f6217a00700 1 -- 172.21.15.90:0/3594208516 --> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f6210062f30 con 0x7f61f80a3d40 2024-04-24T09:21:18.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:18.035+0000 7f620d600700 1 -- 172.21.15.90:0/3594208516 <== mgr.14152 v2:172.21.15.90:6800/4261662956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f6210062f30 con 0x7f61f80a3d40 2024-04-24T09:21:18.035 INFO:teuthology.orchestra.run.smithi090.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-24T09:21:18.035 INFO:teuthology.orchestra.run.smithi090.stdout:mds.all 12/12 66s ago 2m smithi090=a;smithi090=d;smithi090=g;smithi090=j;smithi114=b;smithi114=e;smithi114=h;smithi114=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l;count:12 2024-04-24T09:21:18.035 INFO:teuthology.orchestra.run.smithi090.stdout:mgr 3/3 66s ago 9m smithi090=x;smithi114=y;smithi149=z;count:3 2024-04-24T09:21:18.035 INFO:teuthology.orchestra.run.smithi090.stdout:mon 3/3 66s ago 9m smithi090:172.21.15.90=a;smithi114:172.21.15.114=b;smithi149:172.21.15.149=c;count:3 2024-04-24T09:21:18.035 INFO:teuthology.orchestra.run.smithi090.stdout:osd 12 66s ago - 2024-04-24T09:21:18.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:18.038+0000 7f6217a00700 1 -- 172.21.15.90:0/3594208516 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f61f80a3d40 msgr2=0x7f61f80a6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:18.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:18.038+0000 7f6217a00700 1 --2- 172.21.15.90:0/3594208516 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f61f80a3d40 0x7f61f80a6200 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6210103bc0 tx=0x7f620400b040 comp rx=0 tx=0).stop 2024-04-24T09:21:18.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:18.038+0000 7f6217a00700 1 -- 172.21.15.90:0/3594208516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6210104700 msgr2=0x7f62101a0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:18.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:18.038+0000 7f6217a00700 1 --2- 172.21.15.90:0/3594208516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6210104700 0x7f62101a0310 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f620800b580 tx=0x7f620800bee0 comp rx=0 tx=0).stop 2024-04-24T09:21:18.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:18.038+0000 7f6217a00700 1 -- 172.21.15.90:0/3594208516 shutdown_connections 2024-04-24T09:21:18.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:18.038+0000 7f6217a00700 1 --2- 172.21.15.90:0/3594208516 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f61f80a3d40 0x7f61f80a6200 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:18.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:18.038+0000 7f6217a00700 1 --2- 172.21.15.90:0/3594208516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6210104700 0x7f62101a0310 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:18.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:18.038+0000 7f6217a00700 1 --2- 172.21.15.90:0/3594208516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6210102b60 0x7f6210198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:18.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:18.038+0000 7f6217a00700 1 --2- 172.21.15.90:0/3594208516 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6210103d60 0x7f6210199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:18.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:18.038+0000 7f6217a00700 1 -- 172.21.15.90:0/3594208516 >> 172.21.15.90:0/3594208516 conn(0x7f62100fe0c0 msgr2=0x7f6210107000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:18.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:18.038+0000 7f6217a00700 1 -- 172.21.15.90:0/3594208516 shutdown_connections 2024-04-24T09:21:18.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:18.038+0000 7f6217a00700 1 -- 172.21.15.90:0/3594208516 wait complete. 2024-04-24T09:21:19.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:18 smithi090 ceph-mon[27825]: pgmap v388: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:21:19.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:18 smithi149 ceph-mon[32592]: pgmap v388: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:21:19.175 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:18 smithi114 ceph-mon[32501]: pgmap v388: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:21:19.295 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-24T09:21:19.759 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:21:19.991 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:19 smithi090 ceph-mon[27825]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:21:20.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:19 smithi149 ceph-mon[32592]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:21:20.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:19 smithi114 ceph-mon[32501]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:21:21.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:20 smithi090 ceph-mon[27825]: pgmap v389: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-24T09:21:21.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:20 smithi149 ceph-mon[32592]: pgmap v389: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-24T09:21:21.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:20 smithi114 ceph-mon[32501]: pgmap v389: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-24T09:21:22.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.543+0000 7f57efa00700 1 -- 172.21.15.90:0/3959036078 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f57e810a390 msgr2=0x7f57e810a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:22.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.543+0000 7f57efa00700 1 --2- 172.21.15.90:0/3959036078 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f57e810a390 0x7f57e810a7f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f57d40099c0 tx=0x7f57d4009cd0 comp rx=0 tx=0).stop 2024-04-24T09:21:22.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.543+0000 7f57efa00700 1 -- 172.21.15.90:0/3959036078 shutdown_connections 2024-04-24T09:21:22.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.543+0000 7f57efa00700 1 --2- 172.21.15.90:0/3959036078 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57e810ad30 0x7f57e81115e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:22.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.543+0000 7f57efa00700 1 --2- 172.21.15.90:0/3959036078 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f57e8109190 0x7f57e81095b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:22.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.543+0000 7f57efa00700 1 --2- 172.21.15.90:0/3959036078 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f57e810a390 0x7f57e810a7f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:22.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.543+0000 7f57efa00700 1 -- 172.21.15.90:0/3959036078 >> 172.21.15.90:0/3959036078 conn(0x7f57e8078160 msgr2=0x7f57e807a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:22.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.543+0000 7f57efa00700 1 -- 172.21.15.90:0/3959036078 shutdown_connections 2024-04-24T09:21:22.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.543+0000 7f57efa00700 1 -- 172.21.15.90:0/3959036078 wait complete. 2024-04-24T09:21:22.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.544+0000 7f57efa00700 1 Processor -- start 2024-04-24T09:21:22.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.544+0000 7f57efa00700 1 -- start start 2024-04-24T09:21:22.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.545+0000 7f57efa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f57e8109190 0x7f57e819f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:22.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.545+0000 7f57efa00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f57e810a390 0x7f57e819f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:22.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.545+0000 7f57efa00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57e810ad30 0x7f57e81a6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:22.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.545+0000 7f57efa00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f57e8114d60 con 0x7f57e8109190 2024-04-24T09:21:22.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.545+0000 7f57efa00700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f57e8114be0 con 0x7f57e810ad30 2024-04-24T09:21:22.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.545+0000 7f57efa00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f57e8114ee0 con 0x7f57e810a390 2024-04-24T09:21:22.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.545+0000 7f57ed000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f57e8109190 0x7f57e819f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:22.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.545+0000 7f57ed000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f57e8109190 0x7f57e819f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51516/0 (socket says 172.21.15.90:51516) 2024-04-24T09:21:22.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.545+0000 7f57e7e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f57e810a390 0x7f57e819f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:22.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.545+0000 7f57ed000700 1 -- 172.21.15.90:0/2596877994 learned_addr learned my addr 172.21.15.90:0/2596877994 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:22.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.545+0000 7f57eda00700 1 --2- 172.21.15.90:0/2596877994 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57e810ad30 0x7f57e81a6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:22.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.546+0000 7f57e7e00700 1 -- 172.21.15.90:0/2596877994 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57e810ad30 msgr2=0x7f57e81a6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:22.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.546+0000 7f57e7e00700 1 --2- 172.21.15.90:0/2596877994 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57e810ad30 0x7f57e81a6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:22.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.546+0000 7f57e7e00700 1 -- 172.21.15.90:0/2596877994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f57e8109190 msgr2=0x7f57e819f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:22.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.546+0000 7f57e7e00700 1 --2- 172.21.15.90:0/2596877994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f57e8109190 0x7f57e819f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:22.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.546+0000 7f57e7e00700 1 -- 172.21.15.90:0/2596877994 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57e81a6fb0 con 0x7f57e810a390 2024-04-24T09:21:22.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.546+0000 7f57eda00700 1 --2- 172.21.15.90:0/2596877994 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57e810ad30 0x7f57e81a6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:21:22.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.546+0000 7f57e7e00700 1 --2- 172.21.15.90:0/2596877994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f57e810a390 0x7f57e819f890 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f57d40098e0 tx=0x7f57d4009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:22.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.546+0000 7f57e5600700 1 -- 172.21.15.90:0/2596877994 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d4009e60 con 0x7f57e810a390 2024-04-24T09:21:22.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.546+0000 7f57e5600700 1 -- 172.21.15.90:0/2596877994 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57d4003e40 con 0x7f57e810a390 2024-04-24T09:21:22.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.546+0000 7f57e5600700 1 -- 172.21.15.90:0/2596877994 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d4005580 con 0x7f57e810a390 2024-04-24T09:21:22.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.546+0000 7f57efa00700 1 -- 172.21.15.90:0/2596877994 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57e81a7240 con 0x7f57e810a390 2024-04-24T09:21:22.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.547+0000 7f57efa00700 1 -- 172.21.15.90:0/2596877994 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57e81a7770 con 0x7f57e810a390 2024-04-24T09:21:22.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.548+0000 7f57efa00700 1 -- 172.21.15.90:0/2596877994 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57e80697e0 con 0x7f57e810a390 2024-04-24T09:21:22.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.550+0000 7f57e5600700 1 -- 172.21.15.90:0/2596877994 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f57d400fbe0 con 0x7f57e810a390 2024-04-24T09:21:22.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.551+0000 7f57e5600700 1 --2- 172.21.15.90:0/2596877994 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f57d00a3cc0 0x7f57d00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:22.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.551+0000 7f57e5600700 1 -- 172.21.15.90:0/2596877994 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f57d40d1370 con 0x7f57e810a390 2024-04-24T09:21:22.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.552+0000 7f57ed000700 1 --2- 172.21.15.90:0/2596877994 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f57d00a3cc0 0x7f57d00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:22.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.552+0000 7f57ed000700 1 --2- 172.21.15.90:0/2596877994 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f57d00a3cc0 0x7f57d00a6180 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f57dc005950 tx=0x7f57dc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:22.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.554+0000 7f57e5600700 1 -- 172.21.15.90:0/2596877994 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f57d405dff0 con 0x7f57e810a390 2024-04-24T09:21:22.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.788+0000 7f57efa00700 1 -- 172.21.15.90:0/2596877994 --> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f57e8062ff0 con 0x7f57d00a3cc0 2024-04-24T09:21:22.789 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:22 smithi090 ceph-mon[27825]: pgmap v390: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T09:21:22.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.792+0000 7f57e5600700 1 -- 172.21.15.90:0/2596877994 <== mgr.14152 v2:172.21.15.90:6800/4261662956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f57e8062ff0 con 0x7f57d00a3cc0 2024-04-24T09:21:22.792 INFO:teuthology.orchestra.run.smithi090.stdout:HOST ADDR LABELS STATUS 2024-04-24T09:21:22.792 INFO:teuthology.orchestra.run.smithi090.stdout:smithi090 172.21.15.90 2024-04-24T09:21:22.792 INFO:teuthology.orchestra.run.smithi090.stdout:smithi114 172.21.15.114 2024-04-24T09:21:22.792 INFO:teuthology.orchestra.run.smithi090.stdout:smithi149 172.21.15.149 2024-04-24T09:21:22.792 INFO:teuthology.orchestra.run.smithi090.stdout:3 hosts in cluster 2024-04-24T09:21:22.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.795+0000 7f57efa00700 1 -- 172.21.15.90:0/2596877994 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f57d00a3cc0 msgr2=0x7f57d00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:22.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.795+0000 7f57efa00700 1 --2- 172.21.15.90:0/2596877994 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f57d00a3cc0 0x7f57d00a6180 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f57dc005950 tx=0x7f57dc009260 comp rx=0 tx=0).stop 2024-04-24T09:21:22.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.795+0000 7f57efa00700 1 -- 172.21.15.90:0/2596877994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f57e810a390 msgr2=0x7f57e819f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:22.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.795+0000 7f57efa00700 1 --2- 172.21.15.90:0/2596877994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f57e810a390 0x7f57e819f890 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f57d40098e0 tx=0x7f57d4009580 comp rx=0 tx=0).stop 2024-04-24T09:21:22.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.795+0000 7f57efa00700 1 -- 172.21.15.90:0/2596877994 shutdown_connections 2024-04-24T09:21:22.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.795+0000 7f57efa00700 1 --2- 172.21.15.90:0/2596877994 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57e810ad30 0x7f57e81a6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:22.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.795+0000 7f57efa00700 1 --2- 172.21.15.90:0/2596877994 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f57d00a3cc0 0x7f57d00a6180 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:22.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.795+0000 7f57efa00700 1 --2- 172.21.15.90:0/2596877994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f57e8109190 0x7f57e819f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:22.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.795+0000 7f57efa00700 1 --2- 172.21.15.90:0/2596877994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f57e810a390 0x7f57e819f890 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:22.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.795+0000 7f57efa00700 1 -- 172.21.15.90:0/2596877994 >> 172.21.15.90:0/2596877994 conn(0x7f57e8078160 msgr2=0x7f57e810d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:22.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.795+0000 7f57efa00700 1 -- 172.21.15.90:0/2596877994 shutdown_connections 2024-04-24T09:21:22.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:22.795+0000 7f57efa00700 1 -- 172.21.15.90:0/2596877994 wait complete. 2024-04-24T09:21:23.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:22 smithi149 ceph-mon[32592]: pgmap v390: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T09:21:23.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:22 smithi114 ceph-mon[32501]: pgmap v390: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T09:21:24.040 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-24T09:21:24.502 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:21:24.978 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:24 smithi090 ceph-mon[27825]: from='client.25090 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:21:24.978 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:24 smithi090 ceph-mon[27825]: pgmap v391: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:21:25.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:24 smithi149 ceph-mon[32592]: from='client.25090 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:21:25.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:24 smithi149 ceph-mon[32592]: pgmap v391: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:21:25.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:24 smithi114 ceph-mon[32501]: from='client.25090 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:21:25.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:24 smithi114 ceph-mon[32501]: pgmap v391: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:21:26.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:26.779+0000 7f282c400700 1 -- 172.21.15.90:0/3446274898 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2824102b60 msgr2=0x7f2824102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:26.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:26.779+0000 7f282c400700 1 --2- 172.21.15.90:0/3446274898 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2824102b60 0x7f2824102f80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f28180098f0 tx=0x7f2818009c00 comp rx=0 tx=0).stop 2024-04-24T09:21:26.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:26.779+0000 7f282c400700 1 -- 172.21.15.90:0/3446274898 shutdown_connections 2024-04-24T09:21:26.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:26.779+0000 7f282c400700 1 --2- 172.21.15.90:0/3446274898 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2824104700 0x7f282410afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:26.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:26.779+0000 7f282c400700 1 --2- 172.21.15.90:0/3446274898 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2824102b60 0x7f2824102f80 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:26.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:26.779+0000 7f282c400700 1 --2- 172.21.15.90:0/3446274898 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2824103d60 0x7f28241041c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:26.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:26.779+0000 7f282c400700 1 -- 172.21.15.90:0/3446274898 >> 172.21.15.90:0/3446274898 conn(0x7f28240fe0c0 msgr2=0x7f2824100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:26.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:26.779+0000 7f282c400700 1 -- 172.21.15.90:0/3446274898 shutdown_connections 2024-04-24T09:21:26.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:26.779+0000 7f282c400700 1 -- 172.21.15.90:0/3446274898 wait complete. 2024-04-24T09:21:26.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:26.780+0000 7f282c400700 1 Processor -- start 2024-04-24T09:21:26.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:26.780+0000 7f282c400700 1 -- start start 2024-04-24T09:21:26.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:26.781+0000 7f282c400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2824102b60 0x7f2824198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:26.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:26.781+0000 7f282c400700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2824103d60 0x7f2824199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:26.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:26.781+0000 7f282c400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2824104700 0x7f28241a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:26.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:26.781+0000 7f282c400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f282410e720 con 0x7f2824104700 2024-04-24T09:21:26.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:26.781+0000 7f282c400700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f282410e5a0 con 0x7f2824103d60 2024-04-24T09:21:26.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:26.781+0000 7f282c400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f282410e8a0 con 0x7f2824102b60 2024-04-24T09:21:26.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:26.781+0000 7f2829a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2824102b60 0x7f2824198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:26.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:26.781+0000 7f282a400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2824104700 0x7f28241a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:26.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:26.781+0000 7f2829000700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2824103d60 0x7f2824199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:26.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:26.781+0000 7f2829a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2824102b60 0x7f2824198e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:47384/0 (socket says 172.21.15.90:47384) 2024-04-24T09:21:26.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:26.781+0000 7f2829a00700 1 -- 172.21.15.90:0/782299800 learned_addr learned my addr 172.21.15.90:0/782299800 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:26.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:26.781+0000 7f2829a00700 1 -- 172.21.15.90:0/782299800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2824103d60 msgr2=0x7f2824199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:26.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:26.781+0000 7f2829a00700 1 --2- 172.21.15.90:0/782299800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2824103d60 0x7f2824199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:26.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:26.781+0000 7f2829a00700 1 -- 172.21.15.90:0/782299800 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2824104700 msgr2=0x7f28241a03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:26.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:26.781+0000 7f2829a00700 1 --2- 172.21.15.90:0/782299800 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2824104700 0x7f28241a03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:26.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:26.781+0000 7f2829a00700 1 -- 172.21.15.90:0/782299800 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28241a0a50 con 0x7f2824102b60 2024-04-24T09:21:26.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:26.782+0000 7f2829a00700 1 --2- 172.21.15.90:0/782299800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2824102b60 0x7f2824198e40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2818009230 tx=0x7f28180052f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:26.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:26.782+0000 7f2822000700 1 -- 172.21.15.90:0/782299800 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281800b560 con 0x7f2824102b60 2024-04-24T09:21:26.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:26.782+0000 7f282c400700 1 -- 172.21.15.90:0/782299800 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28241a0ce0 con 0x7f2824102b60 2024-04-24T09:21:26.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:26.782+0000 7f282c400700 1 -- 172.21.15.90:0/782299800 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28241a1190 con 0x7f2824102b60 2024-04-24T09:21:26.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:26.782+0000 7f2822000700 1 -- 172.21.15.90:0/782299800 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f281800bbe0 con 0x7f2824102b60 2024-04-24T09:21:26.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:26.782+0000 7f2822000700 1 -- 172.21.15.90:0/782299800 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28180094c0 con 0x7f2824102b60 2024-04-24T09:21:26.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:26.784+0000 7f282c400700 1 -- 172.21.15.90:0/782299800 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2804005360 con 0x7f2824102b60 2024-04-24T09:21:26.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:26.785+0000 7f2822000700 1 -- 172.21.15.90:0/782299800 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2818003cf0 con 0x7f2824102b60 2024-04-24T09:21:26.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:26.786+0000 7f2822000700 1 --2- 172.21.15.90:0/782299800 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f280c0a3c40 0x7f280c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:26.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:26.787+0000 7f2822000700 1 -- 172.21.15.90:0/782299800 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f28180d1110 con 0x7f2824102b60 2024-04-24T09:21:26.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:26.787+0000 7f2829000700 1 --2- 172.21.15.90:0/782299800 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f280c0a3c40 0x7f280c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:26.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:26.787+0000 7f2829000700 1 --2- 172.21.15.90:0/782299800 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f280c0a3c40 0x7f280c0a6100 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2810009c60 tx=0x7f2810009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:26.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:26.791+0000 7f2822000700 1 -- 172.21.15.90:0/782299800 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2818013030 con 0x7f2824102b60 2024-04-24T09:21:27.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:27.028+0000 7f282c400700 1 -- 172.21.15.90:0/782299800 --> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f2804000bf0 con 0x7f280c0a3c40 2024-04-24T09:21:27.029 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:26 smithi090 ceph-mon[27825]: pgmap v392: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-24T09:21:27.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:27.032+0000 7f2822000700 1 -- 172.21.15.90:0/782299800 <== mgr.14152 v2:172.21.15.90:6800/4261662956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f2804000bf0 con 0x7f280c0a3c40 2024-04-24T09:21:27.032 INFO:teuthology.orchestra.run.smithi090.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-24T09:21:27.033 INFO:teuthology.orchestra.run.smithi090.stdout:smithi090 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400UK400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-04-24T09:21:27.033 INFO:teuthology.orchestra.run.smithi090.stdout:smithi114 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400WB400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-24T09:21:27.033 INFO:teuthology.orchestra.run.smithi090.stdout:smithi149 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300CK400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-24T09:21:27.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:27.035+0000 7f282c400700 1 -- 172.21.15.90:0/782299800 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f280c0a3c40 msgr2=0x7f280c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:27.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:27.035+0000 7f282c400700 1 --2- 172.21.15.90:0/782299800 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f280c0a3c40 0x7f280c0a6100 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2810009c60 tx=0x7f2810009260 comp rx=0 tx=0).stop 2024-04-24T09:21:27.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:27.035+0000 7f282c400700 1 -- 172.21.15.90:0/782299800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2824102b60 msgr2=0x7f2824198e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:27.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:27.035+0000 7f282c400700 1 --2- 172.21.15.90:0/782299800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2824102b60 0x7f2824198e40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2818009230 tx=0x7f28180052f0 comp rx=0 tx=0).stop 2024-04-24T09:21:27.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:27.035+0000 7f282c400700 1 -- 172.21.15.90:0/782299800 shutdown_connections 2024-04-24T09:21:27.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:27.036+0000 7f282c400700 1 --2- 172.21.15.90:0/782299800 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f280c0a3c40 0x7f280c0a6100 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:27.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:27.036+0000 7f282c400700 1 --2- 172.21.15.90:0/782299800 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2824104700 0x7f28241a03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:27.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:27.036+0000 7f282c400700 1 --2- 172.21.15.90:0/782299800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2824102b60 0x7f2824198e40 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:27.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:27.036+0000 7f282c400700 1 --2- 172.21.15.90:0/782299800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2824103d60 0x7f2824199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:27.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:27.036+0000 7f282c400700 1 -- 172.21.15.90:0/782299800 >> 172.21.15.90:0/782299800 conn(0x7f28240fe0c0 msgr2=0x7f2824107180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:27.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:27.036+0000 7f282c400700 1 -- 172.21.15.90:0/782299800 shutdown_connections 2024-04-24T09:21:27.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:27.036+0000 7f282c400700 1 -- 172.21.15.90:0/782299800 wait complete. 2024-04-24T09:21:27.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:26 smithi149 ceph-mon[32592]: pgmap v392: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-24T09:21:27.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:26 smithi114 ceph-mon[32501]: pgmap v392: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-24T09:21:27.950 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-24T09:21:27.959 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi090.front.sepia.ceph.com 2024-04-24T09:21:27.960 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-24T09:21:28.416 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:21:28.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:28 smithi090 ceph-mon[27825]: from='client.25096 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:21:28.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:28 smithi090 ceph-mon[27825]: pgmap v393: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:21:29.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:28 smithi149 ceph-mon[32592]: from='client.25096 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:21:29.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:28 smithi149 ceph-mon[32592]: pgmap v393: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:21:29.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:28 smithi114 ceph-mon[32501]: from='client.25096 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:21:29.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:28 smithi114 ceph-mon[32501]: pgmap v393: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:21:30.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:30.727+0000 7fa253800700 1 -- 172.21.15.90:0/1845672178 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa24c108840 msgr2=0x7fa24c10acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:30.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:30.727+0000 7fa253800700 1 --2- 172.21.15.90:0/1845672178 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa24c108840 0x7fa24c10acb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa24400b0a0 tx=0x7fa24400b3b0 comp rx=0 tx=0).stop 2024-04-24T09:21:30.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:30.727+0000 7fa253800700 1 -- 172.21.15.90:0/1845672178 shutdown_connections 2024-04-24T09:21:30.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:30.727+0000 7fa253800700 1 --2- 172.21.15.90:0/1845672178 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa24c108840 0x7fa24c10acb0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:30.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:30.727+0000 7fa253800700 1 --2- 172.21.15.90:0/1845672178 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa24c1035e0 0x7fa24c1059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:30.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:30.727+0000 7fa253800700 1 --2- 172.21.15.90:0/1845672178 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa24c105f10 0x7fa24c108300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:30.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:30.727+0000 7fa253800700 1 -- 172.21.15.90:0/1845672178 >> 172.21.15.90:0/1845672178 conn(0x7fa24c0fcff0 msgr2=0x7fa24c0ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:30.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:30.727+0000 7fa253800700 1 -- 172.21.15.90:0/1845672178 shutdown_connections 2024-04-24T09:21:30.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:30.728+0000 7fa253800700 1 -- 172.21.15.90:0/1845672178 wait complete. 2024-04-24T09:21:30.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:30.729+0000 7fa253800700 1 Processor -- start 2024-04-24T09:21:30.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:30.729+0000 7fa253800700 1 -- start start 2024-04-24T09:21:30.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:30.730+0000 7fa253800700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa24c1035e0 0x7fa24c19af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:30.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:30.730+0000 7fa253800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa24c105f10 0x7fa24c19b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:30.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:30.730+0000 7fa253800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa24c108840 0x7fa24c1a2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:30.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:30.730+0000 7fa253800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa24c10e750 con 0x7fa24c108840 2024-04-24T09:21:30.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:30.730+0000 7fa253800700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa24c10e5d0 con 0x7fa24c1035e0 2024-04-24T09:21:30.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:30.730+0000 7fa253800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa24c10e8d0 con 0x7fa24c105f10 2024-04-24T09:21:30.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:30.730+0000 7fa250e00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa24c1035e0 0x7fa24c19af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:30.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:30.730+0000 7fa250e00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa24c1035e0 0x7fa24c19af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.90:38742/0 (socket says 172.21.15.90:38742) 2024-04-24T09:21:30.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:30.730+0000 7fa250e00700 1 -- 172.21.15.90:0/2954995435 learned_addr learned my addr 172.21.15.90:0/2954995435 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:30.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:30.730+0000 7fa24be00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa24c105f10 0x7fa24c19b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:30.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:30.730+0000 7fa251800700 1 --2- 172.21.15.90:0/2954995435 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa24c108840 0x7fa24c1a2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:30.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:30.730+0000 7fa250e00700 1 -- 172.21.15.90:0/2954995435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa24c105f10 msgr2=0x7fa24c19b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:30.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:30.730+0000 7fa250e00700 1 --2- 172.21.15.90:0/2954995435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa24c105f10 0x7fa24c19b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:30.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:30.730+0000 7fa250e00700 1 -- 172.21.15.90:0/2954995435 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa24c108840 msgr2=0x7fa24c1a2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:30.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:30.730+0000 7fa250e00700 1 --2- 172.21.15.90:0/2954995435 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa24c108840 0x7fa24c1a2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:30.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:30.731+0000 7fa250e00700 1 -- 172.21.15.90:0/2954995435 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa24c1a2bf0 con 0x7fa24c1035e0 2024-04-24T09:21:30.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:30.731+0000 7fa251800700 1 --2- 172.21.15.90:0/2954995435 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa24c108840 0x7fa24c1a2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:21:30.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:30.731+0000 7fa250e00700 1 --2- 172.21.15.90:0/2954995435 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa24c1035e0 0x7fa24c19af90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa24000e9a0 tx=0x7fa24000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:30.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:30.731+0000 7fa249600700 1 -- 172.21.15.90:0/2954995435 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24000cbd0 con 0x7fa24c1035e0 2024-04-24T09:21:30.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:30.731+0000 7fa249600700 1 -- 172.21.15.90:0/2954995435 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa24000cd70 con 0x7fa24c1035e0 2024-04-24T09:21:30.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:30.731+0000 7fa253800700 1 -- 172.21.15.90:0/2954995435 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa24c1a2ee0 con 0x7fa24c1035e0 2024-04-24T09:21:30.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:30.731+0000 7fa249600700 1 -- 172.21.15.90:0/2954995435 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2400187a0 con 0x7fa24c1035e0 2024-04-24T09:21:30.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:30.731+0000 7fa253800700 1 -- 172.21.15.90:0/2954995435 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa24c1a3340 con 0x7fa24c1035e0 2024-04-24T09:21:30.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:30.733+0000 7fa253800700 1 -- 172.21.15.90:0/2954995435 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa24c0697e0 con 0x7fa24c1035e0 2024-04-24T09:21:30.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:30.736+0000 7fa249600700 1 -- 172.21.15.90:0/2954995435 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa2400109a0 con 0x7fa24c1035e0 2024-04-24T09:21:30.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:30.738+0000 7fa249600700 1 --2- 172.21.15.90:0/2954995435 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa2340a3cc0 0x7fa2340a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:30.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:30.738+0000 7fa249600700 1 -- 172.21.15.90:0/2954995435 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fa2400d0590 con 0x7fa24c1035e0 2024-04-24T09:21:30.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:30.738+0000 7fa24be00700 1 --2- 172.21.15.90:0/2954995435 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa2340a3cc0 0x7fa2340a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:30.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:30.739+0000 7fa24be00700 1 --2- 172.21.15.90:0/2954995435 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa2340a3cc0 0x7fa2340a6180 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa238009680 tx=0x7fa238009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:30.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:30.742+0000 7fa249600700 1 -- 172.21.15.90:0/2954995435 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa24009cf30 con 0x7fa24c1035e0 2024-04-24T09:21:31.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:31.049+0000 7fa253800700 1 -- 172.21.15.90:0/2954995435 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fa24c0637d0 con 0x7fa24c1035e0 2024-04-24T09:21:31.050 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:30 smithi090 ceph-mon[27825]: pgmap v394: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-24T09:21:31.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:31.050+0000 7fa249600700 1 -- 172.21.15.90:0/2954995435 <== mon.2 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+3087 (secure 0 0 0) 0x7fa24009c830 con 0x7fa24c1035e0 2024-04-24T09:21:31.051 INFO:teuthology.orchestra.run.smithi090.stdout:e27 2024-04-24T09:21:31.051 INFO:teuthology.orchestra.run.smithi090.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T09:21:31.051 INFO:teuthology.orchestra.run.smithi090.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-24T09:21:31.051 INFO:teuthology.orchestra.run.smithi090.stdout:legacy client fscid: 1 2024-04-24T09:21:31.051 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:21:31.051 INFO:teuthology.orchestra.run.smithi090.stdout:Filesystem 'cephfs' (1) 2024-04-24T09:21:31.052 INFO:teuthology.orchestra.run.smithi090.stdout:fs_name cephfs 2024-04-24T09:21:31.052 INFO:teuthology.orchestra.run.smithi090.stdout:epoch 27 2024-04-24T09:21:31.052 INFO:teuthology.orchestra.run.smithi090.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-24T09:21:31.052 INFO:teuthology.orchestra.run.smithi090.stdout:created 2024-04-24T09:19:09.430875+0000 2024-04-24T09:21:31.052 INFO:teuthology.orchestra.run.smithi090.stdout:modified 2024-04-24T09:19:17.259940+0000 2024-04-24T09:21:31.052 INFO:teuthology.orchestra.run.smithi090.stdout:tableserver 0 2024-04-24T09:21:31.052 INFO:teuthology.orchestra.run.smithi090.stdout:root 0 2024-04-24T09:21:31.052 INFO:teuthology.orchestra.run.smithi090.stdout:session_timeout 300 2024-04-24T09:21:31.052 INFO:teuthology.orchestra.run.smithi090.stdout:session_autoclose 300 2024-04-24T09:21:31.052 INFO:teuthology.orchestra.run.smithi090.stdout:max_file_size 1099511627776 2024-04-24T09:21:31.052 INFO:teuthology.orchestra.run.smithi090.stdout:max_xattr_size 65536 2024-04-24T09:21:31.052 INFO:teuthology.orchestra.run.smithi090.stdout:required_client_features {} 2024-04-24T09:21:31.052 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure 0 2024-04-24T09:21:31.052 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure_osd_epoch 0 2024-04-24T09:21:31.053 INFO:teuthology.orchestra.run.smithi090.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-24T09:21:31.053 INFO:teuthology.orchestra.run.smithi090.stdout:max_mds 5 2024-04-24T09:21:31.053 INFO:teuthology.orchestra.run.smithi090.stdout:in 0,1,2,3,4 2024-04-24T09:21:31.053 INFO:teuthology.orchestra.run.smithi090.stdout:up {0=24557,1=24547,2=24545,3=24526,4=24446} 2024-04-24T09:21:31.053 INFO:teuthology.orchestra.run.smithi090.stdout:failed 2024-04-24T09:21:31.053 INFO:teuthology.orchestra.run.smithi090.stdout:damaged 2024-04-24T09:21:31.053 INFO:teuthology.orchestra.run.smithi090.stdout:stopped 2024-04-24T09:21:31.053 INFO:teuthology.orchestra.run.smithi090.stdout:data_pools [3] 2024-04-24T09:21:31.053 INFO:teuthology.orchestra.run.smithi090.stdout:metadata_pool 2 2024-04-24T09:21:31.053 INFO:teuthology.orchestra.run.smithi090.stdout:inline_data disabled 2024-04-24T09:21:31.053 INFO:teuthology.orchestra.run.smithi090.stdout:balancer 2024-04-24T09:21:31.053 INFO:teuthology.orchestra.run.smithi090.stdout:bal_rank_mask -1 2024-04-24T09:21:31.053 INFO:teuthology.orchestra.run.smithi090.stdout:standby_count_wanted 0 2024-04-24T09:21:31.053 INFO:teuthology.orchestra.run.smithi090.stdout:qdb_cluster leader: 24557 members: 24545,24557,24446,24526,24547 2024-04-24T09:21:31.053 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.b{0:24557} state up:active seq 3 addr [v2:172.21.15.114:6838/1025403710,v1:172.21.15.114:6839/1025403710] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T09:21:31.053 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.f{0:24514} state up:standby-replay seq 1 addr [v2:172.21.15.149:6834/3157375624,v1:172.21.15.149:6835/3157375624] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T09:21:31.054 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.i{1:24547} state up:active seq 7 addr [v2:172.21.15.149:6838/652734263,v1:172.21.15.149:6839/652734263] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T09:21:31.054 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.h{1:24500} state up:standby-replay seq 1 addr [v2:172.21.15.114:6836/875324280,v1:172.21.15.114:6837/875324280] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T09:21:31.054 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.a{2:24545} state up:active seq 5 addr [v2:172.21.15.90:6840/397593398,v1:172.21.15.90:6841/397593398] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T09:21:31.054 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.e{2:24476} state up:standby-replay seq 1 addr [v2:172.21.15.114:6834/2565340800,v1:172.21.15.114:6835/2565340800] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T09:21:31.054 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.c{3:24526} state up:active seq 9 addr [v2:172.21.15.149:6836/2250310587,v1:172.21.15.149:6837/2250310587] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T09:21:31.054 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.l{3:24451} state up:standby-replay seq 1 addr [v2:172.21.15.149:6832/779642417,v1:172.21.15.149:6833/779642417] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T09:21:31.054 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.j{4:24446} state up:active seq 12 addr [v2:172.21.15.90:6834/2396995634,v1:172.21.15.90:6835/2396995634] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T09:21:31.054 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.k{4:24428} state up:standby-replay seq 1 addr [v2:172.21.15.114:6832/219377141,v1:172.21.15.114:6833/219377141] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T09:21:31.054 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:21:31.054 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:21:31.054 INFO:teuthology.orchestra.run.smithi090.stdout:Standby daemons: 2024-04-24T09:21:31.054 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:21:31.054 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.d{-1:14403} state up:standby seq 1 addr [v2:172.21.15.90:6836/227251251,v1:172.21.15.90:6837/227251251] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T09:21:31.054 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.g{-1:14451} state up:standby seq 1 addr [v2:172.21.15.90:6838/1904617971,v1:172.21.15.90:6839/1904617971] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T09:21:31.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:31.053+0000 7fa253800700 1 -- 172.21.15.90:0/2954995435 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa2340a3cc0 msgr2=0x7fa2340a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:31.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:31.053+0000 7fa253800700 1 --2- 172.21.15.90:0/2954995435 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa2340a3cc0 0x7fa2340a6180 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa238009680 tx=0x7fa238009260 comp rx=0 tx=0).stop 2024-04-24T09:21:31.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:31.053+0000 7fa253800700 1 -- 172.21.15.90:0/2954995435 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa24c1035e0 msgr2=0x7fa24c19af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:31.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:31.053+0000 7fa253800700 1 --2- 172.21.15.90:0/2954995435 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa24c1035e0 0x7fa24c19af90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa24000e9a0 tx=0x7fa24000ecb0 comp rx=0 tx=0).stop 2024-04-24T09:21:31.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:31.053+0000 7fa253800700 1 -- 172.21.15.90:0/2954995435 shutdown_connections 2024-04-24T09:21:31.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:31.053+0000 7fa253800700 1 --2- 172.21.15.90:0/2954995435 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa2340a3cc0 0x7fa2340a6180 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:31.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:31.053+0000 7fa253800700 1 --2- 172.21.15.90:0/2954995435 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa24c108840 0x7fa24c1a2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:31.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:31.053+0000 7fa253800700 1 --2- 172.21.15.90:0/2954995435 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa24c1035e0 0x7fa24c19af90 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:31.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:31.053+0000 7fa253800700 1 --2- 172.21.15.90:0/2954995435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa24c105f10 0x7fa24c19b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:31.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:31.053+0000 7fa253800700 1 -- 172.21.15.90:0/2954995435 >> 172.21.15.90:0/2954995435 conn(0x7fa24c0fcff0 msgr2=0x7fa24c102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:31.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:31.054+0000 7fa253800700 1 -- 172.21.15.90:0/2954995435 shutdown_connections 2024-04-24T09:21:31.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:31.054+0000 7fa253800700 1 -- 172.21.15.90:0/2954995435 wait complete. 2024-04-24T09:21:31.056 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 27 2024-04-24T09:21:31.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:30 smithi149 ceph-mon[32592]: pgmap v394: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-24T09:21:31.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:30 smithi114 ceph-mon[32501]: pgmap v394: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-24T09:21:31.899 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-24T09:21:32.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:31 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/2954995435' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T09:21:32.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:31 smithi090 ceph-mon[27825]: pgmap v395: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T09:21:32.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:31 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/2954995435' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T09:21:32.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:31 smithi149 ceph-mon[32592]: pgmap v395: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T09:21:32.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:31 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/2954995435' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T09:21:32.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:31 smithi114 ceph-mon[32501]: pgmap v395: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T09:21:32.310 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:21:34.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.550+0000 7fe9b5200700 1 -- 172.21.15.90:0/3474053767 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9b01035e0 msgr2=0x7fe9b01059d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:34.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.550+0000 7fe9b5200700 1 --2- 172.21.15.90:0/3474053767 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9b01035e0 0x7fe9b01059d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0009990 tx=0x7fe9a0009ca0 comp rx=0 tx=0).stop 2024-04-24T09:21:34.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.550+0000 7fe9b5200700 1 -- 172.21.15.90:0/3474053767 shutdown_connections 2024-04-24T09:21:34.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.550+0000 7fe9b5200700 1 --2- 172.21.15.90:0/3474053767 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9b0108840 0x7fe9b010acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:34.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.550+0000 7fe9b5200700 1 --2- 172.21.15.90:0/3474053767 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9b01035e0 0x7fe9b01059d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:34.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.550+0000 7fe9b5200700 1 --2- 172.21.15.90:0/3474053767 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe9b0105f10 0x7fe9b0108300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:34.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.550+0000 7fe9b5200700 1 -- 172.21.15.90:0/3474053767 >> 172.21.15.90:0/3474053767 conn(0x7fe9b00fcff0 msgr2=0x7fe9b00ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:34.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.550+0000 7fe9b5200700 1 -- 172.21.15.90:0/3474053767 shutdown_connections 2024-04-24T09:21:34.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.550+0000 7fe9b5200700 1 -- 172.21.15.90:0/3474053767 wait complete. 2024-04-24T09:21:34.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.551+0000 7fe9b5200700 1 Processor -- start 2024-04-24T09:21:34.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.552+0000 7fe9b5200700 1 -- start start 2024-04-24T09:21:34.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.552+0000 7fe9b5200700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe9b01035e0 0x7fe9b019b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:34.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.552+0000 7fe9b5200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9b0105f10 0x7fe9b019b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:34.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.552+0000 7fe9b5200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9b0108840 0x7fe9b01a25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:34.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.552+0000 7fe9b5200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b010e7a0 con 0x7fe9b0108840 2024-04-24T09:21:34.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.552+0000 7fe9b5200700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b010e620 con 0x7fe9b01035e0 2024-04-24T09:21:34.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.552+0000 7fe9b5200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b010e920 con 0x7fe9b0105f10 2024-04-24T09:21:34.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.552+0000 7fe9ae800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9b0108840 0x7fe9b01a25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:34.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.552+0000 7fe9ad400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9b0105f10 0x7fe9b019b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:34.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.552+0000 7fe9ade00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe9b01035e0 0x7fe9b019b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:34.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.552+0000 7fe9ad400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9b0105f10 0x7fe9b019b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:47404/0 (socket says 172.21.15.90:47404) 2024-04-24T09:21:34.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.552+0000 7fe9ad400700 1 -- 172.21.15.90:0/2148332631 learned_addr learned my addr 172.21.15.90:0/2148332631 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:34.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.552+0000 7fe9ae800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9b0108840 0x7fe9b01a25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40128/0 (socket says 172.21.15.90:40128) 2024-04-24T09:21:34.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.553+0000 7fe9ae800700 1 -- 172.21.15.90:0/2148332631 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9b0105f10 msgr2=0x7fe9b019b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:34.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.553+0000 7fe9ae800700 1 --2- 172.21.15.90:0/2148332631 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9b0105f10 0x7fe9b019b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:34.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.553+0000 7fe9ae800700 1 -- 172.21.15.90:0/2148332631 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe9b01035e0 msgr2=0x7fe9b019b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:34.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.553+0000 7fe9ae800700 1 --2- 172.21.15.90:0/2148332631 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe9b01035e0 0x7fe9b019b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:34.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.553+0000 7fe9ae800700 1 -- 172.21.15.90:0/2148332631 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9b01a2c60 con 0x7fe9b0108840 2024-04-24T09:21:34.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.553+0000 7fe9ad400700 1 --2- 172.21.15.90:0/2148332631 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9b0105f10 0x7fe9b019b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:21:34.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.553+0000 7fe9ae800700 1 --2- 172.21.15.90:0/2148332631 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9b0108840 0x7fe9b01a25c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe9a800efc0 tx=0x7fe9a800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:34.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.554+0000 7fe9a6a00700 1 -- 172.21.15.90:0/2148332631 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a80092e0 con 0x7fe9b0108840 2024-04-24T09:21:34.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.554+0000 7fe9a6a00700 1 -- 172.21.15.90:0/2148332631 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9a8004bb0 con 0x7fe9b0108840 2024-04-24T09:21:34.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.554+0000 7fe9b5200700 1 -- 172.21.15.90:0/2148332631 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9b01a2ef0 con 0x7fe9b0108840 2024-04-24T09:21:34.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.554+0000 7fe9a6a00700 1 -- 172.21.15.90:0/2148332631 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a8004280 con 0x7fe9b0108840 2024-04-24T09:21:34.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.554+0000 7fe9b5200700 1 -- 172.21.15.90:0/2148332631 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9b01a3350 con 0x7fe9b0108840 2024-04-24T09:21:34.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.555+0000 7fe9b5200700 1 -- 172.21.15.90:0/2148332631 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe990005360 con 0x7fe9b0108840 2024-04-24T09:21:34.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.557+0000 7fe9a6a00700 1 -- 172.21.15.90:0/2148332631 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe9a801b030 con 0x7fe9b0108840 2024-04-24T09:21:34.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.558+0000 7fe9a6a00700 1 --2- 172.21.15.90:0/2148332631 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fe9940a3c40 0x7fe9940a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:34.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.559+0000 7fe9a6a00700 1 -- 172.21.15.90:0/2148332631 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fe9a80d0790 con 0x7fe9b0108840 2024-04-24T09:21:34.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.559+0000 7fe9ade00700 1 --2- 172.21.15.90:0/2148332631 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fe9940a3c40 0x7fe9940a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:34.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.559+0000 7fe9ade00700 1 --2- 172.21.15.90:0/2148332631 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fe9940a3c40 0x7fe9940a6100 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe9a000b7b0 tx=0x7fe9a0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:34.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.561+0000 7fe9a6a00700 1 -- 172.21.15.90:0/2148332631 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe9a8022020 con 0x7fe9b0108840 2024-04-24T09:21:34.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.790+0000 7fe9b5200700 1 -- 172.21.15.90:0/2148332631 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fe990005920 con 0x7fe9b0108840 2024-04-24T09:21:34.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.792+0000 7fe9a6a00700 1 -- 172.21.15.90:0/2148332631 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v88) ==== 56+0+5310 (secure 0 0 0) 0x7fe9a8010040 con 0x7fe9b0108840 2024-04-24T09:21:34.792 INFO:teuthology.orchestra.run.smithi090.stdout:epoch 88 2024-04-24T09:21:34.792 INFO:teuthology.orchestra.run.smithi090.stdout:fsid 30ee9be2-021a-11ef-bc93-c7b262605968 2024-04-24T09:21:34.792 INFO:teuthology.orchestra.run.smithi090.stdout:created 2024-04-24T09:09:49.496019+0000 2024-04-24T09:21:34.792 INFO:teuthology.orchestra.run.smithi090.stdout:modified 2024-04-24T09:19:58.609640+0000 2024-04-24T09:21:34.793 INFO:teuthology.orchestra.run.smithi090.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-24T09:21:34.793 INFO:teuthology.orchestra.run.smithi090.stdout:crush_version 26 2024-04-24T09:21:34.793 INFO:teuthology.orchestra.run.smithi090.stdout:full_ratio 0.95 2024-04-24T09:21:34.793 INFO:teuthology.orchestra.run.smithi090.stdout:backfillfull_ratio 0.9 2024-04-24T09:21:34.793 INFO:teuthology.orchestra.run.smithi090.stdout:nearfull_ratio 0.85 2024-04-24T09:21:34.793 INFO:teuthology.orchestra.run.smithi090.stdout:require_min_compat_client luminous 2024-04-24T09:21:34.793 INFO:teuthology.orchestra.run.smithi090.stdout:min_compat_client luminous 2024-04-24T09:21:34.793 INFO:teuthology.orchestra.run.smithi090.stdout:require_osd_release squid 2024-04-24T09:21:34.793 INFO:teuthology.orchestra.run.smithi090.stdout:stretch_mode_enabled false 2024-04-24T09:21:34.793 INFO:teuthology.orchestra.run.smithi090.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-24T09:21:34.793 INFO:teuthology.orchestra.run.smithi090.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-04-24T09:21:34.793 INFO:teuthology.orchestra.run.smithi090.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-24T09:21:34.793 INFO:teuthology.orchestra.run.smithi090.stdout:max_osd 12 2024-04-24T09:21:34.793 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 up in weight 1 up_from 8 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.90:6802/2864288066,v1:172.21.15.90:6803/2864288066] [v2:172.21.15.90:6804/2864288066,v1:172.21.15.90:6805/2864288066] exists,up 13a59b9f-b46e-4ec4-90af-bb7a3d5c753b 2024-04-24T09:21:34.794 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 up in weight 1 up_from 13 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.114:6800/344222414,v1:172.21.15.114:6801/344222414] [v2:172.21.15.114:6802/344222414,v1:172.21.15.114:6803/344222414] exists,up 1567ce8f-24e6-42cf-bf64-f6f0c534a9bd 2024-04-24T09:21:34.794 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 up in weight 1 up_from 18 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6800/3257067419,v1:172.21.15.149:6801/3257067419] [v2:172.21.15.149:6802/3257067419,v1:172.21.15.149:6803/3257067419] exists,up f861b304-9c7d-4629-8522-37f33a747945 2024-04-24T09:21:34.794 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 up in weight 1 up_from 26 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.90:6810/3068524970,v1:172.21.15.90:6811/3068524970] [v2:172.21.15.90:6812/3068524970,v1:172.21.15.90:6813/3068524970] exists,up 9377ab6e-5996-45fb-93a9-475a2e5f17a7 2024-04-24T09:21:34.794 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 up in weight 1 up_from 34 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.114:6808/2782015522,v1:172.21.15.114:6809/2782015522] [v2:172.21.15.114:6810/2782015522,v1:172.21.15.114:6811/2782015522] exists,up e2ddd19e-e641-4d61-b342-02b5f12b1c4b 2024-04-24T09:21:34.794 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 up in weight 1 up_from 39 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6808/4266460479,v1:172.21.15.149:6809/4266460479] [v2:172.21.15.149:6810/4266460479,v1:172.21.15.149:6811/4266460479] exists,up b6ff9d10-5210-4b46-a192-6dcbf9d1f156 2024-04-24T09:21:34.794 INFO:teuthology.orchestra.run.smithi090.stdout:osd.6 up in weight 1 up_from 47 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.90:6818/1121461276,v1:172.21.15.90:6819/1121461276] [v2:172.21.15.90:6820/1121461276,v1:172.21.15.90:6821/1121461276] exists,up 4287b5a1-f953-421c-bf62-5e390bee834d 2024-04-24T09:21:34.794 INFO:teuthology.orchestra.run.smithi090.stdout:osd.7 up in weight 1 up_from 52 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.114:6816/3222546280,v1:172.21.15.114:6817/3222546280] [v2:172.21.15.114:6818/3222546280,v1:172.21.15.114:6819/3222546280] exists,up a54e507d-d6ba-4bf2-9a7f-a7b95c3859bd 2024-04-24T09:21:34.794 INFO:teuthology.orchestra.run.smithi090.stdout:osd.8 up in weight 1 up_from 60 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6816/157615149,v1:172.21.15.149:6817/157615149] [v2:172.21.15.149:6818/157615149,v1:172.21.15.149:6819/157615149] exists,up 9950b010-5865-4109-ae37-582b1dbd2f41 2024-04-24T09:21:34.794 INFO:teuthology.orchestra.run.smithi090.stdout:osd.9 up in weight 1 up_from 68 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.90:6826/1307318076,v1:172.21.15.90:6827/1307318076] [v2:172.21.15.90:6828/1307318076,v1:172.21.15.90:6829/1307318076] exists,up 31e2635c-e234-421f-9757-592fd673c936 2024-04-24T09:21:34.794 INFO:teuthology.orchestra.run.smithi090.stdout:osd.10 up in weight 1 up_from 73 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.114:6824/1719802140,v1:172.21.15.114:6825/1719802140] [v2:172.21.15.114:6826/1719802140,v1:172.21.15.114:6827/1719802140] exists,up 46a3b2a6-e6e5-4578-b4bb-f3e998b9bc95 2024-04-24T09:21:34.794 INFO:teuthology.orchestra.run.smithi090.stdout:osd.11 up in weight 1 up_from 81 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6824/1737275166,v1:172.21.15.149:6825/1737275166] [v2:172.21.15.149:6826/1737275166,v1:172.21.15.149:6827/1737275166] exists,up ccd4456b-e947-46cf-a779-148ffa289b58 2024-04-24T09:21:34.794 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 2.0 [8,4] 2024-04-24T09:21:34.794 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 2.6 [8,11] 2024-04-24T09:21:34.794 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 2.a [2,11] 2024-04-24T09:21:34.794 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 2.c [2,4] 2024-04-24T09:21:34.795 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 3.0 [5,9] 2024-04-24T09:21:34.795 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 3.c [3,9] 2024-04-24T09:21:34.795 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 3.23 [4,7] 2024-04-24T09:21:34.795 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 3.31 [4,8] 2024-04-24T09:21:34.795 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 3.34 [4,9] 2024-04-24T09:21:34.795 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:0/3090852580 expires 2024-04-25T09:10:57.377655+0000 2024-04-24T09:21:34.795 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:0/687197394 expires 2024-04-25T09:10:57.377655+0000 2024-04-24T09:21:34.795 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:6801/1809038230 expires 2024-04-25T09:10:57.377655+0000 2024-04-24T09:21:34.795 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:6800/1809038230 expires 2024-04-25T09:10:57.377655+0000 2024-04-24T09:21:34.795 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:0/2964149071 expires 2024-04-25T09:10:22.599674+0000 2024-04-24T09:21:34.795 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:0/3617388113 expires 2024-04-25T09:10:22.599674+0000 2024-04-24T09:21:34.795 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:6801/3089140842 expires 2024-04-25T09:10:22.599674+0000 2024-04-24T09:21:34.795 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:6800/3089140842 expires 2024-04-25T09:10:22.599674+0000 2024-04-24T09:21:34.795 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:0/2840416149 expires 2024-04-25T09:10:22.599674+0000 2024-04-24T09:21:34.795 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:0/305432710 expires 2024-04-25T09:10:57.377655+0000 2024-04-24T09:21:34.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.795+0000 7fe9b5200700 1 -- 172.21.15.90:0/2148332631 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fe9940a3c40 msgr2=0x7fe9940a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:34.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.795+0000 7fe9b5200700 1 --2- 172.21.15.90:0/2148332631 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fe9940a3c40 0x7fe9940a6100 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe9a000b7b0 tx=0x7fe9a0005fd0 comp rx=0 tx=0).stop 2024-04-24T09:21:34.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.795+0000 7fe9b5200700 1 -- 172.21.15.90:0/2148332631 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9b0108840 msgr2=0x7fe9b01a25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:34.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.795+0000 7fe9b5200700 1 --2- 172.21.15.90:0/2148332631 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9b0108840 0x7fe9b01a25c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe9a800efc0 tx=0x7fe9a800c3a0 comp rx=0 tx=0).stop 2024-04-24T09:21:34.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.795+0000 7fe9b5200700 1 -- 172.21.15.90:0/2148332631 shutdown_connections 2024-04-24T09:21:34.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.795+0000 7fe9b5200700 1 --2- 172.21.15.90:0/2148332631 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fe9940a3c40 0x7fe9940a6100 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:34.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.795+0000 7fe9b5200700 1 --2- 172.21.15.90:0/2148332631 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9b0108840 0x7fe9b01a25c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:34.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.795+0000 7fe9b5200700 1 --2- 172.21.15.90:0/2148332631 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe9b01035e0 0x7fe9b019b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:34.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.795+0000 7fe9b5200700 1 --2- 172.21.15.90:0/2148332631 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9b0105f10 0x7fe9b019b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:34.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.795+0000 7fe9b5200700 1 -- 172.21.15.90:0/2148332631 >> 172.21.15.90:0/2148332631 conn(0x7fe9b00fcff0 msgr2=0x7fe9b0102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:34.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.796+0000 7fe9b5200700 1 -- 172.21.15.90:0/2148332631 shutdown_connections 2024-04-24T09:21:34.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:34.796+0000 7fe9b5200700 1 -- 172.21.15.90:0/2148332631 wait complete. 2024-04-24T09:21:34.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:34 smithi090 ceph-mon[27825]: pgmap v396: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:21:34.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:34 smithi114 ceph-mon[32501]: pgmap v396: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:21:35.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:34 smithi149 ceph-mon[32592]: pgmap v396: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:21:35.733 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-24T09:21:35.745 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T09:21:35.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:35 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/2148332631' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-24T09:21:35.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:35.957+0000 7fa592800640 1 -- 172.21.15.90:0/3838220606 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa58c077c00 msgr2=0x7fa58c078060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:35.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:35.957+0000 7fa592800640 1 --2- 172.21.15.90:0/3838220606 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa58c077c00 0x7fa58c078060 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa578009900 tx=0x7fa57802a410 comp rx=0 tx=0).stop 2024-04-24T09:21:35.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:35.957+0000 7fa592800640 1 -- 172.21.15.90:0/3838220606 shutdown_connections 2024-04-24T09:21:35.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:35.957+0000 7fa592800640 1 --2- 172.21.15.90:0/3838220606 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa58c0785a0 0x7fa58c078e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:35.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:35.957+0000 7fa592800640 1 --2- 172.21.15.90:0/3838220606 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa58c077c00 0x7fa58c078060 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:35.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:35.957+0000 7fa592800640 1 --2- 172.21.15.90:0/3838220606 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa58c0769c0 0x7fa58c076de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:35.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:35.957+0000 7fa592800640 1 -- 172.21.15.90:0/3838220606 >> 172.21.15.90:0/3838220606 conn(0x7fa58c0ff610 msgr2=0x7fa58c101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:35.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:35.957+0000 7fa592800640 1 -- 172.21.15.90:0/3838220606 shutdown_connections 2024-04-24T09:21:35.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:35.957+0000 7fa592800640 1 -- 172.21.15.90:0/3838220606 wait complete. 2024-04-24T09:21:35.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:35.958+0000 7fa592800640 1 Processor -- start 2024-04-24T09:21:35.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:35.958+0000 7fa592800640 1 -- start start 2024-04-24T09:21:35.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:35.958+0000 7fa592800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa58c0769c0 0x7fa58c19e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:35.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:35.958+0000 7fa592800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa58c077c00 0x7fa58c19ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:35.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:35.958+0000 7fa592800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa58c0785a0 0x7fa58c1a5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:35.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:35.958+0000 7fa592800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa58c114110 con 0x7fa58c0785a0 2024-04-24T09:21:35.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:35.958+0000 7fa592800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa58c113f90 con 0x7fa58c0769c0 2024-04-24T09:21:35.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:35.958+0000 7fa592800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa58c114290 con 0x7fa58c077c00 2024-04-24T09:21:35.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:35.959+0000 7fa590a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa58c077c00 0x7fa58c19ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:35.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:35.959+0000 7fa591e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa58c0785a0 0x7fa58c1a5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:35.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:35.959+0000 7fa591400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa58c0769c0 0x7fa58c19e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:35.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:35.959+0000 7fa591e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa58c0785a0 0x7fa58c1a5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42068/0 (socket says 172.21.15.90:42068) 2024-04-24T09:21:35.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:35.959+0000 7fa590a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa58c077c00 0x7fa58c19ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:54256/0 (socket says 172.21.15.90:54256) 2024-04-24T09:21:35.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:35.959+0000 7fa591e00640 1 -- 172.21.15.90:0/3451871899 learned_addr learned my addr 172.21.15.90:0/3451871899 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:35.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:35.959+0000 7fa591400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa58c0769c0 0x7fa58c19e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.90:55738/0 (socket says 172.21.15.90:55738) 2024-04-24T09:21:35.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:35.959+0000 7fa591e00640 1 -- 172.21.15.90:0/3451871899 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa58c077c00 msgr2=0x7fa58c19ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:35.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:35.959+0000 7fa591e00640 1 --2- 172.21.15.90:0/3451871899 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa58c077c00 0x7fa58c19ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:35.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:35.959+0000 7fa591e00640 1 -- 172.21.15.90:0/3451871899 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa58c0769c0 msgr2=0x7fa58c19e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:35.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:35.959+0000 7fa591e00640 1 --2- 172.21.15.90:0/3451871899 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa58c0769c0 0x7fa58c19e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:35.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:35.959+0000 7fa591e00640 1 -- 172.21.15.90:0/3451871899 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa58c1a6370 con 0x7fa58c0785a0 2024-04-24T09:21:35.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:35.959+0000 7fa591e00640 1 --2- 172.21.15.90:0/3451871899 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa58c0785a0 0x7fa58c1a5d00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa58400e970 tx=0x7fa58400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:35.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:35.960+0000 7fa58a000640 1 -- 172.21.15.90:0/3451871899 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58400ccc0 con 0x7fa58c0785a0 2024-04-24T09:21:35.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:35.960+0000 7fa58a000640 1 -- 172.21.15.90:0/3451871899 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa58400ce60 con 0x7fa58c0785a0 2024-04-24T09:21:35.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:35.960+0000 7fa58a000640 1 -- 172.21.15.90:0/3451871899 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa584010490 con 0x7fa58c0785a0 2024-04-24T09:21:35.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:35.960+0000 7fa592800640 1 -- 172.21.15.90:0/3451871899 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa58c1a6660 con 0x7fa58c0785a0 2024-04-24T09:21:35.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:35.960+0000 7fa592800640 1 -- 172.21.15.90:0/3451871899 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa58c1a6b30 con 0x7fa58c0785a0 2024-04-24T09:21:35.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:35.961+0000 7fa592800640 1 -- 172.21.15.90:0/3451871899 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa58c110b50 con 0x7fa58c0785a0 2024-04-24T09:21:35.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:35.963+0000 7fa58a000640 1 -- 172.21.15.90:0/3451871899 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa584005020 con 0x7fa58c0785a0 2024-04-24T09:21:35.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:35.965+0000 7fa58a000640 1 --2- 172.21.15.90:0/3451871899 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa5740a3bb0 0x7fa5740a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:35.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:35.966+0000 7fa58a000640 1 -- 172.21.15.90:0/3451871899 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fa5840d0860 con 0x7fa58c0785a0 2024-04-24T09:21:35.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:35.966+0000 7fa591400640 1 --2- 172.21.15.90:0/3451871899 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa5740a3bb0 0x7fa5740a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:35.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:35.966+0000 7fa591400640 1 --2- 172.21.15.90:0/3451871899 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa5740a3bb0 0x7fa5740a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa580005fd0 tx=0x7fa580009760 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:35.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:35.970+0000 7fa58a000640 1 -- 172.21.15.90:0/3451871899 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa58409d200 con 0x7fa58c0785a0 2024-04-24T09:21:36.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:35 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/2148332631' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-24T09:21:36.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:35 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/2148332631' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-24T09:21:36.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.176+0000 7fa592800640 1 -- 172.21.15.90:0/3451871899 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa58c062c00 con 0x7fa58c0785a0 2024-04-24T09:21:36.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.180+0000 7fa58a000640 1 -- 172.21.15.90:0/3451871899 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22652 (secure 0 0 0) 0x7fa58409cb00 con 0x7fa58c0785a0 2024-04-24T09:21:36.181 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:21:36.181 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":88,"fsid":"30ee9be2-021a-11ef-bc93-c7b262605968","created":"2024-04-24T09:09:49.496019+0000","modified":"2024-04-24T09:19:58.609640+0000","last_up_change":"2024-04-24T09:18:10.455632+0000","last_in_change":"2024-04-24T09:17:50.600366+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T09:13:45.548774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T09:19:07.185580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T09:19:08.374818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13a59b9f-b46e-4ec4-90af-bb7a3d5c753b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6803","nonce":2864288066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6805","nonce":2864288066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6809","nonce":2864288066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6807","nonce":2864288066}]},"public_addr":"172.21.15.90:6803/2864288066","cluster_addr":"172.21.15.90:6805/2864288066","heartbeat_back_addr":"172.21.15.90:6809/2864288066","heartbeat_front_addr":"172.21.15.90:6807/2864288066","state":["exists","up"]},{"osd":1,"uuid":"1567ce8f-24e6-42cf-bf64-f6f0c534a9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6801","nonce":344222414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6803","nonce":344222414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6807","nonce":344222414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6805","nonce":344222414}]},"public_addr":"172.21.15.114:6801/344222414","cluster_addr":"172.21.15.114:6803/344222414","heartbeat_back_addr":"172.21.15.114:6807/344222414","heartbeat_front_addr":"172.21.15.114:6805/344222414","state":["exists","up"]},{"osd":2,"uuid":"f861b304-9c7d-4629-8522-37f33a747945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6801","nonce":3257067419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6803","nonce":3257067419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6807","nonce":3257067419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6805","nonce":3257067419}]},"public_addr":"172.21.15.149:6801/3257067419","cluster_addr":"172.21.15.149:6803/3257067419","heartbeat_back_addr":"172.21.15.149:6807/3257067419","heartbeat_front_addr":"172.21.15.149:6805/3257067419","state":["exists","up"]},{"osd":3,"uuid":"9377ab6e-5996-45fb-93a9-475a2e5f17a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6811","nonce":3068524970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6813","nonce":3068524970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6817","nonce":3068524970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6815","nonce":3068524970}]},"public_addr":"172.21.15.90:6811/3068524970","cluster_addr":"172.21.15.90:6813/3068524970","heartbeat_back_addr":"172.21.15.90:6817/3068524970","heartbeat_front_addr":"172.21.15.90:6815/3068524970","state":["exists","up"]},{"osd":4,"uuid":"e2ddd19e-e641-4d61-b342-02b5f12b1c4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6809","nonce":2782015522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6811","nonce":2782015522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6815","nonce":2782015522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6813","nonce":2782015522}]},"public_addr":"172.21.15.114:6809/2782015522","cluster_addr":"172.21.15.114:6811/2782015522","heartbeat_back_addr":"172.21.15.114:6815/2782015522","heartbeat_front_addr":"172.21.15.114:6813/2782015522","state":["exists","up"]},{"osd":5,"uuid":"b6ff9d10-5210-4b46-a192-6dcbf9d1f156","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6809","nonce":4266460479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6811","nonce":4266460479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6815","nonce":4266460479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6813","nonce":4266460479}]},"public_addr":"172.21.15.149:6809/4266460479","cluster_addr":"172.21.15.149:6811/4266460479","heartbeat_back_addr":"172.21.15.149:6815/4266460479","heartbeat_front_addr":"172.21.15.149:6813/4266460479","state":["exists","up"]},{"osd":6,"uuid":"4287b5a1-f953-421c-bf62-5e390bee834d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6819","nonce":1121461276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6821","nonce":1121461276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6825","nonce":1121461276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6823","nonce":1121461276}]},"public_addr":"172.21.15.90:6819/1121461276","cluster_addr":"172.21.15.90:6821/1121461276","heartbeat_back_addr":"172.21.15.90:6825/1121461276","heartbeat_front_addr":"172.21.15.90:6823/1121461276","state":["exists","up"]},{"osd":7,"uuid":"a54e507d-d6ba-4bf2-9a7f-a7b95c3859bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6817","nonce":3222546280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6819","nonce":3222546280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6823","nonce":3222546280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6821","nonce":3222546280}]},"public_addr":"172.21.15.114:6817/3222546280","cluster_addr":"172.21.15.114:6819/3222546280","heartbeat_back_addr":"172.21.15.114:6823/3222546280","heartbeat_front_addr":"172.21.15.114:6821/3222546280","state":["exists","up"]},{"osd":8,"uuid":"9950b010-5865-4109-ae37-582b1dbd2f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6817","nonce":157615149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6819","nonce":157615149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6823","nonce":157615149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6821","nonce":157615149}]},"public_addr":"172.21.15.149:6817/157615149","cluster_addr":"172.21.15.149:6819/157615149","heartbeat_back_addr":"172.21.15.149:6823/157615149","heartbeat_front_addr":"172.21.15.149:6821/157615149","state":["exists","up"]},{"osd":9,"uuid":"31e2635c-e234-421f-9757-592fd673c936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6827","nonce":1307318076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6829","nonce":1307318076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6833","nonce":1307318076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6831","nonce":1307318076}]},"public_addr":"172.21.15.90:6827/1307318076","cluster_addr":"172.21.15.90:6829/1307318076","heartbeat_back_addr":"172.21.15.90:6833/1307318076","heartbeat_front_addr":"172.21.15.90:6831/1307318076","state":["exists","up"]},{"osd":10,"uuid":"46a3b2a6-e6e5-4578-b4bb-f3e998b9bc95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6825","nonce":1719802140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6827","nonce":1719802140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6831","nonce":1719802140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6829","nonce":1719802140}]},"public_addr":"172.21.15.114:6825/1719802140","cluster_addr":"172.21.15.114:6827/1719802140","heartbeat_back_addr":"172.21.15.114:6831/1719802140","heartbeat_front_addr":"172.21.15.114:6829/1719802140","state":["exists","up"]},{"osd":11,"uuid":"ccd4456b-e947-46cf-a779-148ffa289b58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6825","nonce":1737275166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6827","nonce":1737275166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6831","nonce":1737275166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6829","nonce":1737275166}]},"public_addr":"172.21.15.149:6825/1737275166","cluster_addr":"172.21.15.149:6827/1737275166","heartbeat_back_addr":"172.21.15.149:6831/1737275166","heartbeat_front_addr":"172.21.15.149:6829/1737275166","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:12:53.015078+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:13:18.047053+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:13:41.935891+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:14:15.095388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:14:42.512355+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:15:08.113775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:15:43.828550+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:16:10.901909+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:16:36.422962+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:17:13.924446+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:17:42.814401+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:18:08.672793+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/3090852580":"2024-04-25T09:10:57.377655+0000","172.21.15.90:0/687197394":"2024-04-25T09:10:57.377655+0000","172.21.15.90:6801/1809038230":"2024-04-25T09:10:57.377655+0000","172.21.15.90:6800/1809038230":"2024-04-25T09:10:57.377655+0000","172.21.15.90:0/2964149071":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/3617388113":"2024-04-25T09:10:22.599674+0000","172.21.15.90:6801/3089140842":"2024-04-25T09:10:22.599674+0000","172.21.15.90:6800/3089140842":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/2840416149":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/305432710":"2024-04-25T09:10:57.377655+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T09:21:36.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.183+0000 7fa592800640 1 -- 172.21.15.90:0/3451871899 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa5740a3bb0 msgr2=0x7fa5740a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:36.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.183+0000 7fa592800640 1 --2- 172.21.15.90:0/3451871899 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa5740a3bb0 0x7fa5740a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa580005fd0 tx=0x7fa580009760 comp rx=0 tx=0).stop 2024-04-24T09:21:36.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.183+0000 7fa592800640 1 -- 172.21.15.90:0/3451871899 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa58c0785a0 msgr2=0x7fa58c1a5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:36.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.183+0000 7fa592800640 1 --2- 172.21.15.90:0/3451871899 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa58c0785a0 0x7fa58c1a5d00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa58400e970 tx=0x7fa58400ed70 comp rx=0 tx=0).stop 2024-04-24T09:21:36.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.183+0000 7fa592800640 1 -- 172.21.15.90:0/3451871899 shutdown_connections 2024-04-24T09:21:36.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.183+0000 7fa592800640 1 --2- 172.21.15.90:0/3451871899 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa5740a3bb0 0x7fa5740a6070 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:36.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.183+0000 7fa592800640 1 --2- 172.21.15.90:0/3451871899 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa58c0785a0 0x7fa58c1a5d00 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:36.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.183+0000 7fa592800640 1 --2- 172.21.15.90:0/3451871899 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa58c077c00 0x7fa58c19ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:36.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.183+0000 7fa592800640 1 --2- 172.21.15.90:0/3451871899 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa58c0769c0 0x7fa58c19e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:36.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.183+0000 7fa592800640 1 -- 172.21.15.90:0/3451871899 >> 172.21.15.90:0/3451871899 conn(0x7fa58c0ff610 msgr2=0x7fa58c100ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:36.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.184+0000 7fa592800640 1 -- 172.21.15.90:0/3451871899 shutdown_connections 2024-04-24T09:21:36.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.184+0000 7fa592800640 1 -- 172.21.15.90:0/3451871899 wait complete. 2024-04-24T09:21:36.203 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T09:13:45.548774+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T09:19:07.185580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T09:19:08.374818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T09:21:36.204 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T09:21:36.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.413+0000 7f7ad9400640 1 -- 172.21.15.90:0/2470934589 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ad4109730 msgr2=0x7f7ad4109bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:36.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.413+0000 7f7ad9400640 1 --2- 172.21.15.90:0/2470934589 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ad4109730 0x7f7ad4109bb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7abc009900 tx=0x7f7abc02a410 comp rx=0 tx=0).stop 2024-04-24T09:21:36.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.413+0000 7f7ad9400640 1 -- 172.21.15.90:0/2470934589 shutdown_connections 2024-04-24T09:21:36.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.413+0000 7f7ad9400640 1 --2- 172.21.15.90:0/2470934589 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ad410a0f0 0x7f7ad4110980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:36.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.413+0000 7f7ad9400640 1 --2- 172.21.15.90:0/2470934589 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ad4109730 0x7f7ad4109bb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:36.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.413+0000 7f7ad9400640 1 --2- 172.21.15.90:0/2470934589 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ad4108540 0x7f7ad4108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:36.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.413+0000 7f7ad9400640 1 -- 172.21.15.90:0/2470934589 >> 172.21.15.90:0/2470934589 conn(0x7f7ad40774d0 msgr2=0x7f7ad4079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:36.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.413+0000 7f7ad9400640 1 -- 172.21.15.90:0/2470934589 shutdown_connections 2024-04-24T09:21:36.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.414+0000 7f7ad9400640 1 -- 172.21.15.90:0/2470934589 wait complete. 2024-04-24T09:21:36.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.415+0000 7f7ad9400640 1 Processor -- start 2024-04-24T09:21:36.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.415+0000 7f7ad9400640 1 -- start start 2024-04-24T09:21:36.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.415+0000 7f7ad9400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ad4108540 0x7f7ad419e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:36.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.415+0000 7f7ad9400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ad4109730 0x7f7ad419eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:36.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.415+0000 7f7ad9400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ad410a0f0 0x7f7ad41a5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:36.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.415+0000 7f7ad9400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad41141d0 con 0x7f7ad410a0f0 2024-04-24T09:21:36.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.415+0000 7f7ad9400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad4114050 con 0x7f7ad4109730 2024-04-24T09:21:36.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.415+0000 7f7ad9400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad4114350 con 0x7f7ad4108540 2024-04-24T09:21:36.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.416+0000 7f7acb400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ad4109730 0x7f7ad419eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:36.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.416+0000 7f7acb400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ad4109730 0x7f7ad419eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.90:55750/0 (socket says 172.21.15.90:55750) 2024-04-24T09:21:36.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.416+0000 7f7ad3e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ad4108540 0x7f7ad419e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:36.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.416+0000 7f7acb400640 1 -- 172.21.15.90:0/4101945402 learned_addr learned my addr 172.21.15.90:0/4101945402 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:36.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.416+0000 7f7ad8a00640 1 --2- 172.21.15.90:0/4101945402 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ad410a0f0 0x7f7ad41a5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:36.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.416+0000 7f7acb400640 1 -- 172.21.15.90:0/4101945402 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ad4108540 msgr2=0x7f7ad419e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:36.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.416+0000 7f7acb400640 1 --2- 172.21.15.90:0/4101945402 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ad4108540 0x7f7ad419e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:36.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.416+0000 7f7acb400640 1 -- 172.21.15.90:0/4101945402 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ad410a0f0 msgr2=0x7f7ad41a5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:36.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.416+0000 7f7acb400640 1 --2- 172.21.15.90:0/4101945402 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ad410a0f0 0x7f7ad41a5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:36.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.416+0000 7f7acb400640 1 -- 172.21.15.90:0/4101945402 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ad41a63e0 con 0x7f7ad4109730 2024-04-24T09:21:36.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.416+0000 7f7acb400640 1 --2- 172.21.15.90:0/4101945402 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ad4109730 0x7f7ad419eda0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7abc009230 tx=0x7f7abc0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:36.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.417+0000 7f7ad1600640 1 -- 172.21.15.90:0/4101945402 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7abc03f030 con 0x7f7ad4109730 2024-04-24T09:21:36.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.417+0000 7f7ad1600640 1 -- 172.21.15.90:0/4101945402 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7abc003ce0 con 0x7f7ad4109730 2024-04-24T09:21:36.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.417+0000 7f7ad1600640 1 -- 172.21.15.90:0/4101945402 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7abc035070 con 0x7f7ad4109730 2024-04-24T09:21:36.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.417+0000 7f7ad9400640 1 -- 172.21.15.90:0/4101945402 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ad41a6670 con 0x7f7ad4109730 2024-04-24T09:21:36.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.417+0000 7f7ad9400640 1 -- 172.21.15.90:0/4101945402 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ad41a6b10 con 0x7f7ad4109730 2024-04-24T09:21:36.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.418+0000 7f7ad9400640 1 -- 172.21.15.90:0/4101945402 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ad410f860 con 0x7f7ad4109730 2024-04-24T09:21:36.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.420+0000 7f7ad1600640 1 -- 172.21.15.90:0/4101945402 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7abc042030 con 0x7f7ad4109730 2024-04-24T09:21:36.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.421+0000 7f7ad1600640 1 --2- 172.21.15.90:0/4101945402 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f7ab80a3bb0 0x7f7ab80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:36.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.422+0000 7f7ad1600640 1 -- 172.21.15.90:0/4101945402 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f7abc0f0a40 con 0x7f7ad4109730 2024-04-24T09:21:36.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.422+0000 7f7ad3e00640 1 --2- 172.21.15.90:0/4101945402 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f7ab80a3bb0 0x7f7ab80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:36.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.422+0000 7f7ad3e00640 1 --2- 172.21.15.90:0/4101945402 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f7ab80a3bb0 0x7f7ab80a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4109590 tx=0x7f7ac4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:36.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.424+0000 7f7ad1600640 1 -- 172.21.15.90:0/4101945402 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7abc0bcd90 con 0x7f7ad4109730 2024-04-24T09:21:36.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.625+0000 7f7ad9400640 1 -- 172.21.15.90:0/4101945402 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7ad402d870 con 0x7f7ad4109730 2024-04-24T09:21:36.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.626+0000 7f7ad1600640 1 -- 172.21.15.90:0/4101945402 <== mon.2 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f7abc0c1090 con 0x7f7ad4109730 2024-04-24T09:21:36.626 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 1 2024-04-24T09:21:36.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.629+0000 7f7ad9400640 1 -- 172.21.15.90:0/4101945402 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f7ab80a3bb0 msgr2=0x7f7ab80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:36.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.629+0000 7f7ad9400640 1 --2- 172.21.15.90:0/4101945402 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f7ab80a3bb0 0x7f7ab80a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4109590 tx=0x7f7ac4005f90 comp rx=0 tx=0).stop 2024-04-24T09:21:36.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.629+0000 7f7ad9400640 1 -- 172.21.15.90:0/4101945402 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ad4109730 msgr2=0x7f7ad419eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:36.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.629+0000 7f7ad9400640 1 --2- 172.21.15.90:0/4101945402 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ad4109730 0x7f7ad419eda0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7abc009230 tx=0x7f7abc0042f0 comp rx=0 tx=0).stop 2024-04-24T09:21:36.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.629+0000 7f7ad9400640 1 -- 172.21.15.90:0/4101945402 shutdown_connections 2024-04-24T09:21:36.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.629+0000 7f7ad9400640 1 --2- 172.21.15.90:0/4101945402 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f7ab80a3bb0 0x7f7ab80a6070 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:36.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.629+0000 7f7ad9400640 1 --2- 172.21.15.90:0/4101945402 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ad410a0f0 0x7f7ad41a5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:36.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.629+0000 7f7ad9400640 1 --2- 172.21.15.90:0/4101945402 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7ad4109730 0x7f7ad419eda0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:36.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.629+0000 7f7ad9400640 1 --2- 172.21.15.90:0/4101945402 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ad4108540 0x7f7ad419e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:36.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.629+0000 7f7ad9400640 1 -- 172.21.15.90:0/4101945402 >> 172.21.15.90:0/4101945402 conn(0x7f7ad40774d0 msgr2=0x7f7ad410ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:36.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.629+0000 7f7ad9400640 1 -- 172.21.15.90:0/4101945402 shutdown_connections 2024-04-24T09:21:36.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.629+0000 7f7ad9400640 1 -- 172.21.15.90:0/4101945402 wait complete. 2024-04-24T09:21:36.647 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T09:21:36.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.856+0000 7fbad2600640 1 -- 172.21.15.90:0/2030813889 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbacc104280 msgr2=0x7fbacc1046a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:36.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.856+0000 7fbad2600640 1 --2- 172.21.15.90:0/2030813889 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbacc104280 0x7fbacc1046a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbabc009900 tx=0x7fbabc02a410 comp rx=0 tx=0).stop 2024-04-24T09:21:36.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.856+0000 7fbad2600640 1 -- 172.21.15.90:0/2030813889 shutdown_connections 2024-04-24T09:21:36.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.856+0000 7fbad2600640 1 --2- 172.21.15.90:0/2030813889 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbacc105e30 0x7fbacc10c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:36.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.856+0000 7fbad2600640 1 --2- 172.21.15.90:0/2030813889 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbacc105470 0x7fbacc1058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:36.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.856+0000 7fbad2600640 1 --2- 172.21.15.90:0/2030813889 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbacc104280 0x7fbacc1046a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:36.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.856+0000 7fbad2600640 1 -- 172.21.15.90:0/2030813889 >> 172.21.15.90:0/2030813889 conn(0x7fbacc0ff7f0 msgr2=0x7fbacc101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:36.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.856+0000 7fbad2600640 1 -- 172.21.15.90:0/2030813889 shutdown_connections 2024-04-24T09:21:36.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.856+0000 7fbad2600640 1 -- 172.21.15.90:0/2030813889 wait complete. 2024-04-24T09:21:36.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.857+0000 7fbad2600640 1 Processor -- start 2024-04-24T09:21:36.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.857+0000 7fbad2600640 1 -- start start 2024-04-24T09:21:36.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.857+0000 7fbad2600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbacc104280 0x7fbacc19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:36.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.857+0000 7fbad2600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbacc105470 0x7fbacc19aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:36.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.857+0000 7fbad2600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbacc105e30 0x7fbacc1a1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:36.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.858+0000 7fbad2600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fbacc10ff10 con 0x7fbacc105470 2024-04-24T09:21:36.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.858+0000 7fbad2600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fbacc10fd90 con 0x7fbacc104280 2024-04-24T09:21:36.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.858+0000 7fbad2600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbacc110090 con 0x7fbacc105e30 2024-04-24T09:21:36.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.858+0000 7fbacb400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbacc104280 0x7fbacc19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:36.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.858+0000 7fbacb400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbacc104280 0x7fbacc19a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.90:55770/0 (socket says 172.21.15.90:55770) 2024-04-24T09:21:36.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.858+0000 7fbacb400640 1 -- 172.21.15.90:0/2920449400 learned_addr learned my addr 172.21.15.90:0/2920449400 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:36.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.858+0000 7fbacaa00640 1 --2- 172.21.15.90:0/2920449400 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbacc105470 0x7fbacc19aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:36.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.858+0000 7fbacbe00640 1 --2- 172.21.15.90:0/2920449400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbacc105e30 0x7fbacc1a1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:36.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.858+0000 7fbacb400640 1 -- 172.21.15.90:0/2920449400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbacc105e30 msgr2=0x7fbacc1a1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:36.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.858+0000 7fbacb400640 1 --2- 172.21.15.90:0/2920449400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbacc105e30 0x7fbacc1a1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:36.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.858+0000 7fbacb400640 1 -- 172.21.15.90:0/2920449400 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbacc105470 msgr2=0x7fbacc19aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:36.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.858+0000 7fbacb400640 1 --2- 172.21.15.90:0/2920449400 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbacc105470 0x7fbacc19aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:36.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.858+0000 7fbacb400640 1 -- 172.21.15.90:0/2920449400 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbacc1a21c0 con 0x7fbacc104280 2024-04-24T09:21:36.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.859+0000 7fbacb400640 1 --2- 172.21.15.90:0/2920449400 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbacc104280 0x7fbacc19a5a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbabc031610 tx=0x7fbabc004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:36.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.859+0000 7fbac3e00640 1 -- 172.21.15.90:0/2920449400 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc03f030 con 0x7fbacc104280 2024-04-24T09:21:36.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.859+0000 7fbac3e00640 1 -- 172.21.15.90:0/2920449400 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbabc003ad0 con 0x7fbacc104280 2024-04-24T09:21:36.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.859+0000 7fbac3e00640 1 -- 172.21.15.90:0/2920449400 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc035070 con 0x7fbacc104280 2024-04-24T09:21:36.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.859+0000 7fbad2600640 1 -- 172.21.15.90:0/2920449400 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbacc1a2450 con 0x7fbacc104280 2024-04-24T09:21:36.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.859+0000 7fbad2600640 1 -- 172.21.15.90:0/2920449400 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbacc1a28f0 con 0x7fbacc104280 2024-04-24T09:21:36.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.860+0000 7fbad2600640 1 -- 172.21.15.90:0/2920449400 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbacc10b5a0 con 0x7fbacc104280 2024-04-24T09:21:36.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.862+0000 7fbac3e00640 1 -- 172.21.15.90:0/2920449400 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fbabc042030 con 0x7fbacc104280 2024-04-24T09:21:36.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.863+0000 7fbac3e00640 1 --2- 172.21.15.90:0/2920449400 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fbab00a3c30 0x7fbab00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:36.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.864+0000 7fbac3e00640 1 -- 172.21.15.90:0/2920449400 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fbabc0f0950 con 0x7fbacc104280 2024-04-24T09:21:36.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.864+0000 7fbacaa00640 1 --2- 172.21.15.90:0/2920449400 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fbab00a3c30 0x7fbab00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:36.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.864+0000 7fbacaa00640 1 --2- 172.21.15.90:0/2920449400 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fbab00a3c30 0x7fbab00a60f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbacc19bab0 tx=0x7fbab8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:36.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:36.868+0000 7fbac3e00640 1 -- 172.21.15.90:0/2920449400 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbabc0bcca0 con 0x7fbacc104280 2024-04-24T09:21:36.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:36 smithi114 ceph-mon[32501]: pgmap v397: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-24T09:21:36.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:36 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/3451871899' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:21:36.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:36 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/4101945402' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:21:37.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.068+0000 7fbad2600640 1 -- 172.21.15.90:0/2920449400 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbacc062c00 con 0x7fbacc104280 2024-04-24T09:21:37.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.069+0000 7fbac3e00640 1 -- 172.21.15.90:0/2920449400 <== mon.2 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7fbabc0c0fa0 con 0x7fbacc104280 2024-04-24T09:21:37.069 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 32 2024-04-24T09:21:37.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.071+0000 7fbad2600640 1 -- 172.21.15.90:0/2920449400 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fbab00a3c30 msgr2=0x7fbab00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:37.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.071+0000 7fbad2600640 1 --2- 172.21.15.90:0/2920449400 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fbab00a3c30 0x7fbab00a60f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbacc19bab0 tx=0x7fbab8005f90 comp rx=0 tx=0).stop 2024-04-24T09:21:37.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.072+0000 7fbad2600640 1 -- 172.21.15.90:0/2920449400 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbacc104280 msgr2=0x7fbacc19a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:37.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.072+0000 7fbad2600640 1 --2- 172.21.15.90:0/2920449400 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbacc104280 0x7fbacc19a5a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbabc031610 tx=0x7fbabc004270 comp rx=0 tx=0).stop 2024-04-24T09:21:37.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.072+0000 7fbad2600640 1 -- 172.21.15.90:0/2920449400 shutdown_connections 2024-04-24T09:21:37.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.072+0000 7fbad2600640 1 --2- 172.21.15.90:0/2920449400 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fbab00a3c30 0x7fbab00a60f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:37.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.072+0000 7fbad2600640 1 --2- 172.21.15.90:0/2920449400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbacc105e30 0x7fbacc1a1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:37.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.072+0000 7fbad2600640 1 --2- 172.21.15.90:0/2920449400 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbacc105470 0x7fbacc19aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:37.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.072+0000 7fbad2600640 1 --2- 172.21.15.90:0/2920449400 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbacc104280 0x7fbacc19a5a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:37.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.072+0000 7fbad2600640 1 -- 172.21.15.90:0/2920449400 >> 172.21.15.90:0/2920449400 conn(0x7fbacc0ff7f0 msgr2=0x7fbacc108770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:37.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.072+0000 7fbad2600640 1 -- 172.21.15.90:0/2920449400 shutdown_connections 2024-04-24T09:21:37.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.072+0000 7fbad2600640 1 -- 172.21.15.90:0/2920449400 wait complete. 2024-04-24T09:21:37.087 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:36 smithi090 ceph-mon[27825]: pgmap v397: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-24T09:21:37.087 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:36 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/3451871899' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:21:37.087 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:36 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/4101945402' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:21:37.090 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T09:21:37.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:36 smithi149 ceph-mon[32592]: pgmap v397: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-24T09:21:37.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:36 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/3451871899' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:21:37.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:36 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/4101945402' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:21:37.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.288+0000 7ff019c00640 1 -- 172.21.15.90:0/3057156223 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff014109730 msgr2=0x7ff014109bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:37.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.288+0000 7ff019c00640 1 --2- 172.21.15.90:0/3057156223 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff014109730 0x7ff014109bb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff000009900 tx=0x7ff00002a3f0 comp rx=0 tx=0).stop 2024-04-24T09:21:37.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.288+0000 7ff019c00640 1 -- 172.21.15.90:0/3057156223 shutdown_connections 2024-04-24T09:21:37.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.288+0000 7ff019c00640 1 --2- 172.21.15.90:0/3057156223 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff01410a0f0 0x7ff014110980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:37.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.288+0000 7ff019c00640 1 --2- 172.21.15.90:0/3057156223 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff014109730 0x7ff014109bb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:37.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.288+0000 7ff019c00640 1 --2- 172.21.15.90:0/3057156223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff014108540 0x7ff014108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:37.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.288+0000 7ff019c00640 1 -- 172.21.15.90:0/3057156223 >> 172.21.15.90:0/3057156223 conn(0x7ff0140774d0 msgr2=0x7ff014079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:37.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.288+0000 7ff019c00640 1 -- 172.21.15.90:0/3057156223 shutdown_connections 2024-04-24T09:21:37.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.288+0000 7ff019c00640 1 -- 172.21.15.90:0/3057156223 wait complete. 2024-04-24T09:21:37.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.289+0000 7ff019c00640 1 Processor -- start 2024-04-24T09:21:37.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.289+0000 7ff019c00640 1 -- start start 2024-04-24T09:21:37.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.290+0000 7ff019c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff014108540 0x7ff01419e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:37.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.290+0000 7ff019c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff014109730 0x7ff01419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:37.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.290+0000 7ff019c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff01410a0f0 0x7ff0141a5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:37.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.290+0000 7ff019c00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff014114220 con 0x7ff014109730 2024-04-24T09:21:37.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.290+0000 7ff019c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff0141140a0 con 0x7ff014108540 2024-04-24T09:21:37.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.290+0000 7ff019c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff0141143a0 con 0x7ff01410a0f0 2024-04-24T09:21:37.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.290+0000 7ff013400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff014109730 0x7ff01419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:37.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.290+0000 7ff013e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff014108540 0x7ff01419e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:37.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.290+0000 7ff013e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff014108540 0x7ff01419e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.90:55788/0 (socket says 172.21.15.90:55788) 2024-04-24T09:21:37.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.290+0000 7ff013e00640 1 -- 172.21.15.90:0/1698188626 learned_addr learned my addr 172.21.15.90:0/1698188626 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:37.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.290+0000 7ff019200640 1 --2- 172.21.15.90:0/1698188626 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff01410a0f0 0x7ff0141a5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:37.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.290+0000 7ff013e00640 1 -- 172.21.15.90:0/1698188626 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff01410a0f0 msgr2=0x7ff0141a5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:37.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.290+0000 7ff013e00640 1 --2- 172.21.15.90:0/1698188626 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff01410a0f0 0x7ff0141a5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:37.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.290+0000 7ff013e00640 1 -- 172.21.15.90:0/1698188626 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff014109730 msgr2=0x7ff01419ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:37.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.290+0000 7ff013e00640 1 --2- 172.21.15.90:0/1698188626 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff014109730 0x7ff01419ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:37.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.290+0000 7ff013e00640 1 -- 172.21.15.90:0/1698188626 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0141a6460 con 0x7ff014108540 2024-04-24T09:21:37.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.290+0000 7ff019200640 1 --2- 172.21.15.90:0/1698188626 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff01410a0f0 0x7ff0141a5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:21:37.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.291+0000 7ff013e00640 1 --2- 172.21.15.90:0/1698188626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff014108540 0x7ff01419e840 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff00400e9c0 tx=0x7ff00400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:37.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.291+0000 7ff010c00640 1 -- 172.21.15.90:0/1698188626 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00400cd80 con 0x7ff014108540 2024-04-24T09:21:37.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.291+0000 7ff010c00640 1 -- 172.21.15.90:0/1698188626 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0040035f0 con 0x7ff014108540 2024-04-24T09:21:37.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.291+0000 7ff019c00640 1 -- 172.21.15.90:0/1698188626 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0141a6750 con 0x7ff014108540 2024-04-24T09:21:37.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.291+0000 7ff010c00640 1 -- 172.21.15.90:0/1698188626 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff004010690 con 0x7ff014108540 2024-04-24T09:21:37.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.291+0000 7ff019c00640 1 -- 172.21.15.90:0/1698188626 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0141a6c70 con 0x7ff014108540 2024-04-24T09:21:37.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.292+0000 7ff019c00640 1 -- 172.21.15.90:0/1698188626 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff01410f800 con 0x7ff014108540 2024-04-24T09:21:37.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.295+0000 7ff010c00640 1 -- 172.21.15.90:0/1698188626 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff0040037a0 con 0x7ff014108540 2024-04-24T09:21:37.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.296+0000 7ff010c00640 1 --2- 172.21.15.90:0/1698188626 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7feff80a3bb0 0x7feff80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:37.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.296+0000 7ff010c00640 1 -- 172.21.15.90:0/1698188626 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7ff0040d0a50 con 0x7ff014108540 2024-04-24T09:21:37.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.296+0000 7ff013400640 1 --2- 172.21.15.90:0/1698188626 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7feff80a3bb0 0x7feff80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:37.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.297+0000 7ff013400640 1 --2- 172.21.15.90:0/1698188626 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7feff80a3bb0 0x7feff80a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff000005ab0 tx=0x7ff0000315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:37.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.298+0000 7ff010c00640 1 -- 172.21.15.90:0/1698188626 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff00409d3f0 con 0x7ff014108540 2024-04-24T09:21:37.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.504+0000 7ff019c00640 1 -- 172.21.15.90:0/1698188626 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff014062c00 con 0x7ff014108540 2024-04-24T09:21:37.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.505+0000 7ff010c00640 1 -- 172.21.15.90:0/1698188626 <== mon.2 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7ff00409ccf0 con 0x7ff014108540 2024-04-24T09:21:37.505 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 64 2024-04-24T09:21:37.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.508+0000 7ff019c00640 1 -- 172.21.15.90:0/1698188626 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7feff80a3bb0 msgr2=0x7feff80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:37.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.508+0000 7ff019c00640 1 --2- 172.21.15.90:0/1698188626 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7feff80a3bb0 0x7feff80a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff000005ab0 tx=0x7ff0000315b0 comp rx=0 tx=0).stop 2024-04-24T09:21:37.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.508+0000 7ff019c00640 1 -- 172.21.15.90:0/1698188626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff014108540 msgr2=0x7ff01419e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:37.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.508+0000 7ff019c00640 1 --2- 172.21.15.90:0/1698188626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff014108540 0x7ff01419e840 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff00400e9c0 tx=0x7ff00400ee80 comp rx=0 tx=0).stop 2024-04-24T09:21:37.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.508+0000 7ff019c00640 1 -- 172.21.15.90:0/1698188626 shutdown_connections 2024-04-24T09:21:37.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.508+0000 7ff019c00640 1 --2- 172.21.15.90:0/1698188626 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7feff80a3bb0 0x7feff80a6070 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:37.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.508+0000 7ff019c00640 1 --2- 172.21.15.90:0/1698188626 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff01410a0f0 0x7ff0141a5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:37.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.508+0000 7ff019c00640 1 --2- 172.21.15.90:0/1698188626 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff014109730 0x7ff01419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:37.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.508+0000 7ff019c00640 1 --2- 172.21.15.90:0/1698188626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff014108540 0x7ff01419e840 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:37.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.508+0000 7ff019c00640 1 -- 172.21.15.90:0/1698188626 >> 172.21.15.90:0/1698188626 conn(0x7ff0140774d0 msgr2=0x7ff01410c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:37.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.509+0000 7ff019c00640 1 -- 172.21.15.90:0/1698188626 shutdown_connections 2024-04-24T09:21:37.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.509+0000 7ff019c00640 1 -- 172.21.15.90:0/1698188626 wait complete. 2024-04-24T09:21:37.527 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:21:37.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.730+0000 7f66d8400640 1 -- 172.21.15.90:0/36005326 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66d0105470 msgr2=0x7f66d01058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:37.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.730+0000 7f66d8400640 1 --2- 172.21.15.90:0/36005326 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66d0105470 0x7f66d01058f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f66bc009900 tx=0x7f66bc02a3f0 comp rx=0 tx=0).stop 2024-04-24T09:21:37.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.730+0000 7f66d8400640 1 -- 172.21.15.90:0/36005326 shutdown_connections 2024-04-24T09:21:37.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.730+0000 7f66d8400640 1 --2- 172.21.15.90:0/36005326 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66d0105e30 0x7f66d010c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:37.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.730+0000 7f66d8400640 1 --2- 172.21.15.90:0/36005326 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66d0105470 0x7f66d01058f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:37.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.730+0000 7f66d8400640 1 --2- 172.21.15.90:0/36005326 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66d0104280 0x7f66d01046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:37.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.730+0000 7f66d8400640 1 -- 172.21.15.90:0/36005326 >> 172.21.15.90:0/36005326 conn(0x7f66d00ff7f0 msgr2=0x7f66d0101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:37.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.731+0000 7f66d8400640 1 -- 172.21.15.90:0/36005326 shutdown_connections 2024-04-24T09:21:37.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.731+0000 7f66d8400640 1 -- 172.21.15.90:0/36005326 wait complete. 2024-04-24T09:21:37.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.731+0000 7f66d8400640 1 Processor -- start 2024-04-24T09:21:37.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.732+0000 7f66d8400640 1 -- start start 2024-04-24T09:21:37.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.732+0000 7f66d8400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66d0104280 0x7f66d019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:37.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.732+0000 7f66d8400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66d0105470 0x7f66d019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:37.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.732+0000 7f66d8400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66d0105e30 0x7f66d01a1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:37.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.732+0000 7f66d8400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f66d010ff10 con 0x7f66d0105470 2024-04-24T09:21:37.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.732+0000 7f66d8400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f66d010fd90 con 0x7f66d0105e30 2024-04-24T09:21:37.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.732+0000 7f66d8400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f66d0110090 con 0x7f66d0104280 2024-04-24T09:21:37.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.732+0000 7f66d5000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66d0105470 0x7f66d019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:37.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.732+0000 7f66d5000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66d0105470 0x7f66d019aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42156/0 (socket says 172.21.15.90:42156) 2024-04-24T09:21:37.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.732+0000 7f66d5000640 1 -- 172.21.15.90:0/2146646299 learned_addr learned my addr 172.21.15.90:0/2146646299 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:37.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.732+0000 7f66d5a00640 1 --2- 172.21.15.90:0/2146646299 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66d0104280 0x7f66d019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:37.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.732+0000 7f66d6400640 1 --2- 172.21.15.90:0/2146646299 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66d0105e30 0x7f66d01a1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:37.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.733+0000 7f66d5000640 1 -- 172.21.15.90:0/2146646299 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66d0104280 msgr2=0x7f66d019a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:37.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.733+0000 7f66d5000640 1 --2- 172.21.15.90:0/2146646299 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66d0104280 0x7f66d019a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:37.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.733+0000 7f66d5000640 1 -- 172.21.15.90:0/2146646299 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66d0105e30 msgr2=0x7f66d01a1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:37.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.733+0000 7f66d5000640 1 --2- 172.21.15.90:0/2146646299 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66d0105e30 0x7f66d01a1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:37.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.733+0000 7f66d5000640 1 -- 172.21.15.90:0/2146646299 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66d01a21b0 con 0x7f66d0105470 2024-04-24T09:21:37.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.733+0000 7f66d5000640 1 --2- 172.21.15.90:0/2146646299 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66d0105470 0x7f66d019aad0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f66bc009230 tx=0x7f66bc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:37.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.733+0000 7f66ce000640 1 -- 172.21.15.90:0/2146646299 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66bc03f030 con 0x7f66d0105470 2024-04-24T09:21:37.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.733+0000 7f66ce000640 1 -- 172.21.15.90:0/2146646299 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66bc003ce0 con 0x7f66d0105470 2024-04-24T09:21:37.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.733+0000 7f66d8400640 1 -- 172.21.15.90:0/2146646299 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66d01a2440 con 0x7f66d0105470 2024-04-24T09:21:37.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.733+0000 7f66ce000640 1 -- 172.21.15.90:0/2146646299 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66bc035070 con 0x7f66d0105470 2024-04-24T09:21:37.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.733+0000 7f66d8400640 1 -- 172.21.15.90:0/2146646299 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66d01a2930 con 0x7f66d0105470 2024-04-24T09:21:37.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.734+0000 7f66d8400640 1 -- 172.21.15.90:0/2146646299 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66d010b5a0 con 0x7f66d0105470 2024-04-24T09:21:37.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.736+0000 7f66ce000640 1 -- 172.21.15.90:0/2146646299 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f66bc042030 con 0x7f66d0105470 2024-04-24T09:21:37.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.738+0000 7f66ce000640 1 --2- 172.21.15.90:0/2146646299 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f66b80a3bb0 0x7f66b80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:37.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.738+0000 7f66ce000640 1 -- 172.21.15.90:0/2146646299 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f66bc0f0920 con 0x7f66d0105470 2024-04-24T09:21:37.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.738+0000 7f66d5a00640 1 --2- 172.21.15.90:0/2146646299 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f66b80a3bb0 0x7f66b80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:37.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.739+0000 7f66d5a00640 1 --2- 172.21.15.90:0/2146646299 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f66b80a3bb0 0x7f66b80a6070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f66d01052d0 tx=0x7f66c4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:37.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.742+0000 7f66ce000640 1 -- 172.21.15.90:0/2146646299 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f66bc0bd2c0 con 0x7f66d0105470 2024-04-24T09:21:37.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:37 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/2920449400' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:21:37.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:37 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/1698188626' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:21:37.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:37 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/2920449400' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:21:37.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:37 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/1698188626' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:21:37.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.988+0000 7f66d8400640 1 -- 172.21.15.90:0/2146646299 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66d0062c00 con 0x7f66d0105470 2024-04-24T09:21:37.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.989+0000 7f66ce000640 1 -- 172.21.15.90:0/2146646299 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11928 (secure 0 0 0) 0x7f66bc0bcbc0 con 0x7f66d0105470 2024-04-24T09:21:37.990 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:21:37.990 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/227251251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":227251251},{"type":"v1","addr":"172.21.15.90:6837","nonce":227251251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/1904617971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":1904617971},{"type":"v1","addr":"172.21.15.90:6839","nonce":1904617971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T09:19:09.430875+0000","modified":"2024-04-24T09:19:17.259940+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24547,"mds_2":24545,"mds_3":24526,"mds_4":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6833/219377141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":219377141},{"type":"v1","addr":"172.21.15.114:6833","nonce":219377141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"j","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.90:6835/2396995634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2396995634},{"type":"v1","addr":"172.21.15.90:6835","nonce":2396995634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/779642417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":779642417},{"type":"v1","addr":"172.21.15.149:6833","nonce":779642417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6835/2565340800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":2565340800},{"type":"v1","addr":"172.21.15.114:6835","nonce":2565340800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6837/875324280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":875324280},{"type":"v1","addr":"172.21.15.114:6837","nonce":875324280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/3157375624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3157375624},{"type":"v1","addr":"172.21.15.149:6835","nonce":3157375624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.149:6837/2250310587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2250310587},{"type":"v1","addr":"172.21.15.149:6837","nonce":2250310587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.90:6841/397593398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":397593398},{"type":"v1","addr":"172.21.15.90:6841","nonce":397593398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.149:6839/652734263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":652734263},{"type":"v1","addr":"172.21.15.149:6839","nonce":652734263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6839/1025403710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":1025403710},{"type":"v1","addr":"172.21.15.114:6839","nonce":1025403710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24545,24557,24446,24526,24547]},"id":1}]} 2024-04-24T09:21:37.990 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 27 2024-04-24T09:21:37.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.992+0000 7f66d8400640 1 -- 172.21.15.90:0/2146646299 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f66b80a3bb0 msgr2=0x7f66b80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:37.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.992+0000 7f66d8400640 1 --2- 172.21.15.90:0/2146646299 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f66b80a3bb0 0x7f66b80a6070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f66d01052d0 tx=0x7f66c4005f90 comp rx=0 tx=0).stop 2024-04-24T09:21:37.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.992+0000 7f66d8400640 1 -- 172.21.15.90:0/2146646299 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66d0105470 msgr2=0x7f66d019aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:37.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.992+0000 7f66d8400640 1 --2- 172.21.15.90:0/2146646299 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66d0105470 0x7f66d019aad0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f66bc009230 tx=0x7f66bc0042f0 comp rx=0 tx=0).stop 2024-04-24T09:21:37.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.992+0000 7f66d8400640 1 -- 172.21.15.90:0/2146646299 shutdown_connections 2024-04-24T09:21:37.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.992+0000 7f66d8400640 1 --2- 172.21.15.90:0/2146646299 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f66b80a3bb0 0x7f66b80a6070 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:37.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.992+0000 7f66d8400640 1 --2- 172.21.15.90:0/2146646299 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66d0105e30 0x7f66d01a1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:37.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.992+0000 7f66d8400640 1 --2- 172.21.15.90:0/2146646299 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66d0105470 0x7f66d019aad0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:37.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.992+0000 7f66d8400640 1 --2- 172.21.15.90:0/2146646299 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66d0104280 0x7f66d019a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:37.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.992+0000 7f66d8400640 1 -- 172.21.15.90:0/2146646299 >> 172.21.15.90:0/2146646299 conn(0x7f66d00ff7f0 msgr2=0x7f66d0108770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:37.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.992+0000 7f66d8400640 1 -- 172.21.15.90:0/2146646299 shutdown_connections 2024-04-24T09:21:37.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:37.992+0000 7f66d8400640 1 -- 172.21.15.90:0/2146646299 wait complete. 2024-04-24T09:21:38.010 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T09:21:38.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:37 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/2920449400' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:21:38.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:37 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/1698188626' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:21:38.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.208+0000 7feb7fe00640 1 -- 172.21.15.90:0/1886687620 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb780769c0 msgr2=0x7feb78076de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:38.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.208+0000 7feb7fe00640 1 --2- 172.21.15.90:0/1886687620 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb780769c0 0x7feb78076de0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7feb6c009900 tx=0x7feb6c02a410 comp rx=0 tx=0).stop 2024-04-24T09:21:38.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.208+0000 7feb7fe00640 1 -- 172.21.15.90:0/1886687620 shutdown_connections 2024-04-24T09:21:38.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.208+0000 7feb7fe00640 1 --2- 172.21.15.90:0/1886687620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb780785a0 0x7feb78078e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:38.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.208+0000 7feb7fe00640 1 --2- 172.21.15.90:0/1886687620 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb78077c00 0x7feb78078060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:38.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.208+0000 7feb7fe00640 1 --2- 172.21.15.90:0/1886687620 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb780769c0 0x7feb78076de0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:38.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.208+0000 7feb7fe00640 1 -- 172.21.15.90:0/1886687620 >> 172.21.15.90:0/1886687620 conn(0x7feb780ff7c0 msgr2=0x7feb78101c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:38.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.208+0000 7feb7fe00640 1 -- 172.21.15.90:0/1886687620 shutdown_connections 2024-04-24T09:21:38.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.208+0000 7feb7fe00640 1 -- 172.21.15.90:0/1886687620 wait complete. 2024-04-24T09:21:38.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.209+0000 7feb7fe00640 1 Processor -- start 2024-04-24T09:21:38.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.209+0000 7feb7fe00640 1 -- start start 2024-04-24T09:21:38.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.209+0000 7feb7fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb780769c0 0x7feb7819e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:38.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.209+0000 7feb7fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb78077c00 0x7feb7819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:38.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.210+0000 7feb7fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb780785a0 0x7feb781a5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:38.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.210+0000 7feb7fe00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7feb78114320 con 0x7feb780785a0 2024-04-24T09:21:38.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.210+0000 7feb7fe00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7feb781141a0 con 0x7feb78077c00 2024-04-24T09:21:38.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.210+0000 7feb7fe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7feb781144a0 con 0x7feb780769c0 2024-04-24T09:21:38.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.210+0000 7feb7ca00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb78077c00 0x7feb7819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:38.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.210+0000 7feb7de00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb780785a0 0x7feb781a5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:38.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.210+0000 7feb7ca00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb78077c00 0x7feb7819eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.90:55808/0 (socket says 172.21.15.90:55808) 2024-04-24T09:21:38.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.210+0000 7feb7ca00640 1 -- 172.21.15.90:0/3082556434 learned_addr learned my addr 172.21.15.90:0/3082556434 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:38.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.210+0000 7feb7d400640 1 --2- 172.21.15.90:0/3082556434 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb780769c0 0x7feb7819e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:38.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.210+0000 7feb7de00640 1 -- 172.21.15.90:0/3082556434 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb780769c0 msgr2=0x7feb7819e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:38.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.210+0000 7feb7de00640 1 --2- 172.21.15.90:0/3082556434 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb780769c0 0x7feb7819e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:38.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.210+0000 7feb7de00640 1 -- 172.21.15.90:0/3082556434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb78077c00 msgr2=0x7feb7819eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:38.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.210+0000 7feb7de00640 1 --2- 172.21.15.90:0/3082556434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb78077c00 0x7feb7819eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:38.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.210+0000 7feb7de00640 1 -- 172.21.15.90:0/3082556434 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb781a65c0 con 0x7feb780785a0 2024-04-24T09:21:38.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.210+0000 7feb7ca00640 1 --2- 172.21.15.90:0/3082556434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb78077c00 0x7feb7819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:21:38.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.211+0000 7feb7de00640 1 --2- 172.21.15.90:0/3082556434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb780785a0 0x7feb781a5f50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7feb70009fd0 tx=0x7feb7000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:38.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.211+0000 7feb76000640 1 -- 172.21.15.90:0/3082556434 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7000ce20 con 0x7feb780785a0 2024-04-24T09:21:38.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.211+0000 7feb7fe00640 1 -- 172.21.15.90:0/3082556434 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb781a68b0 con 0x7feb780785a0 2024-04-24T09:21:38.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.211+0000 7feb76000640 1 -- 172.21.15.90:0/3082556434 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb70005020 con 0x7feb780785a0 2024-04-24T09:21:38.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.211+0000 7feb76000640 1 -- 172.21.15.90:0/3082556434 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb70010720 con 0x7feb780785a0 2024-04-24T09:21:38.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.211+0000 7feb7fe00640 1 -- 172.21.15.90:0/3082556434 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb781a6db0 con 0x7feb780785a0 2024-04-24T09:21:38.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.212+0000 7feb7fe00640 1 -- 172.21.15.90:0/3082556434 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb78110bc0 con 0x7feb780785a0 2024-04-24T09:21:38.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.214+0000 7feb76000640 1 -- 172.21.15.90:0/3082556434 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7feb70005540 con 0x7feb780785a0 2024-04-24T09:21:38.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.215+0000 7feb76000640 1 --2- 172.21.15.90:0/3082556434 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7feb600a3bb0 0x7feb600a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:38.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.216+0000 7feb76000640 1 -- 172.21.15.90:0/3082556434 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7feb700d0aa0 con 0x7feb780785a0 2024-04-24T09:21:38.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.216+0000 7feb7d400640 1 --2- 172.21.15.90:0/3082556434 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7feb600a3bb0 0x7feb600a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:38.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.216+0000 7feb7d400640 1 --2- 172.21.15.90:0/3082556434 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7feb600a3bb0 0x7feb600a6070 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7feb6c031440 tx=0x7feb6c0315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:38.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.220+0000 7feb76000640 1 -- 172.21.15.90:0/3082556434 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feb7009d440 con 0x7feb780785a0 2024-04-24T09:21:38.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.426+0000 7feb7fe00640 1 -- 172.21.15.90:0/3082556434 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feb78062c00 con 0x7feb780785a0 2024-04-24T09:21:38.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.430+0000 7feb76000640 1 -- 172.21.15.90:0/3082556434 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22652 (secure 0 0 0) 0x7feb7009cd40 con 0x7feb780785a0 2024-04-24T09:21:38.431 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:21:38.431 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":88,"fsid":"30ee9be2-021a-11ef-bc93-c7b262605968","created":"2024-04-24T09:09:49.496019+0000","modified":"2024-04-24T09:19:58.609640+0000","last_up_change":"2024-04-24T09:18:10.455632+0000","last_in_change":"2024-04-24T09:17:50.600366+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T09:13:45.548774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T09:19:07.185580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T09:19:08.374818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13a59b9f-b46e-4ec4-90af-bb7a3d5c753b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6803","nonce":2864288066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6805","nonce":2864288066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6809","nonce":2864288066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6807","nonce":2864288066}]},"public_addr":"172.21.15.90:6803/2864288066","cluster_addr":"172.21.15.90:6805/2864288066","heartbeat_back_addr":"172.21.15.90:6809/2864288066","heartbeat_front_addr":"172.21.15.90:6807/2864288066","state":["exists","up"]},{"osd":1,"uuid":"1567ce8f-24e6-42cf-bf64-f6f0c534a9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6801","nonce":344222414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6803","nonce":344222414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6807","nonce":344222414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6805","nonce":344222414}]},"public_addr":"172.21.15.114:6801/344222414","cluster_addr":"172.21.15.114:6803/344222414","heartbeat_back_addr":"172.21.15.114:6807/344222414","heartbeat_front_addr":"172.21.15.114:6805/344222414","state":["exists","up"]},{"osd":2,"uuid":"f861b304-9c7d-4629-8522-37f33a747945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6801","nonce":3257067419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6803","nonce":3257067419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6807","nonce":3257067419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6805","nonce":3257067419}]},"public_addr":"172.21.15.149:6801/3257067419","cluster_addr":"172.21.15.149:6803/3257067419","heartbeat_back_addr":"172.21.15.149:6807/3257067419","heartbeat_front_addr":"172.21.15.149:6805/3257067419","state":["exists","up"]},{"osd":3,"uuid":"9377ab6e-5996-45fb-93a9-475a2e5f17a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6811","nonce":3068524970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6813","nonce":3068524970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6817","nonce":3068524970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6815","nonce":3068524970}]},"public_addr":"172.21.15.90:6811/3068524970","cluster_addr":"172.21.15.90:6813/3068524970","heartbeat_back_addr":"172.21.15.90:6817/3068524970","heartbeat_front_addr":"172.21.15.90:6815/3068524970","state":["exists","up"]},{"osd":4,"uuid":"e2ddd19e-e641-4d61-b342-02b5f12b1c4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6809","nonce":2782015522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6811","nonce":2782015522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6815","nonce":2782015522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6813","nonce":2782015522}]},"public_addr":"172.21.15.114:6809/2782015522","cluster_addr":"172.21.15.114:6811/2782015522","heartbeat_back_addr":"172.21.15.114:6815/2782015522","heartbeat_front_addr":"172.21.15.114:6813/2782015522","state":["exists","up"]},{"osd":5,"uuid":"b6ff9d10-5210-4b46-a192-6dcbf9d1f156","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6809","nonce":4266460479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6811","nonce":4266460479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6815","nonce":4266460479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6813","nonce":4266460479}]},"public_addr":"172.21.15.149:6809/4266460479","cluster_addr":"172.21.15.149:6811/4266460479","heartbeat_back_addr":"172.21.15.149:6815/4266460479","heartbeat_front_addr":"172.21.15.149:6813/4266460479","state":["exists","up"]},{"osd":6,"uuid":"4287b5a1-f953-421c-bf62-5e390bee834d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6819","nonce":1121461276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6821","nonce":1121461276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6825","nonce":1121461276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6823","nonce":1121461276}]},"public_addr":"172.21.15.90:6819/1121461276","cluster_addr":"172.21.15.90:6821/1121461276","heartbeat_back_addr":"172.21.15.90:6825/1121461276","heartbeat_front_addr":"172.21.15.90:6823/1121461276","state":["exists","up"]},{"osd":7,"uuid":"a54e507d-d6ba-4bf2-9a7f-a7b95c3859bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6817","nonce":3222546280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6819","nonce":3222546280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6823","nonce":3222546280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6821","nonce":3222546280}]},"public_addr":"172.21.15.114:6817/3222546280","cluster_addr":"172.21.15.114:6819/3222546280","heartbeat_back_addr":"172.21.15.114:6823/3222546280","heartbeat_front_addr":"172.21.15.114:6821/3222546280","state":["exists","up"]},{"osd":8,"uuid":"9950b010-5865-4109-ae37-582b1dbd2f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6817","nonce":157615149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6819","nonce":157615149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6823","nonce":157615149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6821","nonce":157615149}]},"public_addr":"172.21.15.149:6817/157615149","cluster_addr":"172.21.15.149:6819/157615149","heartbeat_back_addr":"172.21.15.149:6823/157615149","heartbeat_front_addr":"172.21.15.149:6821/157615149","state":["exists","up"]},{"osd":9,"uuid":"31e2635c-e234-421f-9757-592fd673c936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6827","nonce":1307318076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6829","nonce":1307318076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6833","nonce":1307318076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6831","nonce":1307318076}]},"public_addr":"172.21.15.90:6827/1307318076","cluster_addr":"172.21.15.90:6829/1307318076","heartbeat_back_addr":"172.21.15.90:6833/1307318076","heartbeat_front_addr":"172.21.15.90:6831/1307318076","state":["exists","up"]},{"osd":10,"uuid":"46a3b2a6-e6e5-4578-b4bb-f3e998b9bc95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6825","nonce":1719802140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6827","nonce":1719802140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6831","nonce":1719802140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6829","nonce":1719802140}]},"public_addr":"172.21.15.114:6825/1719802140","cluster_addr":"172.21.15.114:6827/1719802140","heartbeat_back_addr":"172.21.15.114:6831/1719802140","heartbeat_front_addr":"172.21.15.114:6829/1719802140","state":["exists","up"]},{"osd":11,"uuid":"ccd4456b-e947-46cf-a779-148ffa289b58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6825","nonce":1737275166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6827","nonce":1737275166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6831","nonce":1737275166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6829","nonce":1737275166}]},"public_addr":"172.21.15.149:6825/1737275166","cluster_addr":"172.21.15.149:6827/1737275166","heartbeat_back_addr":"172.21.15.149:6831/1737275166","heartbeat_front_addr":"172.21.15.149:6829/1737275166","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:12:53.015078+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:13:18.047053+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:13:41.935891+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:14:15.095388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:14:42.512355+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:15:08.113775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:15:43.828550+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:16:10.901909+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:16:36.422962+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:17:13.924446+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:17:42.814401+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:18:08.672793+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/3090852580":"2024-04-25T09:10:57.377655+0000","172.21.15.90:0/687197394":"2024-04-25T09:10:57.377655+0000","172.21.15.90:6801/1809038230":"2024-04-25T09:10:57.377655+0000","172.21.15.90:6800/1809038230":"2024-04-25T09:10:57.377655+0000","172.21.15.90:0/2964149071":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/3617388113":"2024-04-25T09:10:22.599674+0000","172.21.15.90:6801/3089140842":"2024-04-25T09:10:22.599674+0000","172.21.15.90:6800/3089140842":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/2840416149":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/305432710":"2024-04-25T09:10:57.377655+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T09:21:38.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.433+0000 7feb7fe00640 1 -- 172.21.15.90:0/3082556434 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7feb600a3bb0 msgr2=0x7feb600a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:38.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.433+0000 7feb7fe00640 1 --2- 172.21.15.90:0/3082556434 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7feb600a3bb0 0x7feb600a6070 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7feb6c031440 tx=0x7feb6c0315b0 comp rx=0 tx=0).stop 2024-04-24T09:21:38.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.433+0000 7feb7fe00640 1 -- 172.21.15.90:0/3082556434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb780785a0 msgr2=0x7feb781a5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:38.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.433+0000 7feb7fe00640 1 --2- 172.21.15.90:0/3082556434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb780785a0 0x7feb781a5f50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7feb70009fd0 tx=0x7feb7000ef10 comp rx=0 tx=0).stop 2024-04-24T09:21:38.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.433+0000 7feb7fe00640 1 -- 172.21.15.90:0/3082556434 shutdown_connections 2024-04-24T09:21:38.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.433+0000 7feb7fe00640 1 --2- 172.21.15.90:0/3082556434 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7feb600a3bb0 0x7feb600a6070 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:38.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.433+0000 7feb7fe00640 1 --2- 172.21.15.90:0/3082556434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb780785a0 0x7feb781a5f50 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:38.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.433+0000 7feb7fe00640 1 --2- 172.21.15.90:0/3082556434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb78077c00 0x7feb7819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:38.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.433+0000 7feb7fe00640 1 --2- 172.21.15.90:0/3082556434 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb780769c0 0x7feb7819e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:38.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.433+0000 7feb7fe00640 1 -- 172.21.15.90:0/3082556434 >> 172.21.15.90:0/3082556434 conn(0x7feb780ff7c0 msgr2=0x7feb781012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:38.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.433+0000 7feb7fe00640 1 -- 172.21.15.90:0/3082556434 shutdown_connections 2024-04-24T09:21:38.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.433+0000 7feb7fe00640 1 -- 172.21.15.90:0/3082556434 wait complete. 2024-04-24T09:21:38.453 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T09:13:45.548774+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T09:19:07.185580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T09:19:08.374818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T09:21:38.454 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T09:21:38.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.651+0000 7feff2600640 1 -- 172.21.15.90:0/1425647848 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fefec077c00 msgr2=0x7fefec078060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:38.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.651+0000 7feff2600640 1 --2- 172.21.15.90:0/1425647848 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fefec077c00 0x7fefec078060 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fefd8009860 tx=0x7fefd802a340 comp rx=0 tx=0).stop 2024-04-24T09:21:38.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.651+0000 7feff2600640 1 -- 172.21.15.90:0/1425647848 shutdown_connections 2024-04-24T09:21:38.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.651+0000 7feff2600640 1 --2- 172.21.15.90:0/1425647848 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fefec0785a0 0x7fefec078e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:38.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.651+0000 7feff2600640 1 --2- 172.21.15.90:0/1425647848 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fefec077c00 0x7fefec078060 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:38.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.651+0000 7feff2600640 1 --2- 172.21.15.90:0/1425647848 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fefec0769c0 0x7fefec076de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:38.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.651+0000 7feff2600640 1 -- 172.21.15.90:0/1425647848 >> 172.21.15.90:0/1425647848 conn(0x7fefec0ff7b0 msgr2=0x7fefec101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:38.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.651+0000 7feff2600640 1 -- 172.21.15.90:0/1425647848 shutdown_connections 2024-04-24T09:21:38.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.651+0000 7feff2600640 1 -- 172.21.15.90:0/1425647848 wait complete. 2024-04-24T09:21:38.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.652+0000 7feff2600640 1 Processor -- start 2024-04-24T09:21:38.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.652+0000 7feff2600640 1 -- start start 2024-04-24T09:21:38.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.653+0000 7feff2600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fefec0769c0 0x7fefec19e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:38.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.653+0000 7feff2600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fefec077c00 0x7fefec19eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:38.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.653+0000 7feff2600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fefec0785a0 0x7fefec1a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:38.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.653+0000 7feff2600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fefec1142f0 con 0x7fefec0769c0 2024-04-24T09:21:38.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.653+0000 7feff2600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fefec114170 con 0x7fefec0785a0 2024-04-24T09:21:38.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.653+0000 7feff2600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fefec114470 con 0x7fefec077c00 2024-04-24T09:21:38.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.653+0000 7fefeb400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fefec0769c0 0x7fefec19e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:38.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.653+0000 7fefeaa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fefec077c00 0x7fefec19eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:38.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.653+0000 7fefeb400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fefec0769c0 0x7fefec19e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42192/0 (socket says 172.21.15.90:42192) 2024-04-24T09:21:38.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.653+0000 7fefeb400640 1 -- 172.21.15.90:0/505643283 learned_addr learned my addr 172.21.15.90:0/505643283 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:38.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.653+0000 7fefebe00640 1 --2- 172.21.15.90:0/505643283 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fefec0785a0 0x7fefec1a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:38.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.654+0000 7fefeaa00640 1 -- 172.21.15.90:0/505643283 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fefec0785a0 msgr2=0x7fefec1a5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:38.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.654+0000 7fefeaa00640 1 --2- 172.21.15.90:0/505643283 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fefec0785a0 0x7fefec1a5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:38.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.654+0000 7fefeaa00640 1 -- 172.21.15.90:0/505643283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fefec0769c0 msgr2=0x7fefec19e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:38.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.654+0000 7fefeaa00640 1 --2- 172.21.15.90:0/505643283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fefec0769c0 0x7fefec19e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:38.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.654+0000 7fefeaa00640 1 -- 172.21.15.90:0/505643283 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefec1a6590 con 0x7fefec077c00 2024-04-24T09:21:38.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.654+0000 7fefebe00640 1 --2- 172.21.15.90:0/505643283 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fefec0785a0 0x7fefec1a5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:21:38.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.654+0000 7fefeb400640 1 --2- 172.21.15.90:0/505643283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fefec0769c0 0x7fefec19e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:21:38.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.654+0000 7fefeaa00640 1 --2- 172.21.15.90:0/505643283 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fefec077c00 0x7fefec19eeb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fefd8035040 tx=0x7fefd8033d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:38.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.654+0000 7fefe3e00640 1 -- 172.21.15.90:0/505643283 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd8037040 con 0x7fefec077c00 2024-04-24T09:21:38.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.654+0000 7fefe3e00640 1 -- 172.21.15.90:0/505643283 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefd8030850 con 0x7fefec077c00 2024-04-24T09:21:38.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.654+0000 7fefe3e00640 1 -- 172.21.15.90:0/505643283 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd8004920 con 0x7fefec077c00 2024-04-24T09:21:38.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.654+0000 7feff2600640 1 -- 172.21.15.90:0/505643283 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefec1a6820 con 0x7fefec077c00 2024-04-24T09:21:38.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.655+0000 7feff2600640 1 -- 172.21.15.90:0/505643283 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefec1a6c90 con 0x7fefec077c00 2024-04-24T09:21:38.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.656+0000 7feff2600640 1 -- 172.21.15.90:0/505643283 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefec110db0 con 0x7fefec077c00 2024-04-24T09:21:38.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.658+0000 7fefe3e00640 1 -- 172.21.15.90:0/505643283 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fefd8004050 con 0x7fefec077c00 2024-04-24T09:21:38.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.659+0000 7fefe3e00640 1 --2- 172.21.15.90:0/505643283 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fefd00a3bb0 0x7fefd00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:38.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.659+0000 7fefe3e00640 1 -- 172.21.15.90:0/505643283 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fefd80f0210 con 0x7fefec077c00 2024-04-24T09:21:38.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.659+0000 7fefeb400640 1 --2- 172.21.15.90:0/505643283 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fefd00a3bb0 0x7fefd00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:38.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.660+0000 7fefeb400640 1 --2- 172.21.15.90:0/505643283 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fefd00a3bb0 0x7fefd00a6070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fefdc011500 tx=0x7fefdc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:38.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.662+0000 7fefe3e00640 1 -- 172.21.15.90:0/505643283 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fefd80bcbb0 con 0x7fefec077c00 2024-04-24T09:21:38.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.852+0000 7feff2600640 1 -- 172.21.15.90:0/505643283 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fefec02d7b0 con 0x7fefec077c00 2024-04-24T09:21:38.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.853+0000 7fefe3e00640 1 -- 172.21.15.90:0/505643283 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7fefd80bc4b0 con 0x7fefec077c00 2024-04-24T09:21:38.853 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 1 2024-04-24T09:21:38.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.855+0000 7feff2600640 1 -- 172.21.15.90:0/505643283 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fefd00a3bb0 msgr2=0x7fefd00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:38.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.855+0000 7feff2600640 1 --2- 172.21.15.90:0/505643283 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fefd00a3bb0 0x7fefd00a6070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fefdc011500 tx=0x7fefdc005f90 comp rx=0 tx=0).stop 2024-04-24T09:21:38.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.855+0000 7feff2600640 1 -- 172.21.15.90:0/505643283 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fefec077c00 msgr2=0x7fefec19eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:38.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.855+0000 7feff2600640 1 --2- 172.21.15.90:0/505643283 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fefec077c00 0x7fefec19eeb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fefd8035040 tx=0x7fefd8033d90 comp rx=0 tx=0).stop 2024-04-24T09:21:38.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.856+0000 7feff2600640 1 -- 172.21.15.90:0/505643283 shutdown_connections 2024-04-24T09:21:38.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.856+0000 7feff2600640 1 --2- 172.21.15.90:0/505643283 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fefd00a3bb0 0x7fefd00a6070 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:38.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.856+0000 7feff2600640 1 --2- 172.21.15.90:0/505643283 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fefec0785a0 0x7fefec1a5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:38.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.856+0000 7feff2600640 1 --2- 172.21.15.90:0/505643283 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fefec077c00 0x7fefec19eeb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:38.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.856+0000 7feff2600640 1 --2- 172.21.15.90:0/505643283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fefec0769c0 0x7fefec19e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:38.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.856+0000 7feff2600640 1 -- 172.21.15.90:0/505643283 >> 172.21.15.90:0/505643283 conn(0x7fefec0ff7b0 msgr2=0x7fefec100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:38.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.856+0000 7feff2600640 1 -- 172.21.15.90:0/505643283 shutdown_connections 2024-04-24T09:21:38.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:38.856+0000 7feff2600640 1 -- 172.21.15.90:0/505643283 wait complete. 2024-04-24T09:21:38.874 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T09:21:38.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:38 smithi114 ceph-mon[32501]: pgmap v398: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:21:38.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:38 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/2146646299' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:21:38.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:38 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/3082556434' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:21:38.941 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:38 smithi090 ceph-mon[27825]: pgmap v398: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:21:38.941 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:38 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/2146646299' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:21:38.941 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:38 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/3082556434' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:21:39.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.075+0000 7fab11e00640 1 -- 172.21.15.90:0/537237831 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab0c108540 msgr2=0x7fab0c108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:39.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.075+0000 7fab11e00640 1 --2- 172.21.15.90:0/537237831 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab0c108540 0x7fab0c108960 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7faafc009900 tx=0x7faafc02a410 comp rx=0 tx=0).stop 2024-04-24T09:21:39.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.075+0000 7fab11e00640 1 -- 172.21.15.90:0/537237831 shutdown_connections 2024-04-24T09:21:39.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.075+0000 7fab11e00640 1 --2- 172.21.15.90:0/537237831 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fab0c10a0f0 0x7fab0c110980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:39.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.075+0000 7fab11e00640 1 --2- 172.21.15.90:0/537237831 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab0c109730 0x7fab0c109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:39.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.075+0000 7fab11e00640 1 --2- 172.21.15.90:0/537237831 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab0c108540 0x7fab0c108960 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:39.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.075+0000 7fab11e00640 1 -- 172.21.15.90:0/537237831 >> 172.21.15.90:0/537237831 conn(0x7fab0c0774d0 msgr2=0x7fab0c079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:39.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.075+0000 7fab11e00640 1 -- 172.21.15.90:0/537237831 shutdown_connections 2024-04-24T09:21:39.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.075+0000 7fab11e00640 1 -- 172.21.15.90:0/537237831 wait complete. 2024-04-24T09:21:39.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.076+0000 7fab11e00640 1 Processor -- start 2024-04-24T09:21:39.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.076+0000 7fab11e00640 1 -- start start 2024-04-24T09:21:39.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.076+0000 7fab11e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fab0c108540 0x7fab0c19e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:39.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.076+0000 7fab11e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab0c109730 0x7fab0c19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:39.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.077+0000 7fab11e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab0c10a0f0 0x7fab0c1a5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:39.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.077+0000 7fab11e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c1141d0 con 0x7fab0c109730 2024-04-24T09:21:39.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.077+0000 7fab11e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c114050 con 0x7fab0c108540 2024-04-24T09:21:39.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.077+0000 7fab11e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c114350 con 0x7fab0c10a0f0 2024-04-24T09:21:39.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.077+0000 7fab03e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab0c109730 0x7fab0c19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:39.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.077+0000 7fab10a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fab0c108540 0x7fab0c19e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:39.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.077+0000 7fab03e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab0c109730 0x7fab0c19ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42198/0 (socket says 172.21.15.90:42198) 2024-04-24T09:21:39.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.077+0000 7fab10a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fab0c108540 0x7fab0c19e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.90:55842/0 (socket says 172.21.15.90:55842) 2024-04-24T09:21:39.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.077+0000 7fab03e00640 1 -- 172.21.15.90:0/1883139514 learned_addr learned my addr 172.21.15.90:0/1883139514 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:39.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.077+0000 7fab11400640 1 --2- 172.21.15.90:0/1883139514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab0c10a0f0 0x7fab0c1a5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:39.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.077+0000 7fab03e00640 1 -- 172.21.15.90:0/1883139514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab0c10a0f0 msgr2=0x7fab0c1a5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:39.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.077+0000 7fab03e00640 1 --2- 172.21.15.90:0/1883139514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab0c10a0f0 0x7fab0c1a5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:39.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.077+0000 7fab03e00640 1 -- 172.21.15.90:0/1883139514 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fab0c108540 msgr2=0x7fab0c19e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:39.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.077+0000 7fab03e00640 1 --2- 172.21.15.90:0/1883139514 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fab0c108540 0x7fab0c19e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:39.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.077+0000 7fab03e00640 1 -- 172.21.15.90:0/1883139514 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab0c1a6440 con 0x7fab0c109730 2024-04-24T09:21:39.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.077+0000 7fab10a00640 1 --2- 172.21.15.90:0/1883139514 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fab0c108540 0x7fab0c19e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:21:39.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.077+0000 7fab11400640 1 --2- 172.21.15.90:0/1883139514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab0c10a0f0 0x7fab0c1a5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:21:39.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.078+0000 7fab03e00640 1 --2- 172.21.15.90:0/1883139514 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab0c109730 0x7fab0c19ed60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7faaf400e9f0 tx=0x7faaf400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:39.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.078+0000 7fab0a000640 1 -- 172.21.15.90:0/1883139514 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf400cd80 con 0x7fab0c109730 2024-04-24T09:21:39.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.078+0000 7fab11e00640 1 -- 172.21.15.90:0/1883139514 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab0c1a6730 con 0x7fab0c109730 2024-04-24T09:21:39.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.078+0000 7fab0a000640 1 -- 172.21.15.90:0/1883139514 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaf4005500 con 0x7fab0c109730 2024-04-24T09:21:39.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.078+0000 7fab0a000640 1 -- 172.21.15.90:0/1883139514 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf4010690 con 0x7fab0c109730 2024-04-24T09:21:39.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.078+0000 7fab11e00640 1 -- 172.21.15.90:0/1883139514 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab0c1a6c30 con 0x7fab0c109730 2024-04-24T09:21:39.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.079+0000 7fab11e00640 1 -- 172.21.15.90:0/1883139514 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab0c10f860 con 0x7fab0c109730 2024-04-24T09:21:39.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.081+0000 7fab0a000640 1 -- 172.21.15.90:0/1883139514 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7faaf40046c0 con 0x7fab0c109730 2024-04-24T09:21:39.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.082+0000 7fab0a000640 1 --2- 172.21.15.90:0/1883139514 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7faaf00a3bb0 0x7faaf00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:39.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.083+0000 7fab0a000640 1 -- 172.21.15.90:0/1883139514 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7faaf40d08f0 con 0x7fab0c109730 2024-04-24T09:21:39.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.083+0000 7fab10a00640 1 --2- 172.21.15.90:0/1883139514 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7faaf00a3bb0 0x7faaf00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:39.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.084+0000 7fab10a00640 1 --2- 172.21.15.90:0/1883139514 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7faaf00a3bb0 0x7faaf00a6070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7faafc005ab0 tx=0x7faafc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:39.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.087+0000 7fab0a000640 1 -- 172.21.15.90:0/1883139514 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faaf409d290 con 0x7fab0c109730 2024-04-24T09:21:39.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:38 smithi149 ceph-mon[32592]: pgmap v398: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:21:39.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:38 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/2146646299' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:21:39.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:38 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/3082556434' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:21:39.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.287+0000 7fab11e00640 1 -- 172.21.15.90:0/1883139514 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fab0c062c00 con 0x7fab0c109730 2024-04-24T09:21:39.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.289+0000 7fab0a000640 1 -- 172.21.15.90:0/1883139514 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7faaf409cb90 con 0x7fab0c109730 2024-04-24T09:21:39.289 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 32 2024-04-24T09:21:39.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.291+0000 7fab11e00640 1 -- 172.21.15.90:0/1883139514 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7faaf00a3bb0 msgr2=0x7faaf00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:39.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.291+0000 7fab11e00640 1 --2- 172.21.15.90:0/1883139514 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7faaf00a3bb0 0x7faaf00a6070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7faafc005ab0 tx=0x7faafc000bc0 comp rx=0 tx=0).stop 2024-04-24T09:21:39.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.291+0000 7fab11e00640 1 -- 172.21.15.90:0/1883139514 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab0c109730 msgr2=0x7fab0c19ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:39.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.291+0000 7fab11e00640 1 --2- 172.21.15.90:0/1883139514 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab0c109730 0x7fab0c19ed60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7faaf400e9f0 tx=0x7faaf400edf0 comp rx=0 tx=0).stop 2024-04-24T09:21:39.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.291+0000 7fab11e00640 1 -- 172.21.15.90:0/1883139514 shutdown_connections 2024-04-24T09:21:39.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.291+0000 7fab11e00640 1 --2- 172.21.15.90:0/1883139514 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7faaf00a3bb0 0x7faaf00a6070 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:39.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.291+0000 7fab11e00640 1 --2- 172.21.15.90:0/1883139514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab0c10a0f0 0x7fab0c1a5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:39.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.291+0000 7fab11e00640 1 --2- 172.21.15.90:0/1883139514 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab0c109730 0x7fab0c19ed60 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:39.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.291+0000 7fab11e00640 1 --2- 172.21.15.90:0/1883139514 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fab0c108540 0x7fab0c19e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:39.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.291+0000 7fab11e00640 1 -- 172.21.15.90:0/1883139514 >> 172.21.15.90:0/1883139514 conn(0x7fab0c0774d0 msgr2=0x7fab0c10ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:39.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.291+0000 7fab11e00640 1 -- 172.21.15.90:0/1883139514 shutdown_connections 2024-04-24T09:21:39.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.291+0000 7fab11e00640 1 -- 172.21.15.90:0/1883139514 wait complete. 2024-04-24T09:21:39.310 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T09:21:39.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.517+0000 7feb71600640 1 -- 172.21.15.90:0/4060366663 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb6c103cb0 msgr2=0x7feb6c10a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:39.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.517+0000 7feb71600640 1 --2- 172.21.15.90:0/4060366663 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb6c103cb0 0x7feb6c10a540 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7feb64009cd0 tx=0x7feb6402a8e0 comp rx=0 tx=0).stop 2024-04-24T09:21:39.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.517+0000 7feb71600640 1 -- 172.21.15.90:0/4060366663 shutdown_connections 2024-04-24T09:21:39.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.517+0000 7feb71600640 1 --2- 172.21.15.90:0/4060366663 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb6c103cb0 0x7feb6c10a540 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:39.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.517+0000 7feb71600640 1 --2- 172.21.15.90:0/4060366663 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb6c1032f0 0x7feb6c103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:39.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.517+0000 7feb71600640 1 --2- 172.21.15.90:0/4060366663 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb6c100320 0x7feb6c100740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:39.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.517+0000 7feb71600640 1 -- 172.21.15.90:0/4060366663 >> 172.21.15.90:0/4060366663 conn(0x7feb6c0fbc30 msgr2=0x7feb6c0fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:39.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.517+0000 7feb71600640 1 -- 172.21.15.90:0/4060366663 shutdown_connections 2024-04-24T09:21:39.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.517+0000 7feb71600640 1 -- 172.21.15.90:0/4060366663 wait complete. 2024-04-24T09:21:39.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.518+0000 7feb71600640 1 Processor -- start 2024-04-24T09:21:39.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.518+0000 7feb71600640 1 -- start start 2024-04-24T09:21:39.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.519+0000 7feb71600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb6c100320 0x7feb6c198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:39.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.519+0000 7feb71600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb6c1032f0 0x7feb6c198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:39.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.519+0000 7feb71600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb6c103cb0 0x7feb6c19f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:39.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.519+0000 7feb71600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7feb6c10dd20 con 0x7feb6c100320 2024-04-24T09:21:39.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.519+0000 7feb71600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7feb6c10dba0 con 0x7feb6c1032f0 2024-04-24T09:21:39.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.519+0000 7feb71600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7feb6c10dea0 con 0x7feb6c103cb0 2024-04-24T09:21:39.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.519+0000 7feb6aa00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb6c100320 0x7feb6c198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:39.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.519+0000 7feb63e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb6c1032f0 0x7feb6c198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:39.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.519+0000 7feb63e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb6c1032f0 0x7feb6c198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.90:55864/0 (socket says 172.21.15.90:55864) 2024-04-24T09:21:39.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.519+0000 7feb6aa00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb6c100320 0x7feb6c198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42212/0 (socket says 172.21.15.90:42212) 2024-04-24T09:21:39.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.519+0000 7feb63e00640 1 -- 172.21.15.90:0/3299397633 learned_addr learned my addr 172.21.15.90:0/3299397633 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:39.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.519+0000 7feb6b400640 1 --2- 172.21.15.90:0/3299397633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb6c103cb0 0x7feb6c19f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:39.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.520+0000 7feb6aa00640 1 -- 172.21.15.90:0/3299397633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb6c103cb0 msgr2=0x7feb6c19f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:39.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.520+0000 7feb6aa00640 1 --2- 172.21.15.90:0/3299397633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb6c103cb0 0x7feb6c19f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:39.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.520+0000 7feb6aa00640 1 -- 172.21.15.90:0/3299397633 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb6c1032f0 msgr2=0x7feb6c198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:39.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.520+0000 7feb6aa00640 1 --2- 172.21.15.90:0/3299397633 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb6c1032f0 0x7feb6c198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:39.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.520+0000 7feb6aa00640 1 -- 172.21.15.90:0/3299397633 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb6c19ff50 con 0x7feb6c100320 2024-04-24T09:21:39.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.520+0000 7feb63e00640 1 --2- 172.21.15.90:0/3299397633 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb6c1032f0 0x7feb6c198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:21:39.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.520+0000 7feb6b400640 1 --2- 172.21.15.90:0/3299397633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb6c103cb0 0x7feb6c19f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:21:39.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.520+0000 7feb6aa00640 1 --2- 172.21.15.90:0/3299397633 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb6c100320 0x7feb6c198330 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7feb5c00e9c0 tx=0x7feb5c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:39.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.520+0000 7feb63400640 1 -- 172.21.15.90:0/3299397633 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5c00cd80 con 0x7feb6c100320 2024-04-24T09:21:39.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.520+0000 7feb63400640 1 -- 172.21.15.90:0/3299397633 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb5c010910 con 0x7feb6c100320 2024-04-24T09:21:39.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.520+0000 7feb63400640 1 -- 172.21.15.90:0/3299397633 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5c018980 con 0x7feb6c100320 2024-04-24T09:21:39.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.520+0000 7feb71600640 1 -- 172.21.15.90:0/3299397633 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb6c1a0240 con 0x7feb6c100320 2024-04-24T09:21:39.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.520+0000 7feb71600640 1 -- 172.21.15.90:0/3299397633 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb6c1a0790 con 0x7feb6c100320 2024-04-24T09:21:39.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.522+0000 7feb71600640 1 -- 172.21.15.90:0/3299397633 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb6c1093c0 con 0x7feb6c100320 2024-04-24T09:21:39.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.524+0000 7feb63400640 1 -- 172.21.15.90:0/3299397633 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7feb5c01f020 con 0x7feb6c100320 2024-04-24T09:21:39.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.525+0000 7feb63400640 1 --2- 172.21.15.90:0/3299397633 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7feb500a3bb0 0x7feb500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:39.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.525+0000 7feb63400640 1 -- 172.21.15.90:0/3299397633 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7feb5c0d03e0 con 0x7feb6c100320 2024-04-24T09:21:39.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.525+0000 7feb63e00640 1 --2- 172.21.15.90:0/3299397633 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7feb500a3bb0 0x7feb500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:39.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.526+0000 7feb63e00640 1 --2- 172.21.15.90:0/3299397633 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7feb500a3bb0 0x7feb500a6070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7feb6c199840 tx=0x7feb54005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:39.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.529+0000 7feb63400640 1 -- 172.21.15.90:0/3299397633 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feb5c09cd80 con 0x7feb6c100320 2024-04-24T09:21:39.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.732+0000 7feb71600640 1 -- 172.21.15.90:0/3299397633 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7feb6c062c00 con 0x7feb6c100320 2024-04-24T09:21:39.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.732+0000 7feb63400640 1 -- 172.21.15.90:0/3299397633 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7feb5c09c680 con 0x7feb6c100320 2024-04-24T09:21:39.732 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 64 2024-04-24T09:21:39.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.735+0000 7feb71600640 1 -- 172.21.15.90:0/3299397633 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7feb500a3bb0 msgr2=0x7feb500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:39.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.735+0000 7feb71600640 1 --2- 172.21.15.90:0/3299397633 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7feb500a3bb0 0x7feb500a6070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7feb6c199840 tx=0x7feb54005f90 comp rx=0 tx=0).stop 2024-04-24T09:21:39.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.735+0000 7feb71600640 1 -- 172.21.15.90:0/3299397633 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb6c100320 msgr2=0x7feb6c198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:39.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.735+0000 7feb71600640 1 --2- 172.21.15.90:0/3299397633 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb6c100320 0x7feb6c198330 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7feb5c00e9c0 tx=0x7feb5c00ee80 comp rx=0 tx=0).stop 2024-04-24T09:21:39.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.735+0000 7feb71600640 1 -- 172.21.15.90:0/3299397633 shutdown_connections 2024-04-24T09:21:39.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.735+0000 7feb71600640 1 --2- 172.21.15.90:0/3299397633 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7feb500a3bb0 0x7feb500a6070 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:39.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.735+0000 7feb71600640 1 --2- 172.21.15.90:0/3299397633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb6c103cb0 0x7feb6c19f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:39.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.735+0000 7feb71600640 1 --2- 172.21.15.90:0/3299397633 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feb6c1032f0 0x7feb6c198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:39.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.735+0000 7feb71600640 1 --2- 172.21.15.90:0/3299397633 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb6c100320 0x7feb6c198330 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:39.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.735+0000 7feb71600640 1 -- 172.21.15.90:0/3299397633 >> 172.21.15.90:0/3299397633 conn(0x7feb6c0fbc30 msgr2=0x7feb6c106590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:39.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.735+0000 7feb71600640 1 -- 172.21.15.90:0/3299397633 shutdown_connections 2024-04-24T09:21:39.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.736+0000 7feb71600640 1 -- 172.21.15.90:0/3299397633 wait complete. 2024-04-24T09:21:39.753 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:21:39.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:39 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/505643283' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:21:39.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:39 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/1883139514' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:21:39.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.959+0000 7f281ea00640 1 -- 172.21.15.90:0/3351439271 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2818104080 msgr2=0x7f28181044a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:39.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.959+0000 7f281ea00640 1 --2- 172.21.15.90:0/3351439271 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2818104080 0x7f28181044a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f280c009900 tx=0x7f280c02a410 comp rx=0 tx=0).stop 2024-04-24T09:21:39.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.959+0000 7f281ea00640 1 -- 172.21.15.90:0/3351439271 shutdown_connections 2024-04-24T09:21:39.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.959+0000 7f281ea00640 1 --2- 172.21.15.90:0/3351439271 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2818105c30 0x7f281810c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:39.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.959+0000 7f281ea00640 1 --2- 172.21.15.90:0/3351439271 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2818105270 0x7f28181056f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:39.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.959+0000 7f281ea00640 1 --2- 172.21.15.90:0/3351439271 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2818104080 0x7f28181044a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:39.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.959+0000 7f281ea00640 1 -- 172.21.15.90:0/3351439271 >> 172.21.15.90:0/3351439271 conn(0x7f28180ff610 msgr2=0x7f2818101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:39.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.960+0000 7f281ea00640 1 -- 172.21.15.90:0/3351439271 shutdown_connections 2024-04-24T09:21:39.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.960+0000 7f281ea00640 1 -- 172.21.15.90:0/3351439271 wait complete. 2024-04-24T09:21:39.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.961+0000 7f281ea00640 1 Processor -- start 2024-04-24T09:21:39.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.961+0000 7f281ea00640 1 -- start start 2024-04-24T09:21:39.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.961+0000 7f281ea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2818104080 0x7f281819a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:39.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.961+0000 7f281ea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2818105270 0x7f281819a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:39.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.961+0000 7f281ea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2818105c30 0x7f28181a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:39.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.961+0000 7f281ea00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f281810fd60 con 0x7f2818105270 2024-04-24T09:21:39.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.961+0000 7f281ea00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f281810fbe0 con 0x7f2818105c30 2024-04-24T09:21:39.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.961+0000 7f281ea00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f281810fee0 con 0x7f2818104080 2024-04-24T09:21:39.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.961+0000 7f281cc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2818105270 0x7f281819a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:39.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.961+0000 7f281cc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2818105270 0x7f281819a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42220/0 (socket says 172.21.15.90:42220) 2024-04-24T09:21:39.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.962+0000 7f281cc00640 1 -- 172.21.15.90:0/340635996 learned_addr learned my addr 172.21.15.90:0/340635996 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:39.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.962+0000 7f281d600640 1 --2- 172.21.15.90:0/340635996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2818104080 0x7f281819a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:39.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.962+0000 7f281e000640 1 --2- 172.21.15.90:0/340635996 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2818105c30 0x7f28181a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:39.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.962+0000 7f281cc00640 1 -- 172.21.15.90:0/340635996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2818104080 msgr2=0x7f281819a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:39.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.962+0000 7f281cc00640 1 --2- 172.21.15.90:0/340635996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2818104080 0x7f281819a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:39.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.962+0000 7f281cc00640 1 -- 172.21.15.90:0/340635996 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2818105c30 msgr2=0x7f28181a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:39.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.962+0000 7f281cc00640 1 --2- 172.21.15.90:0/340635996 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2818105c30 0x7f28181a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:39.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.962+0000 7f281cc00640 1 -- 172.21.15.90:0/340635996 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28181a1f70 con 0x7f2818105270 2024-04-24T09:21:39.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.962+0000 7f281e000640 1 --2- 172.21.15.90:0/340635996 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2818105c30 0x7f28181a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:21:39.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.962+0000 7f281cc00640 1 --2- 172.21.15.90:0/340635996 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2818105270 0x7f281819a890 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f280400e9f0 tx=0x7f280400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:39.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.962+0000 7f2816000640 1 -- 172.21.15.90:0/340635996 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280400cd80 con 0x7f2818105270 2024-04-24T09:21:39.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.962+0000 7f281ea00640 1 -- 172.21.15.90:0/340635996 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28181a2260 con 0x7f2818105270 2024-04-24T09:21:39.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.962+0000 7f2816000640 1 -- 172.21.15.90:0/340635996 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2804010910 con 0x7f2818105270 2024-04-24T09:21:39.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.962+0000 7f2816000640 1 -- 172.21.15.90:0/340635996 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28040188f0 con 0x7f2818105270 2024-04-24T09:21:39.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.963+0000 7f281ea00640 1 -- 172.21.15.90:0/340635996 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28181a27b0 con 0x7f2818105270 2024-04-24T09:21:39.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.964+0000 7f281ea00640 1 -- 172.21.15.90:0/340635996 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f281810b3a0 con 0x7f2818105270 2024-04-24T09:21:39.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.966+0000 7f2816000640 1 -- 172.21.15.90:0/340635996 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2804010ac0 con 0x7f2818105270 2024-04-24T09:21:39.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.968+0000 7f2816000640 1 --2- 172.21.15.90:0/340635996 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f28000a3bb0 0x7f28000a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:39.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.968+0000 7f2816000640 1 -- 172.21.15.90:0/340635996 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f28040d0880 con 0x7f2818105270 2024-04-24T09:21:39.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.968+0000 7f281d600640 1 --2- 172.21.15.90:0/340635996 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f28000a3bb0 0x7f28000a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:39.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.969+0000 7f281d600640 1 --2- 172.21.15.90:0/340635996 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f28000a3bb0 0x7f28000a6070 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f280c02aa00 tx=0x7f280c0315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:39.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:39.972+0000 7f2816000640 1 -- 172.21.15.90:0/340635996 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f280409d220 con 0x7f2818105270 2024-04-24T09:21:40.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:39 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/505643283' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:21:40.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:39 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/1883139514' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:21:40.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:39 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/505643283' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:21:40.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:39 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/1883139514' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:21:40.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.218+0000 7f281ea00640 1 -- 172.21.15.90:0/340635996 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2818062c00 con 0x7f2818105270 2024-04-24T09:21:40.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.220+0000 7f2816000640 1 -- 172.21.15.90:0/340635996 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11928 (secure 0 0 0) 0x7f280409cb20 con 0x7f2818105270 2024-04-24T09:21:40.220 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:21:40.220 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/227251251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":227251251},{"type":"v1","addr":"172.21.15.90:6837","nonce":227251251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/1904617971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":1904617971},{"type":"v1","addr":"172.21.15.90:6839","nonce":1904617971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T09:19:09.430875+0000","modified":"2024-04-24T09:19:17.259940+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24547,"mds_2":24545,"mds_3":24526,"mds_4":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6833/219377141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":219377141},{"type":"v1","addr":"172.21.15.114:6833","nonce":219377141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"j","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.90:6835/2396995634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2396995634},{"type":"v1","addr":"172.21.15.90:6835","nonce":2396995634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/779642417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":779642417},{"type":"v1","addr":"172.21.15.149:6833","nonce":779642417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6835/2565340800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":2565340800},{"type":"v1","addr":"172.21.15.114:6835","nonce":2565340800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6837/875324280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":875324280},{"type":"v1","addr":"172.21.15.114:6837","nonce":875324280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/3157375624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3157375624},{"type":"v1","addr":"172.21.15.149:6835","nonce":3157375624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.149:6837/2250310587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2250310587},{"type":"v1","addr":"172.21.15.149:6837","nonce":2250310587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.90:6841/397593398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":397593398},{"type":"v1","addr":"172.21.15.90:6841","nonce":397593398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.149:6839/652734263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":652734263},{"type":"v1","addr":"172.21.15.149:6839","nonce":652734263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6839/1025403710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":1025403710},{"type":"v1","addr":"172.21.15.114:6839","nonce":1025403710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24545,24557,24446,24526,24547]},"id":1}]} 2024-04-24T09:21:40.220 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 27 2024-04-24T09:21:40.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.222+0000 7f281ea00640 1 -- 172.21.15.90:0/340635996 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f28000a3bb0 msgr2=0x7f28000a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:40.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.222+0000 7f281ea00640 1 --2- 172.21.15.90:0/340635996 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f28000a3bb0 0x7f28000a6070 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f280c02aa00 tx=0x7f280c0315b0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.222+0000 7f281ea00640 1 -- 172.21.15.90:0/340635996 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2818105270 msgr2=0x7f281819a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:40.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.222+0000 7f281ea00640 1 --2- 172.21.15.90:0/340635996 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2818105270 0x7f281819a890 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f280400e9f0 tx=0x7f280400eeb0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.222+0000 7f281ea00640 1 -- 172.21.15.90:0/340635996 shutdown_connections 2024-04-24T09:21:40.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.222+0000 7f281ea00640 1 --2- 172.21.15.90:0/340635996 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f28000a3bb0 0x7f28000a6070 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.222+0000 7f281ea00640 1 --2- 172.21.15.90:0/340635996 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2818105c30 0x7f28181a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.222+0000 7f281ea00640 1 --2- 172.21.15.90:0/340635996 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2818105270 0x7f281819a890 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.222+0000 7f281ea00640 1 --2- 172.21.15.90:0/340635996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2818104080 0x7f281819a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.222+0000 7f281ea00640 1 -- 172.21.15.90:0/340635996 >> 172.21.15.90:0/340635996 conn(0x7f28180ff610 msgr2=0x7f2818108570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:40.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.223+0000 7f281ea00640 1 -- 172.21.15.90:0/340635996 shutdown_connections 2024-04-24T09:21:40.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.223+0000 7f281ea00640 1 -- 172.21.15.90:0/340635996 wait complete. 2024-04-24T09:21:40.243 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T09:21:40.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.443+0000 7fda87e00640 1 -- 172.21.15.90:0/1332651018 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fda88105a10 msgr2=0x7fda88105e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:40.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.443+0000 7fda87e00640 1 --2- 172.21.15.90:0/1332651018 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fda88105a10 0x7fda88105e90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fda80009be0 tx=0x7fda8002a6d0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.444+0000 7fda87e00640 1 -- 172.21.15.90:0/1332651018 shutdown_connections 2024-04-24T09:21:40.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.444+0000 7fda87e00640 1 --2- 172.21.15.90:0/1332651018 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fda88105a10 0x7fda88105e90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.444+0000 7fda87e00640 1 --2- 172.21.15.90:0/1332651018 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda88103060 0x7fda881054d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.444+0000 7fda87e00640 1 --2- 172.21.15.90:0/1332651018 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda881006f0 0x7fda88102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.444+0000 7fda87e00640 1 -- 172.21.15.90:0/1332651018 >> 172.21.15.90:0/1332651018 conn(0x7fda880fc2d0 msgr2=0x7fda880fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:40.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.444+0000 7fda87e00640 1 -- 172.21.15.90:0/1332651018 shutdown_connections 2024-04-24T09:21:40.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.444+0000 7fda87e00640 1 -- 172.21.15.90:0/1332651018 wait complete. 2024-04-24T09:21:40.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.445+0000 7fda87e00640 1 Processor -- start 2024-04-24T09:21:40.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.445+0000 7fda87e00640 1 -- start start 2024-04-24T09:21:40.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.445+0000 7fda87e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fda881006f0 0x7fda8819e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:40.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.445+0000 7fda87e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda88103060 0x7fda8819ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:40.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.445+0000 7fda87e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda88105a10 0x7fda881a5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:40.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.445+0000 7fda87e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fda88114120 con 0x7fda88105a10 2024-04-24T09:21:40.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.445+0000 7fda87e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fda88113fa0 con 0x7fda881006f0 2024-04-24T09:21:40.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.445+0000 7fda87e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fda881142a0 con 0x7fda88103060 2024-04-24T09:21:40.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.445+0000 7fda87400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda88105a10 0x7fda881a5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:40.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.445+0000 7fda7e000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda88103060 0x7fda8819ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:40.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.445+0000 7fda7e000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda88103060 0x7fda8819ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:54424/0 (socket says 172.21.15.90:54424) 2024-04-24T09:21:40.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.445+0000 7fda87400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda88105a10 0x7fda881a5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42244/0 (socket says 172.21.15.90:42244) 2024-04-24T09:21:40.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.445+0000 7fda7e000640 1 -- 172.21.15.90:0/2618229429 learned_addr learned my addr 172.21.15.90:0/2618229429 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:40.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.445+0000 7fda86a00640 1 --2- 172.21.15.90:0/2618229429 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fda881006f0 0x7fda8819e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:40.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.446+0000 7fda7e000640 1 -- 172.21.15.90:0/2618229429 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fda881006f0 msgr2=0x7fda8819e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:40.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.446+0000 7fda7e000640 1 --2- 172.21.15.90:0/2618229429 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fda881006f0 0x7fda8819e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.446+0000 7fda7e000640 1 -- 172.21.15.90:0/2618229429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda88105a10 msgr2=0x7fda881a5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:40.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.446+0000 7fda7e000640 1 --2- 172.21.15.90:0/2618229429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda88105a10 0x7fda881a5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.446+0000 7fda7e000640 1 -- 172.21.15.90:0/2618229429 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda881a6320 con 0x7fda88103060 2024-04-24T09:21:40.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.446+0000 7fda87400640 1 --2- 172.21.15.90:0/2618229429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda88105a10 0x7fda881a5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:21:40.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.446+0000 7fda86a00640 1 --2- 172.21.15.90:0/2618229429 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fda881006f0 0x7fda8819e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:21:40.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.446+0000 7fda7e000640 1 --2- 172.21.15.90:0/2618229429 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda88103060 0x7fda8819ec40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fda7000e9c0 tx=0x7fda7000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:40.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.446+0000 7fda8ca00640 1 -- 172.21.15.90:0/2618229429 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7000ccc0 con 0x7fda88103060 2024-04-24T09:21:40.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.447+0000 7fda8ca00640 1 -- 172.21.15.90:0/2618229429 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda70005500 con 0x7fda88103060 2024-04-24T09:21:40.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.447+0000 7fda87e00640 1 -- 172.21.15.90:0/2618229429 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda881a6610 con 0x7fda88103060 2024-04-24T09:21:40.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.447+0000 7fda8ca00640 1 -- 172.21.15.90:0/2618229429 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda70010430 con 0x7fda88103060 2024-04-24T09:21:40.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.447+0000 7fda87e00640 1 -- 172.21.15.90:0/2618229429 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda881a6bb0 con 0x7fda88103060 2024-04-24T09:21:40.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.448+0000 7fda87e00640 1 -- 172.21.15.90:0/2618229429 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda880fddc0 con 0x7fda88103060 2024-04-24T09:21:40.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.450+0000 7fda8ca00640 1 -- 172.21.15.90:0/2618229429 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fda700046c0 con 0x7fda88103060 2024-04-24T09:21:40.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.451+0000 7fda8ca00640 1 --2- 172.21.15.90:0/2618229429 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fda6c0a3c30 0x7fda6c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:40.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.451+0000 7fda8ca00640 1 -- 172.21.15.90:0/2618229429 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fda700d06a0 con 0x7fda88103060 2024-04-24T09:21:40.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.451+0000 7fda86a00640 1 --2- 172.21.15.90:0/2618229429 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fda6c0a3c30 0x7fda6c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:40.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.452+0000 7fda86a00640 1 --2- 172.21.15.90:0/2618229429 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fda6c0a3c30 0x7fda6c0a60f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fda78005fd0 tx=0x7fda7800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:40.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.453+0000 7fda8ca00640 1 -- 172.21.15.90:0/2618229429 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fda7009d040 con 0x7fda88103060 2024-04-24T09:21:40.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.660+0000 7fda87e00640 1 -- 172.21.15.90:0/2618229429 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fda8802d870 con 0x7fda88103060 2024-04-24T09:21:40.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.662+0000 7fda8ca00640 1 -- 172.21.15.90:0/2618229429 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22652 (secure 0 0 0) 0x7fda7009c940 con 0x7fda88103060 2024-04-24T09:21:40.663 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:21:40.663 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":88,"fsid":"30ee9be2-021a-11ef-bc93-c7b262605968","created":"2024-04-24T09:09:49.496019+0000","modified":"2024-04-24T09:19:58.609640+0000","last_up_change":"2024-04-24T09:18:10.455632+0000","last_in_change":"2024-04-24T09:17:50.600366+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T09:13:45.548774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T09:19:07.185580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T09:19:08.374818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13a59b9f-b46e-4ec4-90af-bb7a3d5c753b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6803","nonce":2864288066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6805","nonce":2864288066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6809","nonce":2864288066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6807","nonce":2864288066}]},"public_addr":"172.21.15.90:6803/2864288066","cluster_addr":"172.21.15.90:6805/2864288066","heartbeat_back_addr":"172.21.15.90:6809/2864288066","heartbeat_front_addr":"172.21.15.90:6807/2864288066","state":["exists","up"]},{"osd":1,"uuid":"1567ce8f-24e6-42cf-bf64-f6f0c534a9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6801","nonce":344222414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6803","nonce":344222414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6807","nonce":344222414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6805","nonce":344222414}]},"public_addr":"172.21.15.114:6801/344222414","cluster_addr":"172.21.15.114:6803/344222414","heartbeat_back_addr":"172.21.15.114:6807/344222414","heartbeat_front_addr":"172.21.15.114:6805/344222414","state":["exists","up"]},{"osd":2,"uuid":"f861b304-9c7d-4629-8522-37f33a747945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6801","nonce":3257067419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6803","nonce":3257067419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6807","nonce":3257067419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6805","nonce":3257067419}]},"public_addr":"172.21.15.149:6801/3257067419","cluster_addr":"172.21.15.149:6803/3257067419","heartbeat_back_addr":"172.21.15.149:6807/3257067419","heartbeat_front_addr":"172.21.15.149:6805/3257067419","state":["exists","up"]},{"osd":3,"uuid":"9377ab6e-5996-45fb-93a9-475a2e5f17a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6811","nonce":3068524970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6813","nonce":3068524970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6817","nonce":3068524970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6815","nonce":3068524970}]},"public_addr":"172.21.15.90:6811/3068524970","cluster_addr":"172.21.15.90:6813/3068524970","heartbeat_back_addr":"172.21.15.90:6817/3068524970","heartbeat_front_addr":"172.21.15.90:6815/3068524970","state":["exists","up"]},{"osd":4,"uuid":"e2ddd19e-e641-4d61-b342-02b5f12b1c4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6809","nonce":2782015522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6811","nonce":2782015522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6815","nonce":2782015522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6813","nonce":2782015522}]},"public_addr":"172.21.15.114:6809/2782015522","cluster_addr":"172.21.15.114:6811/2782015522","heartbeat_back_addr":"172.21.15.114:6815/2782015522","heartbeat_front_addr":"172.21.15.114:6813/2782015522","state":["exists","up"]},{"osd":5,"uuid":"b6ff9d10-5210-4b46-a192-6dcbf9d1f156","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6809","nonce":4266460479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6811","nonce":4266460479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6815","nonce":4266460479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6813","nonce":4266460479}]},"public_addr":"172.21.15.149:6809/4266460479","cluster_addr":"172.21.15.149:6811/4266460479","heartbeat_back_addr":"172.21.15.149:6815/4266460479","heartbeat_front_addr":"172.21.15.149:6813/4266460479","state":["exists","up"]},{"osd":6,"uuid":"4287b5a1-f953-421c-bf62-5e390bee834d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6819","nonce":1121461276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6821","nonce":1121461276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6825","nonce":1121461276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6823","nonce":1121461276}]},"public_addr":"172.21.15.90:6819/1121461276","cluster_addr":"172.21.15.90:6821/1121461276","heartbeat_back_addr":"172.21.15.90:6825/1121461276","heartbeat_front_addr":"172.21.15.90:6823/1121461276","state":["exists","up"]},{"osd":7,"uuid":"a54e507d-d6ba-4bf2-9a7f-a7b95c3859bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6817","nonce":3222546280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6819","nonce":3222546280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6823","nonce":3222546280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6821","nonce":3222546280}]},"public_addr":"172.21.15.114:6817/3222546280","cluster_addr":"172.21.15.114:6819/3222546280","heartbeat_back_addr":"172.21.15.114:6823/3222546280","heartbeat_front_addr":"172.21.15.114:6821/3222546280","state":["exists","up"]},{"osd":8,"uuid":"9950b010-5865-4109-ae37-582b1dbd2f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6817","nonce":157615149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6819","nonce":157615149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6823","nonce":157615149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6821","nonce":157615149}]},"public_addr":"172.21.15.149:6817/157615149","cluster_addr":"172.21.15.149:6819/157615149","heartbeat_back_addr":"172.21.15.149:6823/157615149","heartbeat_front_addr":"172.21.15.149:6821/157615149","state":["exists","up"]},{"osd":9,"uuid":"31e2635c-e234-421f-9757-592fd673c936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6827","nonce":1307318076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6829","nonce":1307318076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6833","nonce":1307318076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6831","nonce":1307318076}]},"public_addr":"172.21.15.90:6827/1307318076","cluster_addr":"172.21.15.90:6829/1307318076","heartbeat_back_addr":"172.21.15.90:6833/1307318076","heartbeat_front_addr":"172.21.15.90:6831/1307318076","state":["exists","up"]},{"osd":10,"uuid":"46a3b2a6-e6e5-4578-b4bb-f3e998b9bc95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6825","nonce":1719802140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6827","nonce":1719802140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6831","nonce":1719802140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6829","nonce":1719802140}]},"public_addr":"172.21.15.114:6825/1719802140","cluster_addr":"172.21.15.114:6827/1719802140","heartbeat_back_addr":"172.21.15.114:6831/1719802140","heartbeat_front_addr":"172.21.15.114:6829/1719802140","state":["exists","up"]},{"osd":11,"uuid":"ccd4456b-e947-46cf-a779-148ffa289b58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6825","nonce":1737275166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6827","nonce":1737275166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6831","nonce":1737275166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6829","nonce":1737275166}]},"public_addr":"172.21.15.149:6825/1737275166","cluster_addr":"172.21.15.149:6827/1737275166","heartbeat_back_addr":"172.21.15.149:6831/1737275166","heartbeat_front_addr":"172.21.15.149:6829/1737275166","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:12:53.015078+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:13:18.047053+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:13:41.935891+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:14:15.095388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:14:42.512355+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:15:08.113775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:15:43.828550+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:16:10.901909+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:16:36.422962+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:17:13.924446+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:17:42.814401+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:18:08.672793+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/3090852580":"2024-04-25T09:10:57.377655+0000","172.21.15.90:0/687197394":"2024-04-25T09:10:57.377655+0000","172.21.15.90:6801/1809038230":"2024-04-25T09:10:57.377655+0000","172.21.15.90:6800/1809038230":"2024-04-25T09:10:57.377655+0000","172.21.15.90:0/2964149071":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/3617388113":"2024-04-25T09:10:22.599674+0000","172.21.15.90:6801/3089140842":"2024-04-25T09:10:22.599674+0000","172.21.15.90:6800/3089140842":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/2840416149":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/305432710":"2024-04-25T09:10:57.377655+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T09:21:40.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.665+0000 7fda87e00640 1 -- 172.21.15.90:0/2618229429 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fda6c0a3c30 msgr2=0x7fda6c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:40.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.665+0000 7fda87e00640 1 --2- 172.21.15.90:0/2618229429 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fda6c0a3c30 0x7fda6c0a60f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fda78005fd0 tx=0x7fda7800b390 comp rx=0 tx=0).stop 2024-04-24T09:21:40.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.665+0000 7fda87e00640 1 -- 172.21.15.90:0/2618229429 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda88103060 msgr2=0x7fda8819ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:40.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.665+0000 7fda87e00640 1 --2- 172.21.15.90:0/2618229429 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda88103060 0x7fda8819ec40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fda7000e9c0 tx=0x7fda7000ee80 comp rx=0 tx=0).stop 2024-04-24T09:21:40.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.666+0000 7fda87e00640 1 -- 172.21.15.90:0/2618229429 shutdown_connections 2024-04-24T09:21:40.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.666+0000 7fda87e00640 1 --2- 172.21.15.90:0/2618229429 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fda6c0a3c30 0x7fda6c0a60f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.666+0000 7fda87e00640 1 --2- 172.21.15.90:0/2618229429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda88105a10 0x7fda881a5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.666+0000 7fda87e00640 1 --2- 172.21.15.90:0/2618229429 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda88103060 0x7fda8819ec40 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.666+0000 7fda87e00640 1 --2- 172.21.15.90:0/2618229429 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fda881006f0 0x7fda8819e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.666+0000 7fda87e00640 1 -- 172.21.15.90:0/2618229429 >> 172.21.15.90:0/2618229429 conn(0x7fda880fc2d0 msgr2=0x7fda88101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:40.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.666+0000 7fda87e00640 1 -- 172.21.15.90:0/2618229429 shutdown_connections 2024-04-24T09:21:40.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.666+0000 7fda87e00640 1 -- 172.21.15.90:0/2618229429 wait complete. 2024-04-24T09:21:40.686 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-24T09:19:09.430875+0000', 'modified': '2024-04-24T09:19:17.259940+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24557, 'mds_1': 24547, 'mds_2': 24545, 'mds_3': 24526, 'mds_4': 24446}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24428': {'gid': 24428, 'name': 'k', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.114:6833/219377141', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6832', 'nonce': 219377141}, {'type': 'v1', 'addr': '172.21.15.114:6833', 'nonce': 219377141}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24446': {'gid': 24446, 'name': 'j', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.90:6835/2396995634', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6834', 'nonce': 2396995634}, {'type': 'v1', 'addr': '172.21.15.90:6835', 'nonce': 2396995634}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.149:6833/779642417', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6832', 'nonce': 779642417}, {'type': 'v1', 'addr': '172.21.15.149:6833', 'nonce': 779642417}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24476': {'gid': 24476, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.114:6835/2565340800', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6834', 'nonce': 2565340800}, {'type': 'v1', 'addr': '172.21.15.114:6835', 'nonce': 2565340800}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24500': {'gid': 24500, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.114:6837/875324280', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6836', 'nonce': 875324280}, {'type': 'v1', 'addr': '172.21.15.114:6837', 'nonce': 875324280}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24514': {'gid': 24514, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.149:6835/3157375624', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6834', 'nonce': 3157375624}, {'type': 'v1', 'addr': '172.21.15.149:6835', 'nonce': 3157375624}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24526': {'gid': 24526, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.149:6837/2250310587', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6836', 'nonce': 2250310587}, {'type': 'v1', 'addr': '172.21.15.149:6837', 'nonce': 2250310587}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24545': {'gid': 24545, 'name': 'a', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.90:6841/397593398', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6840', 'nonce': 397593398}, {'type': 'v1', 'addr': '172.21.15.90:6841', 'nonce': 397593398}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24547': {'gid': 24547, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.149:6839/652734263', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6838', 'nonce': 652734263}, {'type': 'v1', 'addr': '172.21.15.149:6839', 'nonce': 652734263}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24557': {'gid': 24557, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.114:6839/1025403710', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6838', 'nonce': 1025403710}, {'type': 'v1', 'addr': '172.21.15.114:6839', 'nonce': 1025403710}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24557, 'qdb_cluster': [24545, 24557, 24446, 24526, 24547]} 2024-04-24T09:21:40.686 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-24T09:21:40.687 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.j status 2024-04-24T09:21:40.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.891+0000 7fa12b200640 1 -- 172.21.15.90:0/3327150243 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa12c103a20 msgr2=0x7fa12c10a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:40.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.891+0000 7fa12b200640 1 --2- 172.21.15.90:0/3327150243 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa12c103a20 0x7fa12c10a340 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa124009ca0 tx=0x7fa12402a8c0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.891+0000 7fa12b200640 1 -- 172.21.15.90:0/3327150243 shutdown_connections 2024-04-24T09:21:40.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.891+0000 7fa12b200640 1 --2- 172.21.15.90:0/3327150243 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa12c103a20 0x7fa12c10a340 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.891+0000 7fa12b200640 1 --2- 172.21.15.90:0/3327150243 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa12c103060 0x7fa12c1034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.891+0000 7fa12b200640 1 --2- 172.21.15.90:0/3327150243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa12c101f10 0x7fa12c102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.891+0000 7fa12b200640 1 -- 172.21.15.90:0/3327150243 >> 172.21.15.90:0/3327150243 conn(0x7fa12c0fd4a0 msgr2=0x7fa12c0ff8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:40.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.892+0000 7fa12b200640 1 -- 172.21.15.90:0/3327150243 shutdown_connections 2024-04-24T09:21:40.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.892+0000 7fa12b200640 1 -- 172.21.15.90:0/3327150243 wait complete. 2024-04-24T09:21:40.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.893+0000 7fa12b200640 1 Processor -- start 2024-04-24T09:21:40.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.893+0000 7fa12b200640 1 -- start start 2024-04-24T09:21:40.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.893+0000 7fa12b200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa12c101f10 0x7fa12c19a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:40.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.893+0000 7fa12b200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa12c103060 0x7fa12c19a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:40.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.893+0000 7fa12b200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa12c103a20 0x7fa12c1a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:40.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.893+0000 7fa12b200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa12c10dbf0 con 0x7fa12c103060 2024-04-24T09:21:40.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.893+0000 7fa12b200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa12c10da70 con 0x7fa12c103a20 2024-04-24T09:21:40.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.893+0000 7fa12b200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa12c10dd70 con 0x7fa12c101f10 2024-04-24T09:21:40.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.893+0000 7fa123e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa12c103060 0x7fa12c19a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:40.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.893+0000 7fa129e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa12c101f10 0x7fa12c19a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:40.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.893+0000 7fa129e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa12c101f10 0x7fa12c19a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:54436/0 (socket says 172.21.15.90:54436) 2024-04-24T09:21:40.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.893+0000 7fa129e00640 1 -- 172.21.15.90:0/185718971 learned_addr learned my addr 172.21.15.90:0/185718971 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:40.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.893+0000 7fa12a800640 1 --2- 172.21.15.90:0/185718971 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa12c103a20 0x7fa12c1a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:40.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.894+0000 7fa129e00640 1 -- 172.21.15.90:0/185718971 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa12c103a20 msgr2=0x7fa12c1a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:40.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.894+0000 7fa129e00640 1 --2- 172.21.15.90:0/185718971 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa12c103a20 0x7fa12c1a1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.894+0000 7fa129e00640 1 -- 172.21.15.90:0/185718971 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa12c103060 msgr2=0x7fa12c19a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:40.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.894+0000 7fa129e00640 1 --2- 172.21.15.90:0/185718971 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa12c103060 0x7fa12c19a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.894+0000 7fa129e00640 1 -- 172.21.15.90:0/185718971 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa12c1a1fa0 con 0x7fa12c101f10 2024-04-24T09:21:40.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.894+0000 7fa129e00640 1 --2- 172.21.15.90:0/185718971 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa12c101f10 0x7fa12c19a380 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa11c00e9f0 tx=0x7fa11c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:40.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.894+0000 7fa122a00640 1 -- 172.21.15.90:0/185718971 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c00cd80 con 0x7fa12c101f10 2024-04-24T09:21:40.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.894+0000 7fa122a00640 1 -- 172.21.15.90:0/185718971 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa11c010910 con 0x7fa12c101f10 2024-04-24T09:21:40.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.894+0000 7fa12b200640 1 -- 172.21.15.90:0/185718971 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa12c1a2290 con 0x7fa12c101f10 2024-04-24T09:21:40.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.894+0000 7fa122a00640 1 -- 172.21.15.90:0/185718971 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c018990 con 0x7fa12c101f10 2024-04-24T09:21:40.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.894+0000 7fa12b200640 1 -- 172.21.15.90:0/185718971 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa12c1a27e0 con 0x7fa12c101f10 2024-04-24T09:21:40.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.898+0000 7fa122a00640 1 -- 172.21.15.90:0/185718971 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa11c010ac0 con 0x7fa12c101f10 2024-04-24T09:21:40.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.898+0000 7fa133e1d740 1 Processor -- start 2024-04-24T09:21:40.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.898+0000 7fa133e1d740 1 -- start start 2024-04-24T09:21:40.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.898+0000 7fa133e1d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dfbbc2b260 0x55dfbbc2b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:40.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.898+0000 7fa133e1d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfbbc2bc50 0x55dfbbc30750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:40.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.898+0000 7fa133e1d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dfbbc30dc0 0x55dfbbc33250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:40.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.898+0000 7fa133e1d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55dfbbb90b10 con 0x55dfbbc30dc0 2024-04-24T09:21:40.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.898+0000 7fa133e1d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55dfbbb372b0 con 0x55dfbbc2b260 2024-04-24T09:21:40.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.898+0000 7fa133e1d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55dfbbb8cad0 con 0x55dfbbc2bc50 2024-04-24T09:21:40.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.898+0000 7fa129e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfbbc2bc50 0x55dfbbc30750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:40.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.898+0000 7fa12a800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dfbbc2b260 0x55dfbbc2b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:40.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.898+0000 7fa12a800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dfbbc2b260 0x55dfbbc2b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.90:55926/0 (socket says 172.21.15.90:55926) 2024-04-24T09:21:40.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.898+0000 7fa12a800640 1 -- 172.21.15.90:0/259357304 learned_addr learned my addr 172.21.15.90:0/259357304 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:40.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.898+0000 7fa123e00640 1 --2- 172.21.15.90:0/259357304 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dfbbc30dc0 0x55dfbbc33250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:40.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.899+0000 7fa12a800640 1 -- 172.21.15.90:0/259357304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfbbc2bc50 msgr2=0x55dfbbc30750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:40.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.899+0000 7fa12a800640 1 --2- 172.21.15.90:0/259357304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfbbc2bc50 0x55dfbbc30750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.899+0000 7fa12a800640 1 -- 172.21.15.90:0/259357304 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dfbbc30dc0 msgr2=0x55dfbbc33250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:40.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.899+0000 7fa12a800640 1 --2- 172.21.15.90:0/259357304 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dfbbc30dc0 0x55dfbbc33250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.899+0000 7fa12a800640 1 -- 172.21.15.90:0/259357304 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfbb8a32d0 con 0x55dfbbc2b260 2024-04-24T09:21:40.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.899+0000 7fa129e00640 1 --2- 172.21.15.90:0/259357304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfbbc2bc50 0x55dfbbc30750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:21:40.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.899+0000 7fa12a800640 1 --2- 172.21.15.90:0/259357304 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dfbbc2b260 0x55dfbbc2b680 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa124031c10 tx=0x7fa124005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2d2695b5cc01229 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:40.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.899+0000 7fa122a00640 1 --2- 172.21.15.90:0/185718971 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa1100a3bb0 0x7fa1100a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:40.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.899+0000 7fa11b400640 1 -- 172.21.15.90:0/259357304 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa124031650 con 0x55dfbbc2b260 2024-04-24T09:21:40.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.899+0000 7fa11b400640 1 -- 172.21.15.90:0/259357304 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa124036040 con 0x55dfbbc2b260 2024-04-24T09:21:40.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.899+0000 7fa11b400640 1 -- 172.21.15.90:0/259357304 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa124004710 con 0x55dfbbc2b260 2024-04-24T09:21:40.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.900+0000 7fa122a00640 1 -- 172.21.15.90:0/185718971 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fa11c0d0810 con 0x7fa12c101f10 2024-04-24T09:21:40.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.900+0000 7fa129e00640 1 --2- 172.21.15.90:0/185718971 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa1100a3bb0 0x7fa1100a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:40.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.900+0000 7fa133e1d740 1 -- 172.21.15.90:0/259357304 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dfbbc2b260 msgr2=0x55dfbbc2b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:40.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.900+0000 7fa129e00640 1 --2- 172.21.15.90:0/185718971 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa1100a3bb0 0x7fa1100a6070 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa11c00c6a0 tx=0x7fa11c0d6530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:40.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.900+0000 7fa133e1d740 1 --2- 172.21.15.90:0/259357304 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dfbbc2b260 0x55dfbbc2b680 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa124031c10 tx=0x7fa124005ef0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.900+0000 7fa133e1d740 1 -- 172.21.15.90:0/259357304 shutdown_connections 2024-04-24T09:21:40.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.900+0000 7fa133e1d740 1 --2- 172.21.15.90:0/259357304 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dfbbc30dc0 0x55dfbbc33250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.900+0000 7fa133e1d740 1 --2- 172.21.15.90:0/259357304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfbbc2bc50 0x55dfbbc30750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.900+0000 7fa133e1d740 1 --2- 172.21.15.90:0/259357304 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dfbbc2b260 0x55dfbbc2b680 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.900+0000 7fa133e1d740 1 -- 172.21.15.90:0/259357304 >> 172.21.15.90:0/259357304 conn(0x55dfbbc268d0 msgr2=0x55dfbbc26cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:40.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.900+0000 7fa133e1d740 1 -- 172.21.15.90:0/259357304 shutdown_connections 2024-04-24T09:21:40.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.901+0000 7fa133e1d740 1 -- 172.21.15.90:0/259357304 wait complete. 2024-04-24T09:21:40.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.901+0000 7fa133e1d740 1 -- start start 2024-04-24T09:21:40.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.901+0000 7fa133e1d740 1 Processor -- start 2024-04-24T09:21:40.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.902+0000 7fa133e1d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dfbbc2b260 0x55dfbbc35620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:40.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.902+0000 7fa133e1d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dfbbc2bc50 0x55dfbbc35b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:40.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.902+0000 7fa133e1d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfbbc30dc0 0x55dfbbc3cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:40.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.902+0000 7fa133e1d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55dfbbc33d10 con 0x55dfbbc2bc50 2024-04-24T09:21:40.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.902+0000 7fa133e1d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55dfbbb4a590 con 0x55dfbbc2b260 2024-04-24T09:21:40.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.902+0000 7fa133e1d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55dfbbc33e90 con 0x55dfbbc30dc0 2024-04-24T09:21:40.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.902+0000 7fa133e1d740 10 client.0 _ll_register_callbacks cb 0x55dfbbb2d3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:21:40.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.902+0000 7fa123e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dfbbc2bc50 0x55dfbbc35b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:40.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.902+0000 7fa12a800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dfbbc2b260 0x55dfbbc35620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:40.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.902+0000 7fa12a800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dfbbc2b260 0x55dfbbc35620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.90:55928/0 (socket says 172.21.15.90:55928) 2024-04-24T09:21:40.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.902+0000 7fa12a800640 1 -- 172.21.15.90:0/1447577216 learned_addr learned my addr 172.21.15.90:0/1447577216 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:40.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.902+0000 7fa12a800640 1 --2- 172.21.15.90:0/1447577216 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfbbc30dc0 0x55dfbbc3cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:40.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.902+0000 7fa123e00640 1 -- 172.21.15.90:0/1447577216 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfbbc30dc0 msgr2=0x55dfbbc3cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:40.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.902+0000 7fa123e00640 1 --2- 172.21.15.90:0/1447577216 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfbbc30dc0 0x55dfbbc3cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.902+0000 7fa123e00640 1 -- 172.21.15.90:0/1447577216 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dfbbc2b260 msgr2=0x55dfbbc35620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:40.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.902+0000 7fa123e00640 1 --2- 172.21.15.90:0/1447577216 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dfbbc2b260 0x55dfbbc35620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.902+0000 7fa123e00640 1 -- 172.21.15.90:0/1447577216 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfbb96d6b0 con 0x55dfbbc2bc50 2024-04-24T09:21:40.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.902+0000 7fa12a800640 1 --2- 172.21.15.90:0/1447577216 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfbbc30dc0 0x55dfbbc3cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:21:40.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.903+0000 7fa123e00640 1 --2- 172.21.15.90:0/1447577216 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dfbbc2bc50 0x55dfbbc35b60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa12c19b890 tx=0x7fa114012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a62d07e69411f3c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:40.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.903+0000 7fa10fe00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-24T09:21:40.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.903+0000 7fa10fe00640 1 -- 172.21.15.90:0/1447577216 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa114004050 con 0x55dfbbc2bc50 2024-04-24T09:21:40.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.903+0000 7fa10fe00640 1 -- 172.21.15.90:0/1447577216 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fa1140041f0 con 0x55dfbbc2bc50 2024-04-24T09:21:40.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.903+0000 7fa133e1d740 1 -- 172.21.15.90:0/1447577216 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa124036040 con 0x55dfbbc2bc50 2024-04-24T09:21:40.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.903+0000 7fa10fe00640 1 -- 172.21.15.90:0/1447577216 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa1140043f0 con 0x55dfbbc2bc50 2024-04-24T09:21:40.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.903+0000 7fa133e1d740 10 client.15078 fetch_fsmap learned FSMap version 27 2024-04-24T09:21:40.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.903+0000 7fa133e1d740 1 -- 172.21.15.90:0/1447577216 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dfbbc3ffe0 con 0x55dfbbc2bc50 2024-04-24T09:21:40.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.903+0000 7fa10fe00640 1 -- 172.21.15.90:0/1447577216 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa124036040 con 0x55dfbbc2bc50 2024-04-24T09:21:40.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.904+0000 7fa10fe00640 1 -- 172.21.15.90:0/1447577216 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa114003b80 con 0x55dfbbc2bc50 2024-04-24T09:21:40.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.904+0000 7fa133e1d740 10 client.15078 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T09:21:40.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.904+0000 7fa133e1d740 10 client.15078 resolve_mds: resolved name 'j' to daemon mds.j 2024-04-24T09:21:40.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.904+0000 7fa133e1d740 20 client.15078 populate_metadata read hostname 'smithi090' 2024-04-24T09:21:40.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.904+0000 7fa133e1d740 1 --2- 172.21.15.90:0/1447577216 >> [v2:172.21.15.90:6834/2396995634,v1:172.21.15.90:6835/2396995634] conn(0x55dfbbc42660 0x55dfbbc43ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:40.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.904+0000 7fa133e1d740 4 client.15078 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:21:40.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.904+0000 7fa133e1d740 1 -- 172.21.15.90:0/1447577216 --> [v2:172.21.15.90:6834/2396995634,v1:172.21.15.90:6835/2396995634] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dfbbb9e530 con 0x55dfbbc42660 2024-04-24T09:21:40.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.904+0000 7fa12a800640 1 --2- 172.21.15.90:0/1447577216 >> [v2:172.21.15.90:6834/2396995634,v1:172.21.15.90:6835/2396995634] conn(0x55dfbbc42660 0x55dfbbc43ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:40.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.905+0000 7fa12a800640 1 --2- 172.21.15.90:0/1447577216 >> [v2:172.21.15.90:6834/2396995634,v1:172.21.15.90:6835/2396995634] conn(0x55dfbbc42660 0x55dfbbc43ad0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=eb8ff5b5425c6915 server_cookie=48406bed59daca37 in_seq=0 out_seq=0 2024-04-24T09:21:40.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.905+0000 7fa10fe00640 10 client.15078 ms_handle_connect on v2:172.21.15.90:6834/2396995634 2024-04-24T09:21:40.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.905+0000 7fa10fe00640 1 -- 172.21.15.90:0/1447577216 <== mds.4 v2:172.21.15.90:6834/2396995634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dfbbb9e530 con 0x55dfbbc42660 2024-04-24T09:21:40.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.905+0000 7fa10fe00640 10 client.15078 handle_command_reply: tid=0 2024-04-24T09:21:40.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.905+0000 7fa133e1d740 1 client.15078 shutdown 2024-04-24T09:21:40.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.906+0000 7fa133e1d740 1 -- 172.21.15.90:0/1447577216 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dfbbc2bc50 msgr2=0x55dfbbc35b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:40.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.906+0000 7fa133e1d740 1 --2- 172.21.15.90:0/1447577216 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dfbbc2bc50 0x55dfbbc35b60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa12c19b890 tx=0x7fa114012b60 comp rx=0 tx=0).stop 2024-04-24T09:21:40.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.906+0000 7fa133e1d740 1 -- 172.21.15.90:0/1447577216 shutdown_connections 2024-04-24T09:21:40.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.906+0000 7fa133e1d740 1 --2- 172.21.15.90:0/1447577216 >> [v2:172.21.15.90:6834/2396995634,v1:172.21.15.90:6835/2396995634] conn(0x55dfbbc42660 0x55dfbbc43ad0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.906+0000 7fa133e1d740 1 --2- 172.21.15.90:0/1447577216 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfbbc30dc0 0x55dfbbc3cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.906+0000 7fa133e1d740 1 --2- 172.21.15.90:0/1447577216 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dfbbc2bc50 0x55dfbbc35b60 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.906+0000 7fa133e1d740 1 --2- 172.21.15.90:0/1447577216 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dfbbc2b260 0x55dfbbc35620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.906+0000 7fa10fe00640 0 client.15078 ms_handle_reset on v2:172.21.15.90:6834/2396995634 2024-04-24T09:21:40.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.906+0000 7fa133e1d740 1 -- 172.21.15.90:0/1447577216 >> 172.21.15.90:0/1447577216 conn(0x55dfbbc268d0 msgr2=0x55dfbbc25970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:40.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.906+0000 7fa133e1d740 1 -- 172.21.15.90:0/1447577216 shutdown_connections 2024-04-24T09:21:40.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.906+0000 7fa133e1d740 1 -- 172.21.15.90:0/1447577216 wait complete. 2024-04-24T09:21:40.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.906+0000 7fa133e1d740 20 client.15078 trim_cache size 0 max 16384 2024-04-24T09:21:40.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.926+0000 7fa133e1d740 1 Processor -- start 2024-04-24T09:21:40.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.926+0000 7fa133e1d740 1 -- start start 2024-04-24T09:21:40.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.926+0000 7fa133e1d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dfbbc2b260 0x55dfbbc34290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:40.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.926+0000 7fa133e1d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dfbbc2bc50 0x55dfbbc3cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:40.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.926+0000 7fa133e1d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfbbc30dc0 0x55dfbbc35230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:40.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.926+0000 7fa133e1d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55dfbbc41e10 con 0x55dfbbc2b260 2024-04-24T09:21:40.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.926+0000 7fa133e1d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55dfbbc424e0 con 0x55dfbbc2bc50 2024-04-24T09:21:40.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.926+0000 7fa133e1d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55dfbbc42bb0 con 0x55dfbbc30dc0 2024-04-24T09:21:40.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.927+0000 7fa12a800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dfbbc2b260 0x55dfbbc34290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:40.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.927+0000 7fa12a800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dfbbc2b260 0x55dfbbc34290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42306/0 (socket says 172.21.15.90:42306) 2024-04-24T09:21:40.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.927+0000 7fa12a800640 1 -- 172.21.15.90:0/523093715 learned_addr learned my addr 172.21.15.90:0/523093715 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:40.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.927+0000 7fa12a800640 1 --2- 172.21.15.90:0/523093715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfbbc30dc0 0x55dfbbc35230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:40.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.927+0000 7fa123e00640 1 --2- 172.21.15.90:0/523093715 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dfbbc2bc50 0x55dfbbc3cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:40.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.927+0000 7fa12a800640 1 -- 172.21.15.90:0/523093715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfbbc30dc0 msgr2=0x55dfbbc35230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:40.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.927+0000 7fa12a800640 1 --2- 172.21.15.90:0/523093715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfbbc30dc0 0x55dfbbc35230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.927+0000 7fa12a800640 1 -- 172.21.15.90:0/523093715 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dfbbc2bc50 msgr2=0x55dfbbc3cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:40.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.927+0000 7fa12a800640 1 --2- 172.21.15.90:0/523093715 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dfbbc2bc50 0x55dfbbc3cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.927+0000 7fa12a800640 1 -- 172.21.15.90:0/523093715 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfbbc302c0 con 0x55dfbbc2b260 2024-04-24T09:21:40.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.927+0000 7fa12a800640 1 --2- 172.21.15.90:0/523093715 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dfbbc2b260 0x55dfbbc34290 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa12402ae80 tx=0x7fa124005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd650abb73dc11d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:40.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.928+0000 7fa11aa00640 1 -- 172.21.15.90:0/523093715 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12404a040 con 0x55dfbbc2b260 2024-04-24T09:21:40.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.928+0000 7fa11aa00640 1 -- 172.21.15.90:0/523093715 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa124036070 con 0x55dfbbc2b260 2024-04-24T09:21:40.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.928+0000 7fa11aa00640 1 -- 172.21.15.90:0/523093715 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa124058320 con 0x55dfbbc2b260 2024-04-24T09:21:40.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.928+0000 7fa133e1d740 1 -- 172.21.15.90:0/523093715 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dfbbc2b260 msgr2=0x55dfbbc34290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:40.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.928+0000 7fa133e1d740 1 --2- 172.21.15.90:0/523093715 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dfbbc2b260 0x55dfbbc34290 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa12402ae80 tx=0x7fa124005d80 comp rx=0 tx=0).stop 2024-04-24T09:21:40.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.928+0000 7fa133e1d740 1 -- 172.21.15.90:0/523093715 shutdown_connections 2024-04-24T09:21:40.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.928+0000 7fa133e1d740 1 --2- 172.21.15.90:0/523093715 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfbbc30dc0 0x55dfbbc35230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.928+0000 7fa133e1d740 1 --2- 172.21.15.90:0/523093715 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dfbbc2bc50 0x55dfbbc3cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.928+0000 7fa133e1d740 1 --2- 172.21.15.90:0/523093715 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dfbbc2b260 0x55dfbbc34290 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.928+0000 7fa133e1d740 1 -- 172.21.15.90:0/523093715 >> 172.21.15.90:0/523093715 conn(0x55dfbbc268d0 msgr2=0x55dfbbc2d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:40.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.928+0000 7fa133e1d740 1 -- 172.21.15.90:0/523093715 shutdown_connections 2024-04-24T09:21:40.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.929+0000 7fa133e1d740 1 -- 172.21.15.90:0/523093715 wait complete. 2024-04-24T09:21:40.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.929+0000 7fa133e1d740 1 -- start start 2024-04-24T09:21:40.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.929+0000 7fa133e1d740 1 Processor -- start 2024-04-24T09:21:40.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.929+0000 7fa133e1d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfbbc2b260 0x55dfbbc3ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:40.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.929+0000 7fa133e1d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dfbbc2bc50 0x55dfbbc3b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:40.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.929+0000 7fa133e1d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dfbbc30dc0 0x55dfbbc3b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:40.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.929+0000 7fa133e1d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55dfbbc43950 con 0x55dfbbc30dc0 2024-04-24T09:21:40.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.929+0000 7fa133e1d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55dfbbc43280 con 0x55dfbbc2bc50 2024-04-24T09:21:40.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.930+0000 7fa133e1d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55dfbbc44020 con 0x55dfbbc2b260 2024-04-24T09:21:40.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.930+0000 7fa133e1d740 10 client.0 _ll_register_callbacks cb 0x7fa114009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:21:40.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.930+0000 7fa12a800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dfbbc30dc0 0x55dfbbc3b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:40.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.930+0000 7fa12a800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dfbbc30dc0 0x55dfbbc3b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42308/0 (socket says 172.21.15.90:42308) 2024-04-24T09:21:40.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.930+0000 7fa12a800640 1 -- 172.21.15.90:0/3279508085 learned_addr learned my addr 172.21.15.90:0/3279508085 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:40.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.930+0000 7fa123e00640 1 --2- 172.21.15.90:0/3279508085 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dfbbc2bc50 0x55dfbbc3b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:40.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.930+0000 7fa12a800640 1 --2- 172.21.15.90:0/3279508085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfbbc2b260 0x55dfbbc3ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:40.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.930+0000 7fa123e00640 1 -- 172.21.15.90:0/3279508085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfbbc2b260 msgr2=0x55dfbbc3ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:40.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.930+0000 7fa123e00640 1 --2- 172.21.15.90:0/3279508085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfbbc2b260 0x55dfbbc3ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.930+0000 7fa123e00640 1 -- 172.21.15.90:0/3279508085 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dfbbc30dc0 msgr2=0x55dfbbc3b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:40.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.930+0000 7fa123e00640 1 --2- 172.21.15.90:0/3279508085 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dfbbc30dc0 0x55dfbbc3b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.930+0000 7fa123e00640 1 -- 172.21.15.90:0/3279508085 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfbbc3c0d0 con 0x55dfbbc2bc50 2024-04-24T09:21:40.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.930+0000 7fa12a800640 1 --2- 172.21.15.90:0/3279508085 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dfbbc30dc0 0x55dfbbc3b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:21:40.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.931+0000 7fa123e00640 1 --2- 172.21.15.90:0/3279508085 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dfbbc2bc50 0x55dfbbc3b2e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa114000c00 tx=0x7fa114009e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8289340271486004 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:40.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.931+0000 7fa10fe00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-04-24T09:21:40.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.931+0000 7fa10fe00640 1 -- 172.21.15.90:0/3279508085 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa114035060 con 0x55dfbbc2bc50 2024-04-24T09:21:40.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.931+0000 7fa133e1d740 1 -- 172.21.15.90:0/3279508085 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa124036070 con 0x55dfbbc2bc50 2024-04-24T09:21:40.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.931+0000 7fa10fe00640 1 -- 172.21.15.90:0/3279508085 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fa11400f790 con 0x55dfbbc2bc50 2024-04-24T09:21:40.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.931+0000 7fa10fe00640 1 -- 172.21.15.90:0/3279508085 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa11402c070 con 0x55dfbbc2bc50 2024-04-24T09:21:40.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.931+0000 7fa133e1d740 10 client.25169 fetch_fsmap learned FSMap version 27 2024-04-24T09:21:40.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.931+0000 7fa133e1d740 1 -- 172.21.15.90:0/3279508085 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dfbbca8970 con 0x55dfbbc2bc50 2024-04-24T09:21:40.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.932+0000 7fa10fe00640 1 -- 172.21.15.90:0/3279508085 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa124036070 con 0x55dfbbc2bc50 2024-04-24T09:21:40.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.932+0000 7fa10fe00640 1 -- 172.21.15.90:0/3279508085 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa11403a070 con 0x55dfbbc2bc50 2024-04-24T09:21:40.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.932+0000 7fa133e1d740 10 client.25169 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T09:21:40.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.932+0000 7fa133e1d740 10 client.25169 resolve_mds: resolved name 'j' to daemon mds.j 2024-04-24T09:21:40.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.932+0000 7fa133e1d740 20 client.25169 populate_metadata read hostname 'smithi090' 2024-04-24T09:21:40.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.932+0000 7fa133e1d740 1 --2- 172.21.15.90:0/3279508085 >> [v2:172.21.15.90:6834/2396995634,v1:172.21.15.90:6835/2396995634] conn(0x55dfbbcad570 0x55dfbbcac280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:40.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.932+0000 7fa133e1d740 4 client.25169 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T09:21:40.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.932+0000 7fa133e1d740 1 -- 172.21.15.90:0/3279508085 --> [v2:172.21.15.90:6834/2396995634,v1:172.21.15.90:6835/2396995634] -- command(tid 0: {"prefix": "status"}) -- 0x55dfbbb983a0 con 0x55dfbbcad570 2024-04-24T09:21:40.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.932+0000 7fa12a800640 1 --2- 172.21.15.90:0/3279508085 >> [v2:172.21.15.90:6834/2396995634,v1:172.21.15.90:6835/2396995634] conn(0x55dfbbcad570 0x55dfbbcac280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:40.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.933+0000 7fa12a800640 1 --2- 172.21.15.90:0/3279508085 >> [v2:172.21.15.90:6834/2396995634,v1:172.21.15.90:6835/2396995634] conn(0x55dfbbcad570 0x55dfbbcac280 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4980a9b29608b717 server_cookie=45911033d73a5757 in_seq=0 out_seq=0 2024-04-24T09:21:40.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.933+0000 7fa10fe00640 10 client.25169 ms_handle_connect on v2:172.21.15.90:6834/2396995634 2024-04-24T09:21:40.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.933+0000 7fa10fe00640 1 -- 172.21.15.90:0/3279508085 <== mds.4 v2:172.21.15.90:6834/2396995634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55dfbbb983a0 con 0x55dfbbcad570 2024-04-24T09:21:40.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.933+0000 7fa10fe00640 10 client.25169 handle_command_reply: tid=0 2024-04-24T09:21:40.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.933+0000 7fa133e1d740 1 client.25169 shutdown 2024-04-24T09:21:40.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.933+0000 7fa133e1d740 1 -- 172.21.15.90:0/3279508085 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dfbbc2bc50 msgr2=0x55dfbbc3b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:40.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.933+0000 7fa133e1d740 1 --2- 172.21.15.90:0/3279508085 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dfbbc2bc50 0x55dfbbc3b2e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa114000c00 tx=0x7fa114009e50 comp rx=0 tx=0).stop 2024-04-24T09:21:40.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.933+0000 7fa133e1d740 1 -- 172.21.15.90:0/3279508085 shutdown_connections 2024-04-24T09:21:40.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.933+0000 7fa133e1d740 1 --2- 172.21.15.90:0/3279508085 >> [v2:172.21.15.90:6834/2396995634,v1:172.21.15.90:6835/2396995634] conn(0x55dfbbcad570 0x55dfbbcac280 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.933+0000 7fa133e1d740 1 --2- 172.21.15.90:0/3279508085 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dfbbc30dc0 0x55dfbbc3b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.933+0000 7fa133e1d740 1 --2- 172.21.15.90:0/3279508085 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dfbbc2bc50 0x55dfbbc3b2e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.933+0000 7fa133e1d740 1 --2- 172.21.15.90:0/3279508085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfbbc2b260 0x55dfbbc3ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.933+0000 7fa10fe00640 0 client.25169 ms_handle_reset on v2:172.21.15.90:6834/2396995634 2024-04-24T09:21:40.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.933+0000 7fa133e1d740 1 -- 172.21.15.90:0/3279508085 >> 172.21.15.90:0/3279508085 conn(0x55dfbbc268d0 msgr2=0x55dfbbc35230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:40.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.933+0000 7fa133e1d740 1 -- 172.21.15.90:0/3279508085 shutdown_connections 2024-04-24T09:21:40.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.933+0000 7fa133e1d740 1 -- 172.21.15.90:0/3279508085 wait complete. 2024-04-24T09:21:40.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.933+0000 7fa133e1d740 20 client.25169 trim_cache size 0 max 16384 2024-04-24T09:21:40.938 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T09:21:40.938 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "30ee9be2-021a-11ef-bc93-c7b262605968", 2024-04-24T09:21:40.938 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 4, 2024-04-24T09:21:40.938 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 24446, 2024-04-24T09:21:40.938 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-04-24T09:21:40.939 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-04-24T09:21:40.939 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-04-24T09:21:40.939 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 144.677455931, 2024-04-24T09:21:40.939 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 27, 2024-04-24T09:21:40.939 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 88, 2024-04-24T09:21:40.939 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 86, 2024-04-24T09:21:40.939 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 187.20295546899999 2024-04-24T09:21:40.939 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T09:21:40.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.934+0000 7fa12b200640 1 -- 172.21.15.90:0/185718971 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa1100a3bb0 msgr2=0x7fa1100a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:40.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.934+0000 7fa12b200640 1 --2- 172.21.15.90:0/185718971 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa1100a3bb0 0x7fa1100a6070 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa11c00c6a0 tx=0x7fa11c0d6530 comp rx=0 tx=0).stop 2024-04-24T09:21:40.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.934+0000 7fa12b200640 1 -- 172.21.15.90:0/185718971 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa12c101f10 msgr2=0x7fa12c19a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:40.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.934+0000 7fa12b200640 1 --2- 172.21.15.90:0/185718971 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa12c101f10 0x7fa12c19a380 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa11c00e9f0 tx=0x7fa11c00eeb0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.934+0000 7fa12b200640 1 -- 172.21.15.90:0/185718971 shutdown_connections 2024-04-24T09:21:40.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.934+0000 7fa12b200640 1 --2- 172.21.15.90:0/185718971 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fa1100a3bb0 0x7fa1100a6070 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.934+0000 7fa12b200640 1 --2- 172.21.15.90:0/185718971 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa12c103a20 0x7fa12c1a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.934+0000 7fa12b200640 1 --2- 172.21.15.90:0/185718971 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa12c103060 0x7fa12c19a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.934+0000 7fa12b200640 1 --2- 172.21.15.90:0/185718971 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa12c101f10 0x7fa12c19a380 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:40.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.934+0000 7fa12b200640 1 -- 172.21.15.90:0/185718971 >> 172.21.15.90:0/185718971 conn(0x7fa12c0fd4a0 msgr2=0x7fa12c074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:40.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.934+0000 7fa12b200640 1 -- 172.21.15.90:0/185718971 shutdown_connections 2024-04-24T09:21:40.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:40.934+0000 7fa12b200640 1 -- 172.21.15.90:0/185718971 wait complete. 2024-04-24T09:21:40.950 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:40 smithi090 ceph-mon[27825]: pgmap v399: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-24T09:21:40.950 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:40 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/3299397633' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:21:40.950 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:40 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/340635996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:21:40.950 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:40 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/2618229429' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:21:40.954 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-24T09:21:41.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:40 smithi149 ceph-mon[32592]: pgmap v399: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-24T09:21:41.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:40 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/3299397633' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:21:41.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:40 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/340635996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:21:41.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:40 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/2618229429' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:21:41.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.152+0000 7f3745400640 1 -- 172.21.15.90:0/1894959266 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3740105330 msgr2=0x7f37401057b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:41.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.152+0000 7f3745400640 1 --2- 172.21.15.90:0/1894959266 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3740105330 0x7f37401057b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3728009900 tx=0x7f372802a3f0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.152+0000 7f3745400640 1 -- 172.21.15.90:0/1894959266 shutdown_connections 2024-04-24T09:21:41.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.152+0000 7f3745400640 1 --2- 172.21.15.90:0/1894959266 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3740105cf0 0x7f374010c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.152+0000 7f3745400640 1 --2- 172.21.15.90:0/1894959266 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3740105330 0x7f37401057b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.152+0000 7f3745400640 1 --2- 172.21.15.90:0/1894959266 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3740104140 0x7f3740104560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.152+0000 7f3745400640 1 -- 172.21.15.90:0/1894959266 >> 172.21.15.90:0/1894959266 conn(0x7f37400ff6b0 msgr2=0x7f3740101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:41.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.153+0000 7f3745400640 1 -- 172.21.15.90:0/1894959266 shutdown_connections 2024-04-24T09:21:41.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.153+0000 7f3745400640 1 -- 172.21.15.90:0/1894959266 wait complete. 2024-04-24T09:21:41.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.154+0000 7f3745400640 1 Processor -- start 2024-04-24T09:21:41.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.154+0000 7f3745400640 1 -- start start 2024-04-24T09:21:41.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.154+0000 7f3745400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3740104140 0x7f374019a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.154+0000 7f3745400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3740105330 0x7f374019a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.154+0000 7f3745400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3740105cf0 0x7f37401a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.154+0000 7f3745400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f374010fe20 con 0x7f3740105330 2024-04-24T09:21:41.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.154+0000 7f3745400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f374010fca0 con 0x7f3740104140 2024-04-24T09:21:41.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.154+0000 7f3745400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f374010ffa0 con 0x7f3740105cf0 2024-04-24T09:21:41.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.154+0000 7f373fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3740104140 0x7f374019a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.154+0000 7f3744a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3740105cf0 0x7f37401a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.154+0000 7f3744a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3740105cf0 0x7f37401a19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:54474/0 (socket says 172.21.15.90:54474) 2024-04-24T09:21:41.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.154+0000 7f3737400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3740105330 0x7f374019a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.154+0000 7f3744a00640 1 -- 172.21.15.90:0/999820522 learned_addr learned my addr 172.21.15.90:0/999820522 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:41.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.154+0000 7f373fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3740104140 0x7f374019a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.90:55960/0 (socket says 172.21.15.90:55960) 2024-04-24T09:21:41.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.155+0000 7f373fe00640 1 -- 172.21.15.90:0/999820522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3740105cf0 msgr2=0x7f37401a19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:41.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.155+0000 7f373fe00640 1 --2- 172.21.15.90:0/999820522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3740105cf0 0x7f37401a19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.155+0000 7f373fe00640 1 -- 172.21.15.90:0/999820522 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3740105330 msgr2=0x7f374019a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:41.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.155+0000 7f373fe00640 1 --2- 172.21.15.90:0/999820522 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3740105330 0x7f374019a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.155+0000 7f373fe00640 1 -- 172.21.15.90:0/999820522 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37401a2010 con 0x7f3740104140 2024-04-24T09:21:41.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.155+0000 7f373fe00640 1 --2- 172.21.15.90:0/999820522 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3740104140 0x7f374019a3f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f373000d6a0 tx=0x7f373000db60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:41.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.155+0000 7f373d600640 1 -- 172.21.15.90:0/999820522 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3730017070 con 0x7f3740104140 2024-04-24T09:21:41.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.155+0000 7f373d600640 1 -- 172.21.15.90:0/999820522 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3730009d00 con 0x7f3740104140 2024-04-24T09:21:41.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.155+0000 7f373d600640 1 -- 172.21.15.90:0/999820522 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3730003c60 con 0x7f3740104140 2024-04-24T09:21:41.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.155+0000 7f3745400640 1 -- 172.21.15.90:0/999820522 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37401a2300 con 0x7f3740104140 2024-04-24T09:21:41.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.156+0000 7f3745400640 1 -- 172.21.15.90:0/999820522 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37401a2820 con 0x7f3740104140 2024-04-24T09:21:41.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.159+0000 7f37499d2740 1 Processor -- start 2024-04-24T09:21:41.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.159+0000 7f373d600640 1 -- 172.21.15.90:0/999820522 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3730009820 con 0x7f3740104140 2024-04-24T09:21:41.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.159+0000 7f37499d2740 1 -- start start 2024-04-24T09:21:41.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.159+0000 7f37499d2740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a3d2397260 0x55a3d2397680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.159+0000 7f37499d2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3d2397c50 0x55a3d239c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.159+0000 7f37499d2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3d239cdc0 0x55a3d239f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.159+0000 7f37499d2740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55a3d22fcb10 con 0x55a3d2397260 2024-04-24T09:21:41.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.159+0000 7f37499d2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55a3d22a32b0 con 0x55a3d239cdc0 2024-04-24T09:21:41.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.159+0000 7f37499d2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a3d22f8ad0 con 0x55a3d2397c50 2024-04-24T09:21:41.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.160+0000 7f3744a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a3d2397260 0x55a3d2397680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.160+0000 7f373fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3d2397c50 0x55a3d239c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.160+0000 7f3737400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3d239cdc0 0x55a3d239f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.160+0000 7f3744a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a3d2397260 0x55a3d2397680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42336/0 (socket says 172.21.15.90:42336) 2024-04-24T09:21:41.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.160+0000 7f3737400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3d239cdc0 0x55a3d239f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.90:55966/0 (socket says 172.21.15.90:55966) 2024-04-24T09:21:41.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.160+0000 7f3744a00640 1 -- 172.21.15.90:0/961996832 learned_addr learned my addr 172.21.15.90:0/961996832 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:41.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.160+0000 7f3744a00640 1 -- 172.21.15.90:0/961996832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3d2397c50 msgr2=0x55a3d239c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:41.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.160+0000 7f3744a00640 1 --2- 172.21.15.90:0/961996832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3d2397c50 0x55a3d239c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.160+0000 7f3744a00640 1 -- 172.21.15.90:0/961996832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3d239cdc0 msgr2=0x55a3d239f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:41.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.160+0000 7f3744a00640 1 --2- 172.21.15.90:0/961996832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3d239cdc0 0x55a3d239f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.160+0000 7f3744a00640 1 -- 172.21.15.90:0/961996832 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3d200f2d0 con 0x55a3d2397260 2024-04-24T09:21:41.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.160+0000 7f373fe00640 1 --2- 172.21.15.90:0/961996832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3d2397c50 0x55a3d239c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:21:41.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.160+0000 7f3744a00640 1 --2- 172.21.15.90:0/961996832 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a3d2397260 0x55a3d2397680 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3738011570 tx=0x7f3738011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7e930bbfd5c1ac6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:41.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.161+0000 7f3735200640 1 -- 172.21.15.90:0/961996832 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3738004b20 con 0x55a3d2397260 2024-04-24T09:21:41.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.161+0000 7f3735200640 1 -- 172.21.15.90:0/961996832 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3738004cc0 con 0x55a3d2397260 2024-04-24T09:21:41.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.161+0000 7f373d600640 1 --2- 172.21.15.90:0/999820522 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f37200a3bb0 0x7f37200a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.161+0000 7f3735200640 1 -- 172.21.15.90:0/961996832 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3738005560 con 0x55a3d2397260 2024-04-24T09:21:41.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.161+0000 7f373d600640 1 -- 172.21.15.90:0/999820522 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f37300cfd20 con 0x7f3740104140 2024-04-24T09:21:41.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.161+0000 7f373fe00640 1 --2- 172.21.15.90:0/999820522 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f37200a3bb0 0x7f37200a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.161+0000 7f37499d2740 1 -- 172.21.15.90:0/961996832 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a3d2397260 msgr2=0x55a3d2397680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:41.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.161+0000 7f37499d2740 1 --2- 172.21.15.90:0/961996832 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a3d2397260 0x55a3d2397680 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3738011570 tx=0x7f3738011a30 comp rx=0 tx=0).stop 2024-04-24T09:21:41.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.161+0000 7f37499d2740 1 -- 172.21.15.90:0/961996832 shutdown_connections 2024-04-24T09:21:41.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.161+0000 7f37499d2740 1 --2- 172.21.15.90:0/961996832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3d239cdc0 0x55a3d239f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.161+0000 7f37499d2740 1 --2- 172.21.15.90:0/961996832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3d2397c50 0x55a3d239c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.161+0000 7f37499d2740 1 --2- 172.21.15.90:0/961996832 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a3d2397260 0x55a3d2397680 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.161+0000 7f37499d2740 1 -- 172.21.15.90:0/961996832 >> 172.21.15.90:0/961996832 conn(0x55a3d23928d0 msgr2=0x55a3d2392cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:41.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.162+0000 7f37499d2740 1 -- 172.21.15.90:0/961996832 shutdown_connections 2024-04-24T09:21:41.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.162+0000 7f37499d2740 1 -- 172.21.15.90:0/961996832 wait complete. 2024-04-24T09:21:41.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.162+0000 7f373fe00640 1 --2- 172.21.15.90:0/999820522 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f37200a3bb0 0x7f37200a6070 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f373009c000 tx=0x7f37300d3760 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:41.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.162+0000 7f37499d2740 1 -- start start 2024-04-24T09:21:41.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.162+0000 7f37499d2740 1 Processor -- start 2024-04-24T09:21:41.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.163+0000 7f37499d2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3d2397260 0x55a3d23a1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.163+0000 7f37499d2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3d2397c50 0x55a3d23a1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.163+0000 7f37499d2740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a3d239cdc0 0x55a3d23a8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.163+0000 7f37499d2740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55a3d239fd70 con 0x55a3d239cdc0 2024-04-24T09:21:41.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.163+0000 7f37499d2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55a3d22b6590 con 0x55a3d2397260 2024-04-24T09:21:41.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.163+0000 7f37499d2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a3d239fef0 con 0x55a3d2397c50 2024-04-24T09:21:41.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.163+0000 7f37499d2740 10 client.0 _ll_register_callbacks cb 0x55a3d22993a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:21:41.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.163+0000 7f3744a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3d2397260 0x55a3d23a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.163+0000 7f3744a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a3d239cdc0 0x55a3d23a8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.163+0000 7f3744a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a3d239cdc0 0x55a3d23a8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42342/0 (socket says 172.21.15.90:42342) 2024-04-24T09:21:41.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.163+0000 7f3744a00640 1 -- 172.21.15.90:0/3565530065 learned_addr learned my addr 172.21.15.90:0/3565530065 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:41.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.163+0000 7f3737400640 1 --2- 172.21.15.90:0/3565530065 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3d2397c50 0x55a3d23a1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.163+0000 7f3744a00640 1 -- 172.21.15.90:0/3565530065 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3d2397c50 msgr2=0x55a3d23a1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:41.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.163+0000 7f3744a00640 1 --2- 172.21.15.90:0/3565530065 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3d2397c50 0x55a3d23a1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.163+0000 7f3744a00640 1 -- 172.21.15.90:0/3565530065 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3d2397260 msgr2=0x55a3d23a1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:41.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.163+0000 7f3744a00640 1 --2- 172.21.15.90:0/3565530065 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3d2397260 0x55a3d23a1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.163+0000 7f3744a00640 1 -- 172.21.15.90:0/3565530065 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3d20d96b0 con 0x55a3d239cdc0 2024-04-24T09:21:41.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.164+0000 7f3744a00640 1 --2- 172.21.15.90:0/3565530065 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a3d239cdc0 0x55a3d23a8af0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3738004f90 tx=0x7f3738009cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e30f761c8bec74c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:41.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.164+0000 7f372d600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-24T09:21:41.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.164+0000 7f372d600640 1 -- 172.21.15.90:0/3565530065 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37380058b0 con 0x55a3d239cdc0 2024-04-24T09:21:41.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.164+0000 7f372d600640 1 -- 172.21.15.90:0/3565530065 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f373800f5c0 con 0x55a3d239cdc0 2024-04-24T09:21:41.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.164+0000 7f37499d2740 1 -- 172.21.15.90:0/3565530065 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3738004cc0 con 0x55a3d239cdc0 2024-04-24T09:21:41.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.164+0000 7f372d600640 1 -- 172.21.15.90:0/3565530065 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f373800f7c0 con 0x55a3d239cdc0 2024-04-24T09:21:41.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.164+0000 7f37499d2740 10 client.15096 fetch_fsmap learned FSMap version 27 2024-04-24T09:21:41.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.164+0000 7f37499d2740 1 -- 172.21.15.90:0/3565530065 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a3d23abe60 con 0x55a3d239cdc0 2024-04-24T09:21:41.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.165+0000 7f372d600640 1 -- 172.21.15.90:0/3565530065 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3738004cc0 con 0x55a3d239cdc0 2024-04-24T09:21:41.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.165+0000 7f372d600640 1 -- 172.21.15.90:0/3565530065 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3738030030 con 0x55a3d239cdc0 2024-04-24T09:21:41.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.165+0000 7f37499d2740 10 client.15096 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T09:21:41.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.165+0000 7f37499d2740 10 client.15096 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-24T09:21:41.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.165+0000 7f37499d2740 20 client.15096 populate_metadata read hostname 'smithi090' 2024-04-24T09:21:41.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.165+0000 7f37499d2740 1 --2- 172.21.15.90:0/3565530065 >> [v2:172.21.15.149:6836/2250310587,v1:172.21.15.149:6837/2250310587] conn(0x55a3d23ae2b0 0x55a3d23af720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.165+0000 7f37499d2740 4 client.15096 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:21:41.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.165+0000 7f37499d2740 1 -- 172.21.15.90:0/3565530065 --> [v2:172.21.15.149:6836/2250310587,v1:172.21.15.149:6837/2250310587] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3d230a530 con 0x55a3d23ae2b0 2024-04-24T09:21:41.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.165+0000 7f3737400640 1 --2- 172.21.15.90:0/3565530065 >> [v2:172.21.15.149:6836/2250310587,v1:172.21.15.149:6837/2250310587] conn(0x55a3d23ae2b0 0x55a3d23af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.166+0000 7f3737400640 1 --2- 172.21.15.90:0/3565530065 >> [v2:172.21.15.149:6836/2250310587,v1:172.21.15.149:6837/2250310587] conn(0x55a3d23ae2b0 0x55a3d23af720 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=88ec0b1addec5ecc server_cookie=273035d97f7e88de in_seq=0 out_seq=0 2024-04-24T09:21:41.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.166+0000 7f372d600640 10 client.15096 ms_handle_connect on v2:172.21.15.149:6836/2250310587 2024-04-24T09:21:41.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.167+0000 7f372d600640 1 -- 172.21.15.90:0/3565530065 <== mds.3 v2:172.21.15.149:6836/2250310587 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a3d230a530 con 0x55a3d23ae2b0 2024-04-24T09:21:41.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.167+0000 7f372d600640 10 client.15096 handle_command_reply: tid=0 2024-04-24T09:21:41.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.167+0000 7f37499d2740 1 client.15096 shutdown 2024-04-24T09:21:41.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.167+0000 7f37499d2740 1 -- 172.21.15.90:0/3565530065 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a3d239cdc0 msgr2=0x55a3d23a8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:41.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.167+0000 7f37499d2740 1 --2- 172.21.15.90:0/3565530065 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a3d239cdc0 0x55a3d23a8af0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3738004f90 tx=0x7f3738009cb0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.167+0000 7f37499d2740 1 -- 172.21.15.90:0/3565530065 shutdown_connections 2024-04-24T09:21:41.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.167+0000 7f37499d2740 1 --2- 172.21.15.90:0/3565530065 >> [v2:172.21.15.149:6836/2250310587,v1:172.21.15.149:6837/2250310587] conn(0x55a3d23ae2b0 0x55a3d23af720 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.167+0000 7f37499d2740 1 --2- 172.21.15.90:0/3565530065 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a3d239cdc0 0x55a3d23a8af0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.167+0000 7f37499d2740 1 --2- 172.21.15.90:0/3565530065 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3d2397c50 0x55a3d23a1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.167+0000 7f372d600640 0 client.15096 ms_handle_reset on v2:172.21.15.149:6836/2250310587 2024-04-24T09:21:41.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.167+0000 7f37499d2740 1 --2- 172.21.15.90:0/3565530065 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3d2397260 0x55a3d23a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.167+0000 7f37499d2740 1 -- 172.21.15.90:0/3565530065 >> 172.21.15.90:0/3565530065 conn(0x55a3d23928d0 msgr2=0x55a3d2391970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:41.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.168+0000 7f37499d2740 1 -- 172.21.15.90:0/3565530065 shutdown_connections 2024-04-24T09:21:41.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.168+0000 7f37499d2740 1 -- 172.21.15.90:0/3565530065 wait complete. 2024-04-24T09:21:41.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.168+0000 7f37499d2740 20 client.15096 trim_cache size 0 max 16384 2024-04-24T09:21:41.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:40 smithi114 ceph-mon[32501]: pgmap v399: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-24T09:21:41.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:40 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/3299397633' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:21:41.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:40 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/340635996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:21:41.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:40 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/2618229429' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:21:41.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.187+0000 7f37499d2740 1 Processor -- start 2024-04-24T09:21:41.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.187+0000 7f37499d2740 1 -- start start 2024-04-24T09:21:41.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.188+0000 7f37499d2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3d2397260 0x55a3d2392cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.188+0000 7f37499d2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3d2397c50 0x55a3d23a8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.188+0000 7f37499d2740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a3d239cdc0 0x55a3d23a1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.188+0000 7f37499d2740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55a3d23ada60 con 0x55a3d239cdc0 2024-04-24T09:21:41.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.188+0000 7f37499d2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55a3d23ae130 con 0x55a3d2397260 2024-04-24T09:21:41.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.188+0000 7f37499d2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a3d23ae800 con 0x55a3d2397c50 2024-04-24T09:21:41.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.188+0000 7f3744a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a3d239cdc0 0x55a3d23a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.188+0000 7f3744a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a3d239cdc0 0x55a3d23a1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42352/0 (socket says 172.21.15.90:42352) 2024-04-24T09:21:41.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.188+0000 7f3744a00640 1 -- 172.21.15.90:0/2729807151 learned_addr learned my addr 172.21.15.90:0/2729807151 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:41.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.188+0000 7f3744a00640 1 --2- 172.21.15.90:0/2729807151 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3d2397260 0x55a3d2392cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.188+0000 7f3737400640 1 --2- 172.21.15.90:0/2729807151 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3d2397c50 0x55a3d23a8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.188+0000 7f3744a00640 1 -- 172.21.15.90:0/2729807151 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3d2397c50 msgr2=0x55a3d23a8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:41.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.188+0000 7f3744a00640 1 --2- 172.21.15.90:0/2729807151 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3d2397c50 0x55a3d23a8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.188+0000 7f3744a00640 1 -- 172.21.15.90:0/2729807151 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3d2397260 msgr2=0x55a3d2392cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:41.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.188+0000 7f3744a00640 1 --2- 172.21.15.90:0/2729807151 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3d2397260 0x55a3d2392cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.188+0000 7f3744a00640 1 -- 172.21.15.90:0/2729807151 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3d23998a0 con 0x55a3d239cdc0 2024-04-24T09:21:41.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.188+0000 7f3737400640 1 --2- 172.21.15.90:0/2729807151 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3d2397c50 0x55a3d23a8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:21:41.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.189+0000 7f3744a00640 1 --2- 172.21.15.90:0/2729807151 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a3d239cdc0 0x55a3d23a1180 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3738009eb0 tx=0x7f3738011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f5254be6f2bc569 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:41.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.189+0000 7f372fe00640 1 -- 172.21.15.90:0/2729807151 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37380058b0 con 0x55a3d239cdc0 2024-04-24T09:21:41.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.189+0000 7f372fe00640 1 -- 172.21.15.90:0/2729807151 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3738005a50 con 0x55a3d239cdc0 2024-04-24T09:21:41.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.189+0000 7f372fe00640 1 -- 172.21.15.90:0/2729807151 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3738028be0 con 0x55a3d239cdc0 2024-04-24T09:21:41.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.189+0000 7f37499d2740 1 -- 172.21.15.90:0/2729807151 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a3d239cdc0 msgr2=0x55a3d23a1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:41.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.189+0000 7f37499d2740 1 --2- 172.21.15.90:0/2729807151 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a3d239cdc0 0x55a3d23a1180 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3738009eb0 tx=0x7f3738011a30 comp rx=0 tx=0).stop 2024-04-24T09:21:41.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.190+0000 7f37499d2740 1 -- 172.21.15.90:0/2729807151 shutdown_connections 2024-04-24T09:21:41.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.190+0000 7f37499d2740 1 --2- 172.21.15.90:0/2729807151 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a3d239cdc0 0x55a3d23a1180 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.190+0000 7f37499d2740 1 --2- 172.21.15.90:0/2729807151 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3d2397c50 0x55a3d23a8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.190+0000 7f37499d2740 1 --2- 172.21.15.90:0/2729807151 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3d2397260 0x55a3d2392cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.190+0000 7f37499d2740 1 -- 172.21.15.90:0/2729807151 >> 172.21.15.90:0/2729807151 conn(0x55a3d23928d0 msgr2=0x55a3d23ad2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:41.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.190+0000 7f37499d2740 1 -- 172.21.15.90:0/2729807151 shutdown_connections 2024-04-24T09:21:41.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.190+0000 7f37499d2740 1 -- 172.21.15.90:0/2729807151 wait complete. 2024-04-24T09:21:41.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.190+0000 7f37499d2740 1 -- start start 2024-04-24T09:21:41.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.190+0000 7f37499d2740 1 Processor -- start 2024-04-24T09:21:41.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.191+0000 7f37499d2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3d2397260 0x55a3d23a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.191+0000 7f37499d2740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a3d2397c50 0x55a3d23a5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.191+0000 7f37499d2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3d239cdc0 0x55a3d23a7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.191+0000 7f37499d2740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55a3d23af5a0 con 0x55a3d2397c50 2024-04-24T09:21:41.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.191+0000 7f37499d2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55a3d23aeed0 con 0x55a3d239cdc0 2024-04-24T09:21:41.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.191+0000 7f37499d2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a3d23afc70 con 0x55a3d2397260 2024-04-24T09:21:41.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.191+0000 7f37499d2740 10 client.0 _ll_register_callbacks cb 0x7f3738011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:21:41.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.191+0000 7f3737400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a3d2397c50 0x55a3d23a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.191+0000 7f3744a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3d2397260 0x55a3d23a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.191+0000 7f3744a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3d2397260 0x55a3d23a4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:54520/0 (socket says 172.21.15.90:54520) 2024-04-24T09:21:41.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.191+0000 7f3744a00640 1 -- 172.21.15.90:0/704436457 learned_addr learned my addr 172.21.15.90:0/704436457 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:41.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.191+0000 7f3744a00640 1 --2- 172.21.15.90:0/704436457 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3d239cdc0 0x55a3d23a7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.191+0000 7f3737400640 1 -- 172.21.15.90:0/704436457 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3d2397260 msgr2=0x55a3d23a4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:41.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.191+0000 7f3737400640 1 --2- 172.21.15.90:0/704436457 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3d2397260 0x55a3d23a4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.191+0000 7f3737400640 1 -- 172.21.15.90:0/704436457 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3d239cdc0 msgr2=0x55a3d23a7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:41.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.191+0000 7f3737400640 1 --2- 172.21.15.90:0/704436457 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3d239cdc0 0x55a3d23a7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.191+0000 7f3737400640 1 -- 172.21.15.90:0/704436457 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3d23a83e0 con 0x55a3d2397c50 2024-04-24T09:21:41.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.192+0000 7f3737400640 1 --2- 172.21.15.90:0/704436457 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a3d2397c50 0x55a3d23a5180 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f372802a910 tx=0x7f3728004550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3092a05a12711d23 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:41.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.192+0000 7f372d600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-24T09:21:41.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.192+0000 7f372d600640 1 -- 172.21.15.90:0/704436457 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3728035040 con 0x55a3d2397c50 2024-04-24T09:21:41.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.192+0000 7f372d600640 1 -- 172.21.15.90:0/704436457 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f37280050c0 con 0x55a3d2397c50 2024-04-24T09:21:41.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.192+0000 7f37499d2740 1 -- 172.21.15.90:0/704436457 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a3d23043a0 con 0x55a3d2397c50 2024-04-24T09:21:41.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.192+0000 7f372d600640 1 -- 172.21.15.90:0/704436457 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3728055050 con 0x55a3d2397c50 2024-04-24T09:21:41.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.192+0000 7f37499d2740 10 client.15102 fetch_fsmap learned FSMap version 27 2024-04-24T09:21:41.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.192+0000 7f37499d2740 1 -- 172.21.15.90:0/704436457 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a3d24148d0 con 0x55a3d2397c50 2024-04-24T09:21:41.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.193+0000 7f372d600640 1 -- 172.21.15.90:0/704436457 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a3d23043a0 con 0x55a3d2397c50 2024-04-24T09:21:41.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.193+0000 7f372d600640 1 -- 172.21.15.90:0/704436457 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f37280047e0 con 0x55a3d2397c50 2024-04-24T09:21:41.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.193+0000 7f37499d2740 10 client.15102 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T09:21:41.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.193+0000 7f37499d2740 10 client.15102 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-24T09:21:41.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.193+0000 7f37499d2740 20 client.15102 populate_metadata read hostname 'smithi090' 2024-04-24T09:21:41.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.193+0000 7f37499d2740 1 --2- 172.21.15.90:0/704436457 >> [v2:172.21.15.149:6836/2250310587,v1:172.21.15.149:6837/2250310587] conn(0x55a3d24194c0 0x55a3d24181d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.193+0000 7f37499d2740 4 client.15102 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T09:21:41.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.193+0000 7f37499d2740 1 -- 172.21.15.90:0/704436457 --> [v2:172.21.15.149:6836/2250310587,v1:172.21.15.149:6837/2250310587] -- command(tid 0: {"prefix": "status"}) -- 0x55a3d2280860 con 0x55a3d24194c0 2024-04-24T09:21:41.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.193+0000 7f3744a00640 1 --2- 172.21.15.90:0/704436457 >> [v2:172.21.15.149:6836/2250310587,v1:172.21.15.149:6837/2250310587] conn(0x55a3d24194c0 0x55a3d24181d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.194+0000 7f3744a00640 1 --2- 172.21.15.90:0/704436457 >> [v2:172.21.15.149:6836/2250310587,v1:172.21.15.149:6837/2250310587] conn(0x55a3d24194c0 0x55a3d24181d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=659ee8f6b1272b3c server_cookie=28da58718a605283 in_seq=0 out_seq=0 2024-04-24T09:21:41.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.194+0000 7f372d600640 10 client.15102 ms_handle_connect on v2:172.21.15.149:6836/2250310587 2024-04-24T09:21:41.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.194+0000 7f372d600640 1 -- 172.21.15.90:0/704436457 <== mds.3 v2:172.21.15.149:6836/2250310587 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55a3d2280860 con 0x55a3d24194c0 2024-04-24T09:21:41.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.194+0000 7f372d600640 10 client.15102 handle_command_reply: tid=0 2024-04-24T09:21:41.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.194+0000 7f37499d2740 1 client.15102 shutdown 2024-04-24T09:21:41.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.194+0000 7f37499d2740 1 -- 172.21.15.90:0/704436457 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a3d2397c50 msgr2=0x55a3d23a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:41.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.194+0000 7f37499d2740 1 --2- 172.21.15.90:0/704436457 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a3d2397c50 0x55a3d23a5180 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f372802a910 tx=0x7f3728004550 comp rx=0 tx=0).stop 2024-04-24T09:21:41.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.194+0000 7f37499d2740 1 -- 172.21.15.90:0/704436457 shutdown_connections 2024-04-24T09:21:41.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.194+0000 7f37499d2740 1 --2- 172.21.15.90:0/704436457 >> [v2:172.21.15.149:6836/2250310587,v1:172.21.15.149:6837/2250310587] conn(0x55a3d24194c0 0x55a3d24181d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.194+0000 7f37499d2740 1 --2- 172.21.15.90:0/704436457 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a3d239cdc0 0x55a3d23a7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.194+0000 7f37499d2740 1 --2- 172.21.15.90:0/704436457 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a3d2397c50 0x55a3d23a5180 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.194+0000 7f37499d2740 1 --2- 172.21.15.90:0/704436457 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3d2397260 0x55a3d23a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.194+0000 7f372d600640 0 client.15102 ms_handle_reset on v2:172.21.15.149:6836/2250310587 2024-04-24T09:21:41.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.194+0000 7f37499d2740 1 -- 172.21.15.90:0/704436457 >> 172.21.15.90:0/704436457 conn(0x55a3d23928d0 msgr2=0x55a3d2392cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:41.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.194+0000 7f37499d2740 1 -- 172.21.15.90:0/704436457 shutdown_connections 2024-04-24T09:21:41.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.194+0000 7f37499d2740 1 -- 172.21.15.90:0/704436457 wait complete. 2024-04-24T09:21:41.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.195+0000 7f37499d2740 20 client.15102 trim_cache size 0 max 16384 2024-04-24T09:21:41.199 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T09:21:41.199 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "30ee9be2-021a-11ef-bc93-c7b262605968", 2024-04-24T09:21:41.199 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 3, 2024-04-24T09:21:41.199 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 24526, 2024-04-24T09:21:41.199 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-04-24T09:21:41.199 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-04-24T09:21:41.199 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-04-24T09:21:41.199 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 145.900105596, 2024-04-24T09:21:41.199 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 27, 2024-04-24T09:21:41.199 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 88, 2024-04-24T09:21:41.199 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 86, 2024-04-24T09:21:41.200 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 174.436977963 2024-04-24T09:21:41.200 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T09:21:41.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.195+0000 7f3745400640 1 -- 172.21.15.90:0/999820522 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f37200a3bb0 msgr2=0x7f37200a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:41.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.195+0000 7f3745400640 1 --2- 172.21.15.90:0/999820522 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f37200a3bb0 0x7f37200a6070 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f373009c000 tx=0x7f37300d3760 comp rx=0 tx=0).stop 2024-04-24T09:21:41.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.195+0000 7f3745400640 1 -- 172.21.15.90:0/999820522 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3740104140 msgr2=0x7f374019a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:41.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.195+0000 7f3745400640 1 --2- 172.21.15.90:0/999820522 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3740104140 0x7f374019a3f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f373000d6a0 tx=0x7f373000db60 comp rx=0 tx=0).stop 2024-04-24T09:21:41.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.195+0000 7f3745400640 1 -- 172.21.15.90:0/999820522 shutdown_connections 2024-04-24T09:21:41.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.195+0000 7f3745400640 1 --2- 172.21.15.90:0/999820522 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f37200a3bb0 0x7f37200a6070 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.195+0000 7f3745400640 1 --2- 172.21.15.90:0/999820522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3740105cf0 0x7f37401a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.195+0000 7f3745400640 1 --2- 172.21.15.90:0/999820522 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3740105330 0x7f374019a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.195+0000 7f3745400640 1 --2- 172.21.15.90:0/999820522 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3740104140 0x7f374019a3f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.195+0000 7f3745400640 1 -- 172.21.15.90:0/999820522 >> 172.21.15.90:0/999820522 conn(0x7f37400ff6b0 msgr2=0x7f3740108630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:41.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.195+0000 7f3745400640 1 -- 172.21.15.90:0/999820522 shutdown_connections 2024-04-24T09:21:41.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.195+0000 7f3745400640 1 -- 172.21.15.90:0/999820522 wait complete. 2024-04-24T09:21:41.212 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-24T09:21:41.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.409+0000 7f2fb4e00640 1 -- 172.21.15.90:0/3047258204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2fb01010f0 msgr2=0x7f2fb0101550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:41.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.409+0000 7f2fb4e00640 1 --2- 172.21.15.90:0/3047258204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2fb01010f0 0x7f2fb0101550 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2f98009900 tx=0x7f2f9802a3f0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.409+0000 7f2fb4e00640 1 -- 172.21.15.90:0/3047258204 shutdown_connections 2024-04-24T09:21:41.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.409+0000 7f2fb4e00640 1 --2- 172.21.15.90:0/3047258204 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fb0101a90 0x7f2fb0101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.409+0000 7f2fb4e00640 1 --2- 172.21.15.90:0/3047258204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2fb01010f0 0x7f2fb0101550 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.409+0000 7f2fb4e00640 1 --2- 172.21.15.90:0/3047258204 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2fb0100790 0x7f2fb0100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.409+0000 7f2fb4e00640 1 -- 172.21.15.90:0/3047258204 >> 172.21.15.90:0/3047258204 conn(0x7f2fb00fc3b0 msgr2=0x7f2fb00fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:41.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.409+0000 7f2fb4e00640 1 -- 172.21.15.90:0/3047258204 shutdown_connections 2024-04-24T09:21:41.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.409+0000 7f2fb4e00640 1 -- 172.21.15.90:0/3047258204 wait complete. 2024-04-24T09:21:41.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.410+0000 7f2fb4e00640 1 Processor -- start 2024-04-24T09:21:41.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.410+0000 7f2fb4e00640 1 -- start start 2024-04-24T09:21:41.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.410+0000 7f2fb4e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2fb0100790 0x7f2fb019a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.410+0000 7f2fb4e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2fb01010f0 0x7f2fb019a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.410+0000 7f2fb4e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fb0101a90 0x7f2fb01a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.410+0000 7f2fb4e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb010fdc0 con 0x7f2fb0101a90 2024-04-24T09:21:41.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.410+0000 7f2fb4e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb010fc40 con 0x7f2fb01010f0 2024-04-24T09:21:41.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.410+0000 7f2fb4e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb010ff40 con 0x7f2fb0100790 2024-04-24T09:21:41.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.411+0000 7f2fafe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fb0101a90 0x7f2fb01a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.411+0000 7f2faf400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2fb0100790 0x7f2fb019a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.411+0000 7f2faf400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2fb0100790 0x7f2fb019a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:54538/0 (socket says 172.21.15.90:54538) 2024-04-24T09:21:41.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.411+0000 7f2faf400640 1 -- 172.21.15.90:0/320165929 learned_addr learned my addr 172.21.15.90:0/320165929 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:41.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.411+0000 7f2fa7e00640 1 --2- 172.21.15.90:0/320165929 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2fb01010f0 0x7f2fb019a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.411+0000 7f2faf400640 1 -- 172.21.15.90:0/320165929 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2fb01010f0 msgr2=0x7f2fb019a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:41.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.411+0000 7f2faf400640 1 --2- 172.21.15.90:0/320165929 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2fb01010f0 0x7f2fb019a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.411+0000 7f2faf400640 1 -- 172.21.15.90:0/320165929 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fb0101a90 msgr2=0x7f2fb01a1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:41.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.411+0000 7f2faf400640 1 --2- 172.21.15.90:0/320165929 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fb0101a90 0x7f2fb01a1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.411+0000 7f2faf400640 1 -- 172.21.15.90:0/320165929 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb01a2000 con 0x7f2fb0100790 2024-04-24T09:21:41.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.411+0000 7f2faf400640 1 --2- 172.21.15.90:0/320165929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2fb0100790 0x7f2fb019a3e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2fa000e9f0 tx=0x7f2fa000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:41.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.412+0000 7f2facc00640 1 -- 172.21.15.90:0/320165929 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa000cd80 con 0x7f2fb0100790 2024-04-24T09:21:41.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.412+0000 7f2facc00640 1 -- 172.21.15.90:0/320165929 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fa0010910 con 0x7f2fb0100790 2024-04-24T09:21:41.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.412+0000 7f2fb4e00640 1 -- 172.21.15.90:0/320165929 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fb01a22f0 con 0x7f2fb0100790 2024-04-24T09:21:41.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.412+0000 7f2facc00640 1 -- 172.21.15.90:0/320165929 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa0018990 con 0x7f2fb0100790 2024-04-24T09:21:41.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.412+0000 7f2fb4e00640 1 -- 172.21.15.90:0/320165929 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fb01a2810 con 0x7f2fb0100790 2024-04-24T09:21:41.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.415+0000 7f2fb93c9740 1 Processor -- start 2024-04-24T09:21:41.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.415+0000 7f2fb93c9740 1 -- start start 2024-04-24T09:21:41.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.415+0000 7f2facc00640 1 -- 172.21.15.90:0/320165929 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2fa0010ac0 con 0x7f2fb0100790 2024-04-24T09:21:41.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.415+0000 7f2fb93c9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556280f59260 0x556280f59680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.415+0000 7f2fb93c9740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556280f59c50 0x556280f5e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.415+0000 7f2fb93c9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556280f5edc0 0x556280f61250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.415+0000 7f2fb93c9740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x556280ebeb10 con 0x556280f59c50 2024-04-24T09:21:41.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.415+0000 7f2fb93c9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x556280e652b0 con 0x556280f59260 2024-04-24T09:21:41.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.415+0000 7f2fb93c9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556280ebaad0 con 0x556280f5edc0 2024-04-24T09:21:41.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.416+0000 7f2faf400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556280f59c50 0x556280f5e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.416+0000 7f2fafe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556280f59260 0x556280f59680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.416+0000 7f2fafe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556280f59260 0x556280f59680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.90:56004/0 (socket says 172.21.15.90:56004) 2024-04-24T09:21:41.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.416+0000 7f2fafe00640 1 -- 172.21.15.90:0/34025264 learned_addr learned my addr 172.21.15.90:0/34025264 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:41.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.416+0000 7f2fa7e00640 1 --2- 172.21.15.90:0/34025264 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556280f5edc0 0x556280f61250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.416+0000 7f2fafe00640 1 -- 172.21.15.90:0/34025264 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556280f5edc0 msgr2=0x556280f61250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:41.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.416+0000 7f2fafe00640 1 --2- 172.21.15.90:0/34025264 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556280f5edc0 0x556280f61250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.416+0000 7f2fafe00640 1 -- 172.21.15.90:0/34025264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556280f59c50 msgr2=0x556280f5e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:41.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.416+0000 7f2fafe00640 1 --2- 172.21.15.90:0/34025264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556280f59c50 0x556280f5e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.416+0000 7f2fafe00640 1 -- 172.21.15.90:0/34025264 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556280bd12d0 con 0x556280f59260 2024-04-24T09:21:41.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.416+0000 7f2faf400640 1 --2- 172.21.15.90:0/34025264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556280f59c50 0x556280f5e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:21:41.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.416+0000 7f2fafe00640 1 --2- 172.21.15.90:0/34025264 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556280f59260 0x556280f59680 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8011440 tx=0x7f2fa8011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16689471fe319797 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:41.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.416+0000 7f2fa4c00640 1 -- 172.21.15.90:0/34025264 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8004d70 con 0x556280f59260 2024-04-24T09:21:41.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.417+0000 7f2fa4c00640 1 -- 172.21.15.90:0/34025264 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fa8004050 con 0x556280f59260 2024-04-24T09:21:41.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.417+0000 7f2fa4c00640 1 -- 172.21.15.90:0/34025264 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa800f7a0 con 0x556280f59260 2024-04-24T09:21:41.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.417+0000 7f2facc00640 1 --2- 172.21.15.90:0/320165929 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f2f900a3bb0 0x7f2f900a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.417+0000 7f2faf400640 1 --2- 172.21.15.90:0/320165929 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f2f900a3bb0 0x7f2f900a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.417+0000 7f2facc00640 1 -- 172.21.15.90:0/320165929 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f2fa00d0760 con 0x7f2fb0100790 2024-04-24T09:21:41.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.417+0000 7f2fb93c9740 1 -- 172.21.15.90:0/34025264 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556280f59260 msgr2=0x556280f59680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:41.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.417+0000 7f2fb93c9740 1 --2- 172.21.15.90:0/34025264 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556280f59260 0x556280f59680 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8011440 tx=0x7f2fa8011900 comp rx=0 tx=0).stop 2024-04-24T09:21:41.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.418+0000 7f2fb93c9740 1 -- 172.21.15.90:0/34025264 shutdown_connections 2024-04-24T09:21:41.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.418+0000 7f2fb93c9740 1 --2- 172.21.15.90:0/34025264 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556280f5edc0 0x556280f61250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.418+0000 7f2fb93c9740 1 --2- 172.21.15.90:0/34025264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556280f59c50 0x556280f5e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.418+0000 7f2fb93c9740 1 --2- 172.21.15.90:0/34025264 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556280f59260 0x556280f59680 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.418+0000 7f2fb93c9740 1 -- 172.21.15.90:0/34025264 >> 172.21.15.90:0/34025264 conn(0x556280f548d0 msgr2=0x556280f54cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:41.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.418+0000 7f2fb93c9740 1 -- 172.21.15.90:0/34025264 shutdown_connections 2024-04-24T09:21:41.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.418+0000 7f2fb93c9740 1 -- 172.21.15.90:0/34025264 wait complete. 2024-04-24T09:21:41.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.418+0000 7f2faf400640 1 --2- 172.21.15.90:0/320165929 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f2f900a3bb0 0x7f2f900a6070 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2fa009ca40 tx=0x7f2fa0087040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:41.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.418+0000 7f2fb93c9740 1 -- start start 2024-04-24T09:21:41.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.418+0000 7f2fb93c9740 1 Processor -- start 2024-04-24T09:21:41.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.419+0000 7f2fb93c9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556280f59260 0x556280f63680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.419+0000 7f2fb93c9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556280f59c50 0x556280f63bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.419+0000 7f2fb93c9740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556280f5edc0 0x556280f6ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.419+0000 7f2fb93c9740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x556280f61d70 con 0x556280f5edc0 2024-04-24T09:21:41.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.419+0000 7f2fb93c9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x556280e78590 con 0x556280f59c50 2024-04-24T09:21:41.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.419+0000 7f2fb93c9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556280f61ef0 con 0x556280f59260 2024-04-24T09:21:41.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.419+0000 7f2fb93c9740 10 client.0 _ll_register_callbacks cb 0x556280e5b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:21:41.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.419+0000 7f2fafe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556280f5edc0 0x556280f6ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.419+0000 7f2fa7e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556280f59c50 0x556280f63bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.419+0000 7f2fafe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556280f5edc0 0x556280f6ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42394/0 (socket says 172.21.15.90:42394) 2024-04-24T09:21:41.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.419+0000 7f2fafe00640 1 -- 172.21.15.90:0/3794696316 learned_addr learned my addr 172.21.15.90:0/3794696316 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:41.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.419+0000 7f2fafe00640 1 --2- 172.21.15.90:0/3794696316 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556280f59260 0x556280f63680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.420+0000 7f2fa7e00640 1 -- 172.21.15.90:0/3794696316 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556280f59260 msgr2=0x556280f63680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:41.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.420+0000 7f2fa7e00640 1 --2- 172.21.15.90:0/3794696316 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556280f59260 0x556280f63680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.420+0000 7f2fa7e00640 1 -- 172.21.15.90:0/3794696316 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556280f5edc0 msgr2=0x556280f6ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:41.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.420+0000 7f2fa7e00640 1 --2- 172.21.15.90:0/3794696316 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556280f5edc0 0x556280f6ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.420+0000 7f2fa7e00640 1 -- 172.21.15.90:0/3794696316 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556280c9b6b0 con 0x556280f59c50 2024-04-24T09:21:41.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.420+0000 7f2fa7e00640 1 --2- 172.21.15.90:0/3794696316 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556280f59c50 0x556280f63bc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2f98031440 tx=0x7f2f98032e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=948e6ef453f5941a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:41.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.420+0000 7f2f9d600640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-04-24T09:21:41.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.420+0000 7f2f9d600640 1 -- 172.21.15.90:0/3794696316 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f98035040 con 0x556280f59c50 2024-04-24T09:21:41.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.420+0000 7f2fb93c9740 1 -- 172.21.15.90:0/3794696316 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fa8004050 con 0x556280f59c50 2024-04-24T09:21:41.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.420+0000 7f2f9d600640 1 -- 172.21.15.90:0/3794696316 <== mon.2 v2:172.21.15.114:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f2f98031970 con 0x556280f59c50 2024-04-24T09:21:41.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.421+0000 7f2f9d600640 1 -- 172.21.15.90:0/3794696316 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2f98004040 con 0x556280f59c50 2024-04-24T09:21:41.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.421+0000 7f2fb93c9740 10 client.25199 fetch_fsmap learned FSMap version 27 2024-04-24T09:21:41.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.421+0000 7f2fb93c9740 1 -- 172.21.15.90:0/3794696316 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556280f6e030 con 0x556280f59c50 2024-04-24T09:21:41.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.421+0000 7f2f9d600640 1 -- 172.21.15.90:0/3794696316 <== mon.2 v2:172.21.15.114:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fa8004050 con 0x556280f59c50 2024-04-24T09:21:41.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.421+0000 7f2f9d600640 1 -- 172.21.15.90:0/3794696316 <== mon.2 v2:172.21.15.114:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2f98004810 con 0x556280f59c50 2024-04-24T09:21:41.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.421+0000 7f2fb93c9740 10 client.25199 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T09:21:41.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.422+0000 7f2fb93c9740 10 client.25199 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T09:21:41.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.422+0000 7f2fb93c9740 20 client.25199 populate_metadata read hostname 'smithi090' 2024-04-24T09:21:41.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.422+0000 7f2fb93c9740 1 --2- 172.21.15.90:0/3794696316 >> [v2:172.21.15.90:6840/397593398,v1:172.21.15.90:6841/397593398] conn(0x556280f705a0 0x556280f71a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.422+0000 7f2fb93c9740 4 client.25199 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:21:41.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.422+0000 7f2fb93c9740 1 -- 172.21.15.90:0/3794696316 --> [v2:172.21.15.90:6840/397593398,v1:172.21.15.90:6841/397593398] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556280ecc530 con 0x556280f705a0 2024-04-24T09:21:41.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.422+0000 7f2fafe00640 1 --2- 172.21.15.90:0/3794696316 >> [v2:172.21.15.90:6840/397593398,v1:172.21.15.90:6841/397593398] conn(0x556280f705a0 0x556280f71a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.422+0000 7f2fafe00640 1 --2- 172.21.15.90:0/3794696316 >> [v2:172.21.15.90:6840/397593398,v1:172.21.15.90:6841/397593398] conn(0x556280f705a0 0x556280f71a10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1b2b1d7f6a8f22e0 server_cookie=8ec5a175e6c26c1a in_seq=0 out_seq=0 2024-04-24T09:21:41.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.422+0000 7f2f9d600640 10 client.25199 ms_handle_connect on v2:172.21.15.90:6840/397593398 2024-04-24T09:21:41.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.423+0000 7f2f9d600640 1 -- 172.21.15.90:0/3794696316 <== mds.2 v2:172.21.15.90:6840/397593398 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556280ecc530 con 0x556280f705a0 2024-04-24T09:21:41.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.423+0000 7f2f9d600640 10 client.25199 handle_command_reply: tid=0 2024-04-24T09:21:41.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.423+0000 7f2fb93c9740 1 client.25199 shutdown 2024-04-24T09:21:41.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.423+0000 7f2fb93c9740 1 -- 172.21.15.90:0/3794696316 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556280f59c50 msgr2=0x556280f63bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:41.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.423+0000 7f2fb93c9740 1 --2- 172.21.15.90:0/3794696316 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556280f59c50 0x556280f63bc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2f98031440 tx=0x7f2f98032e60 comp rx=0 tx=0).stop 2024-04-24T09:21:41.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.423+0000 7f2fb93c9740 1 -- 172.21.15.90:0/3794696316 shutdown_connections 2024-04-24T09:21:41.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.423+0000 7f2fb93c9740 1 --2- 172.21.15.90:0/3794696316 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556280f5edc0 0x556280f6ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.423+0000 7f2fb93c9740 1 --2- 172.21.15.90:0/3794696316 >> [v2:172.21.15.90:6840/397593398,v1:172.21.15.90:6841/397593398] conn(0x556280f705a0 0x556280f71a10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.423+0000 7f2fb93c9740 1 --2- 172.21.15.90:0/3794696316 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556280f59c50 0x556280f63bc0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.423+0000 7f2fb93c9740 1 --2- 172.21.15.90:0/3794696316 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556280f59260 0x556280f63680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.423+0000 7f2f9d600640 0 client.25199 ms_handle_reset on v2:172.21.15.90:6840/397593398 2024-04-24T09:21:41.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.423+0000 7f2fb93c9740 1 -- 172.21.15.90:0/3794696316 >> 172.21.15.90:0/3794696316 conn(0x556280f548d0 msgr2=0x556280f53970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:41.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.424+0000 7f2fb93c9740 1 -- 172.21.15.90:0/3794696316 shutdown_connections 2024-04-24T09:21:41.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.424+0000 7f2fb93c9740 1 -- 172.21.15.90:0/3794696316 wait complete. 2024-04-24T09:21:41.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.424+0000 7f2fb93c9740 20 client.25199 trim_cache size 0 max 16384 2024-04-24T09:21:41.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.443+0000 7f2fb93c9740 1 Processor -- start 2024-04-24T09:21:41.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.443+0000 7f2fb93c9740 1 -- start start 2024-04-24T09:21:41.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.443+0000 7f2fb93c9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556280f59260 0x556280f71340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.443+0000 7f2fb93c9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556280f59c50 0x556280f6abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.443+0000 7f2fb93c9740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556280f5edc0 0x556280f537f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.444+0000 7f2fb93c9740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x556280f6fd50 con 0x556280f5edc0 2024-04-24T09:21:41.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.444+0000 7f2fb93c9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x556280f70420 con 0x556280f59260 2024-04-24T09:21:41.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.444+0000 7f2fb93c9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556280f70af0 con 0x556280f59c50 2024-04-24T09:21:41.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.444+0000 7f2fafe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556280f5edc0 0x556280f537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.444+0000 7f2fafe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556280f5edc0 0x556280f537f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42400/0 (socket says 172.21.15.90:42400) 2024-04-24T09:21:41.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.444+0000 7f2fafe00640 1 -- 172.21.15.90:0/2468330140 learned_addr learned my addr 172.21.15.90:0/2468330140 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:41.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.444+0000 7f2fa7e00640 1 --2- 172.21.15.90:0/2468330140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556280f59c50 0x556280f6abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.444+0000 7f2fafe00640 1 --2- 172.21.15.90:0/2468330140 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556280f59260 0x556280f71340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.444+0000 7f2fafe00640 1 -- 172.21.15.90:0/2468330140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556280f59c50 msgr2=0x556280f6abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:41.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.444+0000 7f2fafe00640 1 --2- 172.21.15.90:0/2468330140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556280f59c50 0x556280f6abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.444+0000 7f2fafe00640 1 -- 172.21.15.90:0/2468330140 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556280f59260 msgr2=0x556280f71340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:41.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.444+0000 7f2fafe00640 1 --2- 172.21.15.90:0/2468330140 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556280f59260 0x556280f71340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.444+0000 7f2fafe00640 1 -- 172.21.15.90:0/2468330140 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556280f5b8e0 con 0x556280f5edc0 2024-04-24T09:21:41.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.445+0000 7f2fafe00640 1 --2- 172.21.15.90:0/2468330140 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556280f5edc0 0x556280f537f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8009250 tx=0x7f2fa80094e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9a7ce2d49065167 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:41.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.445+0000 7f2f9fe00640 1 -- 172.21.15.90:0/2468330140 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8003a80 con 0x556280f5edc0 2024-04-24T09:21:41.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.445+0000 7f2f9fe00640 1 -- 172.21.15.90:0/2468330140 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fa80044e0 con 0x556280f5edc0 2024-04-24T09:21:41.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.445+0000 7f2f9fe00640 1 -- 172.21.15.90:0/2468330140 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa800f930 con 0x556280f5edc0 2024-04-24T09:21:41.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.446+0000 7f2fb93c9740 1 -- 172.21.15.90:0/2468330140 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556280f5edc0 msgr2=0x556280f537f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:41.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.446+0000 7f2fb93c9740 1 --2- 172.21.15.90:0/2468330140 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556280f5edc0 0x556280f537f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8009250 tx=0x7f2fa80094e0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.446+0000 7f2fb93c9740 1 -- 172.21.15.90:0/2468330140 shutdown_connections 2024-04-24T09:21:41.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.446+0000 7f2fb93c9740 1 --2- 172.21.15.90:0/2468330140 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556280f5edc0 0x556280f537f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.446+0000 7f2fb93c9740 1 --2- 172.21.15.90:0/2468330140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556280f59c50 0x556280f6abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.446+0000 7f2fb93c9740 1 --2- 172.21.15.90:0/2468330140 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556280f59260 0x556280f71340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.446+0000 7f2fb93c9740 1 -- 172.21.15.90:0/2468330140 >> 172.21.15.90:0/2468330140 conn(0x556280f548d0 msgr2=0x556280f530e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:41.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.446+0000 7f2fb93c9740 1 -- 172.21.15.90:0/2468330140 shutdown_connections 2024-04-24T09:21:41.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.446+0000 7f2fb93c9740 1 -- 172.21.15.90:0/2468330140 wait complete. 2024-04-24T09:21:41.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.446+0000 7f2fb93c9740 1 -- start start 2024-04-24T09:21:41.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.446+0000 7f2fb93c9740 1 Processor -- start 2024-04-24T09:21:41.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.447+0000 7f2fb93c9740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556280f59260 0x556280f66870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.447+0000 7f2fb93c9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556280f59c50 0x556280f66db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.447+0000 7f2fb93c9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556280f5edc0 0x556280f69dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.447+0000 7f2fb93c9740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x556280f71890 con 0x556280f59260 2024-04-24T09:21:41.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.447+0000 7f2fb93c9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x556280f711c0 con 0x556280f59c50 2024-04-24T09:21:41.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.447+0000 7f2fb93c9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556280f71f60 con 0x556280f5edc0 2024-04-24T09:21:41.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.447+0000 7f2fb93c9740 10 client.0 _ll_register_callbacks cb 0x7f2f98005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:21:41.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.447+0000 7f2fafe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556280f59260 0x556280f66870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.447+0000 7f2fafe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556280f59260 0x556280f66870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42408/0 (socket says 172.21.15.90:42408) 2024-04-24T09:21:41.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.447+0000 7f2fafe00640 1 -- 172.21.15.90:0/3069128271 learned_addr learned my addr 172.21.15.90:0/3069128271 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:41.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.447+0000 7f2fa7e00640 1 --2- 172.21.15.90:0/3069128271 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556280f59c50 0x556280f66db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.447+0000 7f2fafe00640 1 --2- 172.21.15.90:0/3069128271 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556280f5edc0 0x556280f69dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.447+0000 7f2fafe00640 1 -- 172.21.15.90:0/3069128271 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556280f5edc0 msgr2=0x556280f69dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:41.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.447+0000 7f2fafe00640 1 --2- 172.21.15.90:0/3069128271 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556280f5edc0 0x556280f69dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.447+0000 7f2fafe00640 1 -- 172.21.15.90:0/3069128271 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556280f59c50 msgr2=0x556280f66db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:41.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.447+0000 7f2fafe00640 1 --2- 172.21.15.90:0/3069128271 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556280f59c50 0x556280f66db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.447+0000 7f2fafe00640 1 -- 172.21.15.90:0/3069128271 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556280f6a3c0 con 0x556280f59260 2024-04-24T09:21:41.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.448+0000 7f2fafe00640 1 --2- 172.21.15.90:0/3069128271 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556280f59260 0x556280f66870 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8011940 tx=0x7f2fa8011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b87756f8641c7af server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:41.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.448+0000 7f2f9d600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-24T09:21:41.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.448+0000 7f2f9d600640 1 -- 172.21.15.90:0/3069128271 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa803b020 con 0x556280f59260 2024-04-24T09:21:41.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.448+0000 7f2f9d600640 1 -- 172.21.15.90:0/3069128271 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f2fa800f970 con 0x556280f59260 2024-04-24T09:21:41.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.448+0000 7f2fb93c9740 1 -- 172.21.15.90:0/3069128271 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fa80044e0 con 0x556280f59260 2024-04-24T09:21:41.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.448+0000 7f2f9d600640 1 -- 172.21.15.90:0/3069128271 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2fa800fb70 con 0x556280f59260 2024-04-24T09:21:41.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.448+0000 7f2fb93c9740 10 client.15120 fetch_fsmap learned FSMap version 27 2024-04-24T09:21:41.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.448+0000 7f2fb93c9740 1 -- 172.21.15.90:0/3069128271 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556280fd6ac0 con 0x556280f59260 2024-04-24T09:21:41.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.448+0000 7f2f9d600640 1 -- 172.21.15.90:0/3069128271 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fa80044e0 con 0x556280f59260 2024-04-24T09:21:41.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.449+0000 7f2f9d600640 1 -- 172.21.15.90:0/3069128271 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2fa8034040 con 0x556280f59260 2024-04-24T09:21:41.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.449+0000 7f2fb93c9740 10 client.15120 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T09:21:41.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.449+0000 7f2fb93c9740 10 client.15120 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-24T09:21:41.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.449+0000 7f2fb93c9740 20 client.15120 populate_metadata read hostname 'smithi090' 2024-04-24T09:21:41.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.449+0000 7f2fb93c9740 1 --2- 172.21.15.90:0/3069128271 >> [v2:172.21.15.90:6840/397593398,v1:172.21.15.90:6841/397593398] conn(0x556280fdb1a0 0x556280fd7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.449+0000 7f2fb93c9740 4 client.15120 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T09:21:41.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.449+0000 7f2fb93c9740 1 -- 172.21.15.90:0/3069128271 --> [v2:172.21.15.90:6840/397593398,v1:172.21.15.90:6841/397593398] -- command(tid 0: {"prefix": "status"}) -- 0x556280ec63a0 con 0x556280fdb1a0 2024-04-24T09:21:41.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.449+0000 7f2fa7e00640 1 --2- 172.21.15.90:0/3069128271 >> [v2:172.21.15.90:6840/397593398,v1:172.21.15.90:6841/397593398] conn(0x556280fdb1a0 0x556280fd7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.450+0000 7f2fa7e00640 1 --2- 172.21.15.90:0/3069128271 >> [v2:172.21.15.90:6840/397593398,v1:172.21.15.90:6841/397593398] conn(0x556280fdb1a0 0x556280fd7cb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9bf54a5f99f90b45 server_cookie=77bf1a6e67c5f284 in_seq=0 out_seq=0 2024-04-24T09:21:41.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.450+0000 7f2f9d600640 10 client.15120 ms_handle_connect on v2:172.21.15.90:6840/397593398 2024-04-24T09:21:41.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.450+0000 7f2f9d600640 1 -- 172.21.15.90:0/3069128271 <== mds.2 v2:172.21.15.90:6840/397593398 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x556280ec63a0 con 0x556280fdb1a0 2024-04-24T09:21:41.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.450+0000 7f2f9d600640 10 client.15120 handle_command_reply: tid=0 2024-04-24T09:21:41.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.450+0000 7f2fb93c9740 1 client.15120 shutdown 2024-04-24T09:21:41.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.450+0000 7f2fb93c9740 1 -- 172.21.15.90:0/3069128271 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556280f59260 msgr2=0x556280f66870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:41.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.450+0000 7f2fb93c9740 1 --2- 172.21.15.90:0/3069128271 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556280f59260 0x556280f66870 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8011940 tx=0x7f2fa8011e00 comp rx=0 tx=0).stop 2024-04-24T09:21:41.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.450+0000 7f2fb93c9740 1 -- 172.21.15.90:0/3069128271 shutdown_connections 2024-04-24T09:21:41.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.450+0000 7f2fb93c9740 1 --2- 172.21.15.90:0/3069128271 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556280f5edc0 0x556280f69dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.450+0000 7f2fb93c9740 1 --2- 172.21.15.90:0/3069128271 >> [v2:172.21.15.90:6840/397593398,v1:172.21.15.90:6841/397593398] conn(0x556280fdb1a0 0x556280fd7cb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.450+0000 7f2fb93c9740 1 --2- 172.21.15.90:0/3069128271 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556280f59c50 0x556280f66db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.450+0000 7f2fb93c9740 1 --2- 172.21.15.90:0/3069128271 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556280f59260 0x556280f66870 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.450+0000 7f2f9d600640 0 client.15120 ms_handle_reset on v2:172.21.15.90:6840/397593398 2024-04-24T09:21:41.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.450+0000 7f2fb93c9740 1 -- 172.21.15.90:0/3069128271 >> 172.21.15.90:0/3069128271 conn(0x556280f548d0 msgr2=0x556280f53970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:41.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.450+0000 7f2fb93c9740 1 -- 172.21.15.90:0/3069128271 shutdown_connections 2024-04-24T09:21:41.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.450+0000 7f2fb93c9740 1 -- 172.21.15.90:0/3069128271 wait complete. 2024-04-24T09:21:41.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.451+0000 7f2fb93c9740 20 client.15120 trim_cache size 0 max 16384 2024-04-24T09:21:41.455 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T09:21:41.455 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "30ee9be2-021a-11ef-bc93-c7b262605968", 2024-04-24T09:21:41.455 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 2, 2024-04-24T09:21:41.455 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 24545, 2024-04-24T09:21:41.455 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-04-24T09:21:41.455 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-04-24T09:21:41.455 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-04-24T09:21:41.455 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 147.16513619599999, 2024-04-24T09:21:41.455 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 27, 2024-04-24T09:21:41.455 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 88, 2024-04-24T09:21:41.456 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 86, 2024-04-24T09:21:41.456 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 160.525967544 2024-04-24T09:21:41.456 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T09:21:41.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.451+0000 7f2fb4e00640 1 -- 172.21.15.90:0/320165929 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f2f900a3bb0 msgr2=0x7f2f900a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:41.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.451+0000 7f2fb4e00640 1 --2- 172.21.15.90:0/320165929 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f2f900a3bb0 0x7f2f900a6070 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2fa009ca40 tx=0x7f2fa0087040 comp rx=0 tx=0).stop 2024-04-24T09:21:41.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.451+0000 7f2fb4e00640 1 -- 172.21.15.90:0/320165929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2fb0100790 msgr2=0x7f2fb019a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:41.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.451+0000 7f2fb4e00640 1 --2- 172.21.15.90:0/320165929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2fb0100790 0x7f2fb019a3e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2fa000e9f0 tx=0x7f2fa000eeb0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.451+0000 7f2fb4e00640 1 -- 172.21.15.90:0/320165929 shutdown_connections 2024-04-24T09:21:41.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.451+0000 7f2fb4e00640 1 --2- 172.21.15.90:0/320165929 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f2f900a3bb0 0x7f2f900a6070 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.451+0000 7f2fb4e00640 1 --2- 172.21.15.90:0/320165929 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fb0101a90 0x7f2fb01a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.451+0000 7f2fb4e00640 1 --2- 172.21.15.90:0/320165929 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2fb01010f0 0x7f2fb019a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.451+0000 7f2fb4e00640 1 --2- 172.21.15.90:0/320165929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2fb0100790 0x7f2fb019a3e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.451+0000 7f2fb4e00640 1 -- 172.21.15.90:0/320165929 >> 172.21.15.90:0/320165929 conn(0x7f2fb00fc3b0 msgr2=0x7f2fb00fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:41.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.451+0000 7f2fb4e00640 1 -- 172.21.15.90:0/320165929 shutdown_connections 2024-04-24T09:21:41.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.451+0000 7f2fb4e00640 1 -- 172.21.15.90:0/320165929 wait complete. 2024-04-24T09:21:41.469 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-24T09:21:41.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.665+0000 7fc5dbe00640 1 -- 172.21.15.90:0/2121171066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc5d4109850 msgr2=0x7fc5d4109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:41.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.665+0000 7fc5dbe00640 1 --2- 172.21.15.90:0/2121171066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc5d4109850 0x7fc5d4109cd0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0009900 tx=0x7fc5c002a3f0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.665+0000 7fc5dbe00640 1 -- 172.21.15.90:0/2121171066 shutdown_connections 2024-04-24T09:21:41.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.665+0000 7fc5dbe00640 1 --2- 172.21.15.90:0/2121171066 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5d410a210 0x7fc5d4110aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.665+0000 7fc5dbe00640 1 --2- 172.21.15.90:0/2121171066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc5d4109850 0x7fc5d4109cd0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.665+0000 7fc5dbe00640 1 --2- 172.21.15.90:0/2121171066 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5d4108660 0x7fc5d4108a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.665+0000 7fc5dbe00640 1 -- 172.21.15.90:0/2121171066 >> 172.21.15.90:0/2121171066 conn(0x7fc5d4077600 msgr2=0x7fc5d4079a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:41.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.665+0000 7fc5dbe00640 1 -- 172.21.15.90:0/2121171066 shutdown_connections 2024-04-24T09:21:41.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.665+0000 7fc5dbe00640 1 -- 172.21.15.90:0/2121171066 wait complete. 2024-04-24T09:21:41.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.666+0000 7fc5dbe00640 1 Processor -- start 2024-04-24T09:21:41.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.666+0000 7fc5dbe00640 1 -- start start 2024-04-24T09:21:41.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.667+0000 7fc5dbe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5d4108660 0x7fc5d419e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.667+0000 7fc5dbe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc5d4109850 0x7fc5d419eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.667+0000 7fc5dbe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5d410a210 0x7fc5d41a5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.667+0000 7fc5dbe00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d4114310 con 0x7fc5d4108660 2024-04-24T09:21:41.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.667+0000 7fc5dbe00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d4114190 con 0x7fc5d410a210 2024-04-24T09:21:41.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.667+0000 7fc5dbe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d4114490 con 0x7fc5d4109850 2024-04-24T09:21:41.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.667+0000 7fc5d9400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5d4108660 0x7fc5d419e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.667+0000 7fc5d9400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5d4108660 0x7fc5d419e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42424/0 (socket says 172.21.15.90:42424) 2024-04-24T09:21:41.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.667+0000 7fc5d9400640 1 -- 172.21.15.90:0/1816581313 learned_addr learned my addr 172.21.15.90:0/1816581313 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:41.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.667+0000 7fc5d8a00640 1 --2- 172.21.15.90:0/1816581313 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc5d4109850 0x7fc5d419eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.667+0000 7fc5d9e00640 1 --2- 172.21.15.90:0/1816581313 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5d410a210 0x7fc5d41a5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.667+0000 7fc5d9400640 1 -- 172.21.15.90:0/1816581313 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc5d4109850 msgr2=0x7fc5d419eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:41.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.667+0000 7fc5d9400640 1 --2- 172.21.15.90:0/1816581313 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc5d4109850 0x7fc5d419eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.667+0000 7fc5d9400640 1 -- 172.21.15.90:0/1816581313 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5d410a210 msgr2=0x7fc5d41a5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:41.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.667+0000 7fc5d9400640 1 --2- 172.21.15.90:0/1816581313 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5d410a210 0x7fc5d41a5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.667+0000 7fc5d9400640 1 -- 172.21.15.90:0/1816581313 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5d41a6580 con 0x7fc5d4108660 2024-04-24T09:21:41.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.667+0000 7fc5d8a00640 1 --2- 172.21.15.90:0/1816581313 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc5d4109850 0x7fc5d419eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:21:41.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.668+0000 7fc5d9400640 1 --2- 172.21.15.90:0/1816581313 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5d4108660 0x7fc5d419e960 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800e9c0 tx=0x7fc5c800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:41.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.668+0000 7fc5d2000640 1 -- 172.21.15.90:0/1816581313 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c800cd80 con 0x7fc5d4108660 2024-04-24T09:21:41.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.668+0000 7fc5d2000640 1 -- 172.21.15.90:0/1816581313 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5c80035f0 con 0x7fc5d4108660 2024-04-24T09:21:41.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.668+0000 7fc5dbe00640 1 -- 172.21.15.90:0/1816581313 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5d41a6870 con 0x7fc5d4108660 2024-04-24T09:21:41.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.668+0000 7fc5d2000640 1 -- 172.21.15.90:0/1816581313 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8010690 con 0x7fc5d4108660 2024-04-24T09:21:41.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.668+0000 7fc5dbe00640 1 -- 172.21.15.90:0/1816581313 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5d41a6d90 con 0x7fc5d4108660 2024-04-24T09:21:41.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.671+0000 7fc5dec4e740 1 Processor -- start 2024-04-24T09:21:41.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.672+0000 7fc5dec4e740 1 -- start start 2024-04-24T09:21:41.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.672+0000 7fc5dec4e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601adc2f090 0x5601adc2f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.672+0000 7fc5dec4e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601adc2fa80 0x5601adc34580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.672+0000 7fc5dec4e740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601adc34bf0 0x5601adc37080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.672+0000 7fc5dec4e740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5601adb94b10 con 0x5601adc34bf0 2024-04-24T09:21:41.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.672+0000 7fc5dec4e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5601adb3b2b0 con 0x5601adc2fa80 2024-04-24T09:21:41.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.672+0000 7fc5dec4e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5601adb90ad0 con 0x5601adc2f090 2024-04-24T09:21:41.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.672+0000 7fc5d2000640 1 -- 172.21.15.90:0/1816581313 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc5c80037a0 con 0x7fc5d4108660 2024-04-24T09:21:41.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.672+0000 7fc5d8a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601adc34bf0 0x5601adc37080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.672+0000 7fc5d8a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601adc34bf0 0x5601adc37080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42432/0 (socket says 172.21.15.90:42432) 2024-04-24T09:21:41.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.672+0000 7fc5d8a00640 1 -- 172.21.15.90:0/27546744 learned_addr learned my addr 172.21.15.90:0/27546744 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:41.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.672+0000 7fc5d9e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601adc2f090 0x5601adc2f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.673+0000 7fc5d9400640 1 --2- 172.21.15.90:0/27546744 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601adc2fa80 0x5601adc34580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.673+0000 7fc5d8a00640 1 -- 172.21.15.90:0/27546744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601adc2f090 msgr2=0x5601adc2f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:41.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.673+0000 7fc5d8a00640 1 --2- 172.21.15.90:0/27546744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601adc2f090 0x5601adc2f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.673+0000 7fc5d8a00640 1 -- 172.21.15.90:0/27546744 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601adc2fa80 msgr2=0x5601adc34580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:41.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.673+0000 7fc5d8a00640 1 --2- 172.21.15.90:0/27546744 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601adc2fa80 0x5601adc34580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.673+0000 7fc5d8a00640 1 -- 172.21.15.90:0/27546744 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601ad8a72d0 con 0x5601adc34bf0 2024-04-24T09:21:41.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.673+0000 7fc5d9400640 1 --2- 172.21.15.90:0/27546744 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601adc2fa80 0x5601adc34580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:21:41.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.673+0000 7fc5d8a00640 1 --2- 172.21.15.90:0/27546744 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601adc34bf0 0x5601adc37080 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0009900 tx=0x7fc5c00043f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=435243718e4bdef6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:41.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.673+0000 7fc5c6600640 1 -- 172.21.15.90:0/27546744 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c0035040 con 0x5601adc34bf0 2024-04-24T09:21:41.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.673+0000 7fc5c6600640 1 -- 172.21.15.90:0/27546744 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5c0004770 con 0x5601adc34bf0 2024-04-24T09:21:41.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.673+0000 7fc5c6600640 1 -- 172.21.15.90:0/27546744 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c0031a30 con 0x5601adc34bf0 2024-04-24T09:21:41.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.674+0000 7fc5d2000640 1 --2- 172.21.15.90:0/1816581313 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fc5bc0a3bb0 0x7fc5bc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.674+0000 7fc5d2000640 1 -- 172.21.15.90:0/1816581313 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fc5c80d07e0 con 0x7fc5d4108660 2024-04-24T09:21:41.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.674+0000 7fc5d9e00640 1 --2- 172.21.15.90:0/1816581313 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fc5bc0a3bb0 0x7fc5bc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.674+0000 7fc5dec4e740 1 -- 172.21.15.90:0/27546744 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601adc34bf0 msgr2=0x5601adc37080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:41.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.674+0000 7fc5dec4e740 1 --2- 172.21.15.90:0/27546744 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601adc34bf0 0x5601adc37080 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0009900 tx=0x7fc5c00043f0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.674+0000 7fc5dec4e740 1 -- 172.21.15.90:0/27546744 shutdown_connections 2024-04-24T09:21:41.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.674+0000 7fc5dec4e740 1 --2- 172.21.15.90:0/27546744 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601adc34bf0 0x5601adc37080 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.674+0000 7fc5dec4e740 1 --2- 172.21.15.90:0/27546744 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601adc2fa80 0x5601adc34580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.674+0000 7fc5dec4e740 1 --2- 172.21.15.90:0/27546744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601adc2f090 0x5601adc2f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.674+0000 7fc5dec4e740 1 -- 172.21.15.90:0/27546744 >> 172.21.15.90:0/27546744 conn(0x5601adc2a700 msgr2=0x5601adc2ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:41.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.674+0000 7fc5dec4e740 1 -- 172.21.15.90:0/27546744 shutdown_connections 2024-04-24T09:21:41.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.675+0000 7fc5dec4e740 1 -- 172.21.15.90:0/27546744 wait complete. 2024-04-24T09:21:41.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.675+0000 7fc5d9e00640 1 --2- 172.21.15.90:0/1816581313 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fc5bc0a3bb0 0x7fc5bc0a6070 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x5601adc302e0 tx=0x7fc5cc009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:41.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.675+0000 7fc5dec4e740 1 -- start start 2024-04-24T09:21:41.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.675+0000 7fc5dec4e740 1 Processor -- start 2024-04-24T09:21:41.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.676+0000 7fc5dec4e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601adc2f090 0x5601adc39470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.676+0000 7fc5dec4e740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601adc2fa80 0x5601adc399b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.676+0000 7fc5dec4e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601adc34bf0 0x5601adc409f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.676+0000 7fc5dec4e740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5601adc37b40 con 0x5601adc2fa80 2024-04-24T09:21:41.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.676+0000 7fc5dec4e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5601adb4e590 con 0x5601adc34bf0 2024-04-24T09:21:41.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.676+0000 7fc5dec4e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5601adc37cc0 con 0x5601adc2f090 2024-04-24T09:21:41.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.676+0000 7fc5dec4e740 10 client.0 _ll_register_callbacks cb 0x5601adb313a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:21:41.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.676+0000 7fc5d8a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601adc2fa80 0x5601adc399b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.676+0000 7fc5d9400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601adc2f090 0x5601adc39470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.676+0000 7fc5d8a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601adc2fa80 0x5601adc399b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42434/0 (socket says 172.21.15.90:42434) 2024-04-24T09:21:41.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.676+0000 7fc5d9400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601adc2f090 0x5601adc39470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:54604/0 (socket says 172.21.15.90:54604) 2024-04-24T09:21:41.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.676+0000 7fc5d8a00640 1 -- 172.21.15.90:0/1363750414 learned_addr learned my addr 172.21.15.90:0/1363750414 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:41.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.676+0000 7fc5d9e00640 1 --2- 172.21.15.90:0/1363750414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601adc34bf0 0x5601adc409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.676+0000 7fc5d9400640 1 -- 172.21.15.90:0/1363750414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601adc34bf0 msgr2=0x5601adc409f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:41.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.676+0000 7fc5d9400640 1 --2- 172.21.15.90:0/1363750414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601adc34bf0 0x5601adc409f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.676+0000 7fc5d9400640 1 -- 172.21.15.90:0/1363750414 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601adc2fa80 msgr2=0x5601adc399b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:41.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.676+0000 7fc5d9400640 1 --2- 172.21.15.90:0/1363750414 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601adc2fa80 0x5601adc399b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.676+0000 7fc5d9400640 1 -- 172.21.15.90:0/1363750414 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601ad9716b0 con 0x5601adc2f090 2024-04-24T09:21:41.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.677+0000 7fc5d9400640 1 --2- 172.21.15.90:0/1363750414 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601adc2f090 0x5601adc39470 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x5601adc304f0 tx=0x7fc5c809c4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d94028faa187b8f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:41.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.677+0000 7fc5baa00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-04-24T09:21:41.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.677+0000 7fc5baa00640 1 -- 172.21.15.90:0/1363750414 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c80a5730 con 0x5601adc2f090 2024-04-24T09:21:41.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.677+0000 7fc5baa00640 1 -- 172.21.15.90:0/1363750414 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fc5c8003b30 con 0x5601adc2f090 2024-04-24T09:21:41.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.677+0000 7fc5dec4e740 1 -- 172.21.15.90:0/1363750414 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5c0004770 con 0x5601adc2f090 2024-04-24T09:21:41.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.681+0000 7fc5baa00640 1 -- 172.21.15.90:0/1363750414 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc5c809bf10 con 0x5601adc2f090 2024-04-24T09:21:41.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.681+0000 7fc5dec4e740 10 client.25240 fetch_fsmap learned FSMap version 27 2024-04-24T09:21:41.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.681+0000 7fc5dec4e740 1 -- 172.21.15.90:0/1363750414 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5601adc43d40 con 0x5601adc2f090 2024-04-24T09:21:41.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.682+0000 7fc5baa00640 1 -- 172.21.15.90:0/1363750414 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5c0004770 con 0x5601adc2f090 2024-04-24T09:21:41.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.682+0000 7fc5baa00640 1 -- 172.21.15.90:0/1363750414 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc5c80a2000 con 0x5601adc2f090 2024-04-24T09:21:41.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.682+0000 7fc5dec4e740 10 client.25240 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T09:21:41.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.682+0000 7fc5dec4e740 10 client.25240 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-24T09:21:41.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.682+0000 7fc5dec4e740 20 client.25240 populate_metadata read hostname 'smithi090' 2024-04-24T09:21:41.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.682+0000 7fc5dec4e740 1 --2- 172.21.15.90:0/1363750414 >> [v2:172.21.15.149:6838/652734263,v1:172.21.15.149:6839/652734263] conn(0x5601adc463b0 0x5601adc47820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.682+0000 7fc5dec4e740 4 client.25240 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:21:41.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.682+0000 7fc5dec4e740 1 -- 172.21.15.90:0/1363750414 --> [v2:172.21.15.149:6838/652734263,v1:172.21.15.149:6839/652734263] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601adba2530 con 0x5601adc463b0 2024-04-24T09:21:41.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.682+0000 7fc5d8a00640 1 --2- 172.21.15.90:0/1363750414 >> [v2:172.21.15.149:6838/652734263,v1:172.21.15.149:6839/652734263] conn(0x5601adc463b0 0x5601adc47820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.683+0000 7fc5d8a00640 1 --2- 172.21.15.90:0/1363750414 >> [v2:172.21.15.149:6838/652734263,v1:172.21.15.149:6839/652734263] conn(0x5601adc463b0 0x5601adc47820 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7809398a6b1e44e8 server_cookie=c5f31dc020833509 in_seq=0 out_seq=0 2024-04-24T09:21:41.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.683+0000 7fc5baa00640 10 client.25240 ms_handle_connect on v2:172.21.15.149:6838/652734263 2024-04-24T09:21:41.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.684+0000 7fc5baa00640 1 -- 172.21.15.90:0/1363750414 <== mds.1 v2:172.21.15.149:6838/652734263 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5601adba2530 con 0x5601adc463b0 2024-04-24T09:21:41.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.684+0000 7fc5baa00640 10 client.25240 handle_command_reply: tid=0 2024-04-24T09:21:41.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.684+0000 7fc5dec4e740 1 client.25240 shutdown 2024-04-24T09:21:41.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.684+0000 7fc5dec4e740 1 -- 172.21.15.90:0/1363750414 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601adc2f090 msgr2=0x5601adc39470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:41.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.684+0000 7fc5dec4e740 1 --2- 172.21.15.90:0/1363750414 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601adc2f090 0x5601adc39470 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x5601adc304f0 tx=0x7fc5c809c4e0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.684+0000 7fc5dec4e740 1 -- 172.21.15.90:0/1363750414 shutdown_connections 2024-04-24T09:21:41.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.684+0000 7fc5dec4e740 1 --2- 172.21.15.90:0/1363750414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601adc34bf0 0x5601adc409f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.684+0000 7fc5dec4e740 1 --2- 172.21.15.90:0/1363750414 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601adc2fa80 0x5601adc399b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.684+0000 7fc5dec4e740 1 --2- 172.21.15.90:0/1363750414 >> [v2:172.21.15.149:6838/652734263,v1:172.21.15.149:6839/652734263] conn(0x5601adc463b0 0x5601adc47820 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.684+0000 7fc5dec4e740 1 --2- 172.21.15.90:0/1363750414 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601adc2f090 0x5601adc39470 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.684+0000 7fc5baa00640 0 client.25240 ms_handle_reset on v2:172.21.15.149:6838/652734263 2024-04-24T09:21:41.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.685+0000 7fc5dec4e740 1 -- 172.21.15.90:0/1363750414 >> 172.21.15.90:0/1363750414 conn(0x5601adc2a700 msgr2=0x5601adc298c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:41.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.685+0000 7fc5dec4e740 1 -- 172.21.15.90:0/1363750414 shutdown_connections 2024-04-24T09:21:41.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.685+0000 7fc5dec4e740 1 -- 172.21.15.90:0/1363750414 wait complete. 2024-04-24T09:21:41.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.685+0000 7fc5dec4e740 20 client.25240 trim_cache size 0 max 16384 2024-04-24T09:21:41.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.705+0000 7fc5dec4e740 1 Processor -- start 2024-04-24T09:21:41.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.705+0000 7fc5dec4e740 1 -- start start 2024-04-24T09:21:41.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.705+0000 7fc5dec4e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601adc2f090 0x5601adc344b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.705+0000 7fc5dec4e740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601adc2fa80 0x5601adc409b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.705+0000 7fc5dec4e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601adc34bf0 0x5601adc36fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.705+0000 7fc5dec4e740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5601adc45b60 con 0x5601adc2fa80 2024-04-24T09:21:41.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.705+0000 7fc5dec4e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5601adc46230 con 0x5601adc2f090 2024-04-24T09:21:41.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.705+0000 7fc5dec4e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5601adc46900 con 0x5601adc34bf0 2024-04-24T09:21:41.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.705+0000 7fc5d8a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601adc2fa80 0x5601adc409b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.705+0000 7fc5d8a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601adc2fa80 0x5601adc409b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42444/0 (socket says 172.21.15.90:42444) 2024-04-24T09:21:41.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.705+0000 7fc5d8a00640 1 -- 172.21.15.90:0/447695872 learned_addr learned my addr 172.21.15.90:0/447695872 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:41.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.705+0000 7fc5d9e00640 1 --2- 172.21.15.90:0/447695872 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601adc34bf0 0x5601adc36fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.705+0000 7fc5d9400640 1 --2- 172.21.15.90:0/447695872 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601adc2f090 0x5601adc344b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.706+0000 7fc5d8a00640 1 -- 172.21.15.90:0/447695872 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601adc34bf0 msgr2=0x5601adc36fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:41.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.706+0000 7fc5d8a00640 1 --2- 172.21.15.90:0/447695872 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601adc34bf0 0x5601adc36fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.706+0000 7fc5d8a00640 1 -- 172.21.15.90:0/447695872 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601adc2f090 msgr2=0x5601adc344b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:41.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.706+0000 7fc5d8a00640 1 --2- 172.21.15.90:0/447695872 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601adc2f090 0x5601adc344b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.706+0000 7fc5d8a00640 1 -- 172.21.15.90:0/447695872 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601adc2fe60 con 0x5601adc2fa80 2024-04-24T09:21:41.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.706+0000 7fc5d9400640 1 --2- 172.21.15.90:0/447695872 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601adc2f090 0x5601adc344b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:21:41.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.706+0000 7fc5d8a00640 1 --2- 172.21.15.90:0/447695872 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601adc2fa80 0x5601adc409b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc5c00096c0 tx=0x7fc5c00315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cd7a58ab00ea95 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:41.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.706+0000 7fc5c5c00640 1 -- 172.21.15.90:0/447695872 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c0035040 con 0x5601adc2fa80 2024-04-24T09:21:41.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.706+0000 7fc5c5c00640 1 -- 172.21.15.90:0/447695872 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5c0031d90 con 0x5601adc2fa80 2024-04-24T09:21:41.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.706+0000 7fc5c5c00640 1 -- 172.21.15.90:0/447695872 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c00035f0 con 0x5601adc2fa80 2024-04-24T09:21:41.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.707+0000 7fc5dec4e740 1 -- 172.21.15.90:0/447695872 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601adc2fa80 msgr2=0x5601adc409b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:41.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.707+0000 7fc5dec4e740 1 --2- 172.21.15.90:0/447695872 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601adc2fa80 0x5601adc409b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc5c00096c0 tx=0x7fc5c00315b0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.707+0000 7fc5dec4e740 1 -- 172.21.15.90:0/447695872 shutdown_connections 2024-04-24T09:21:41.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.707+0000 7fc5dec4e740 1 --2- 172.21.15.90:0/447695872 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601adc34bf0 0x5601adc36fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.707+0000 7fc5dec4e740 1 --2- 172.21.15.90:0/447695872 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601adc2fa80 0x5601adc409b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.707+0000 7fc5dec4e740 1 --2- 172.21.15.90:0/447695872 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601adc2f090 0x5601adc344b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.707+0000 7fc5dec4e740 1 -- 172.21.15.90:0/447695872 >> 172.21.15.90:0/447695872 conn(0x5601adc2a700 msgr2=0x5601adc2aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:41.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.707+0000 7fc5dec4e740 1 -- 172.21.15.90:0/447695872 shutdown_connections 2024-04-24T09:21:41.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.707+0000 7fc5dec4e740 1 -- 172.21.15.90:0/447695872 wait complete. 2024-04-24T09:21:41.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.707+0000 7fc5dec4e740 1 -- start start 2024-04-24T09:21:41.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.708+0000 7fc5dec4e740 1 Processor -- start 2024-04-24T09:21:41.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.708+0000 7fc5dec4e740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601adc2f090 0x5601adc3e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.708+0000 7fc5dec4e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601adc2fa80 0x5601adca8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.708+0000 7fc5dec4e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601adc34bf0 0x5601adcaa890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.708+0000 7fc5dec4e740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5601adc476a0 con 0x5601adc2f090 2024-04-24T09:21:41.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.708+0000 7fc5dec4e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5601adc46fd0 con 0x5601adc34bf0 2024-04-24T09:21:41.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.708+0000 7fc5dec4e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5601adc47d70 con 0x5601adc2fa80 2024-04-24T09:21:41.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.708+0000 7fc5dec4e740 10 client.0 _ll_register_callbacks cb 0x7fc5c809c600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:21:41.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.708+0000 7fc5d8a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601adc2fa80 0x5601adca8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.708+0000 7fc5d9400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601adc2f090 0x5601adc3e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.708+0000 7fc5d9400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601adc2f090 0x5601adc3e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42454/0 (socket says 172.21.15.90:42454) 2024-04-24T09:21:41.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.708+0000 7fc5d9400640 1 -- 172.21.15.90:0/1685940385 learned_addr learned my addr 172.21.15.90:0/1685940385 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:41.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.708+0000 7fc5d8a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601adc2fa80 0x5601adca8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:54622/0 (socket says 172.21.15.90:54622) 2024-04-24T09:21:41.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.709+0000 7fc5d9e00640 1 --2- 172.21.15.90:0/1685940385 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601adc34bf0 0x5601adcaa890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.709+0000 7fc5d8a00640 1 -- 172.21.15.90:0/1685940385 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601adc34bf0 msgr2=0x5601adcaa890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:41.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.709+0000 7fc5d8a00640 1 --2- 172.21.15.90:0/1685940385 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601adc34bf0 0x5601adcaa890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.709+0000 7fc5d8a00640 1 -- 172.21.15.90:0/1685940385 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601adc2f090 msgr2=0x5601adc3e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:41.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.709+0000 7fc5d8a00640 1 --2- 172.21.15.90:0/1685940385 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601adc2f090 0x5601adc3e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.709+0000 7fc5d8a00640 1 -- 172.21.15.90:0/1685940385 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601adcaaf30 con 0x5601adc2fa80 2024-04-24T09:21:41.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.709+0000 7fc5d9400640 1 --2- 172.21.15.90:0/1685940385 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601adc2f090 0x5601adc3e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:21:41.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.709+0000 7fc5d8a00640 1 --2- 172.21.15.90:0/1685940385 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601adc2fa80 0x5601adca8340 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x5601adc3a610 tx=0x7fc5c00538f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd326afc7d54f1bf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:41.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.709+0000 7fc5baa00640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-04-24T09:21:41.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.709+0000 7fc5baa00640 1 -- 172.21.15.90:0/1685940385 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c0061060 con 0x5601adc2fa80 2024-04-24T09:21:41.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.709+0000 7fc5dec4e740 1 -- 172.21.15.90:0/1685940385 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5c0031d90 con 0x5601adc2fa80 2024-04-24T09:21:41.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.709+0000 7fc5baa00640 1 -- 172.21.15.90:0/1685940385 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fc5c0053320 con 0x5601adc2fa80 2024-04-24T09:21:41.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.710+0000 7fc5baa00640 1 -- 172.21.15.90:0/1685940385 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc5c0035070 con 0x5601adc2fa80 2024-04-24T09:21:41.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.710+0000 7fc5dec4e740 10 client.25246 fetch_fsmap learned FSMap version 27 2024-04-24T09:21:41.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.710+0000 7fc5dec4e740 1 -- 172.21.15.90:0/1685940385 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5601adcac230 con 0x5601adc2fa80 2024-04-24T09:21:41.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.710+0000 7fc5baa00640 1 -- 172.21.15.90:0/1685940385 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5c0031d90 con 0x5601adc2fa80 2024-04-24T09:21:41.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.710+0000 7fc5baa00640 1 -- 172.21.15.90:0/1685940385 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc5c0066070 con 0x5601adc2fa80 2024-04-24T09:21:41.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.710+0000 7fc5dec4e740 10 client.25246 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T09:21:41.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.710+0000 7fc5dec4e740 10 client.25246 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-24T09:21:41.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.710+0000 7fc5dec4e740 20 client.25246 populate_metadata read hostname 'smithi090' 2024-04-24T09:21:41.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.710+0000 7fc5dec4e740 1 --2- 172.21.15.90:0/1685940385 >> [v2:172.21.15.149:6838/652734263,v1:172.21.15.149:6839/652734263] conn(0x5601adcb06c0 0x5601adcaf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.710+0000 7fc5dec4e740 4 client.25246 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T09:21:41.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.710+0000 7fc5dec4e740 1 -- 172.21.15.90:0/1685940385 --> [v2:172.21.15.149:6838/652734263,v1:172.21.15.149:6839/652734263] -- command(tid 0: {"prefix": "status"}) -- 0x5601adb9c3a0 con 0x5601adcb06c0 2024-04-24T09:21:41.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.710+0000 7fc5d9400640 1 --2- 172.21.15.90:0/1685940385 >> [v2:172.21.15.149:6838/652734263,v1:172.21.15.149:6839/652734263] conn(0x5601adcb06c0 0x5601adcaf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.711+0000 7fc5d9400640 1 --2- 172.21.15.90:0/1685940385 >> [v2:172.21.15.149:6838/652734263,v1:172.21.15.149:6839/652734263] conn(0x5601adcb06c0 0x5601adcaf3d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6eb7f85da5475d57 server_cookie=ffe7fbafaedda982 in_seq=0 out_seq=0 2024-04-24T09:21:41.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.711+0000 7fc5baa00640 10 client.25246 ms_handle_connect on v2:172.21.15.149:6838/652734263 2024-04-24T09:21:41.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.711+0000 7fc5baa00640 1 -- 172.21.15.90:0/1685940385 <== mds.1 v2:172.21.15.149:6838/652734263 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5601adb9c3a0 con 0x5601adcb06c0 2024-04-24T09:21:41.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.711+0000 7fc5baa00640 10 client.25246 handle_command_reply: tid=0 2024-04-24T09:21:41.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.711+0000 7fc5dec4e740 1 client.25246 shutdown 2024-04-24T09:21:41.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.711+0000 7fc5dec4e740 1 -- 172.21.15.90:0/1685940385 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601adc2fa80 msgr2=0x5601adca8340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:41.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.711+0000 7fc5dec4e740 1 --2- 172.21.15.90:0/1685940385 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601adc2fa80 0x5601adca8340 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x5601adc3a610 tx=0x7fc5c00538f0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.711+0000 7fc5dec4e740 1 -- 172.21.15.90:0/1685940385 shutdown_connections 2024-04-24T09:21:41.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.711+0000 7fc5dec4e740 1 --2- 172.21.15.90:0/1685940385 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601adc34bf0 0x5601adcaa890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.711+0000 7fc5dec4e740 1 --2- 172.21.15.90:0/1685940385 >> [v2:172.21.15.149:6838/652734263,v1:172.21.15.149:6839/652734263] conn(0x5601adcb06c0 0x5601adcaf3d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.711+0000 7fc5dec4e740 1 --2- 172.21.15.90:0/1685940385 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5601adc2fa80 0x5601adca8340 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.711+0000 7fc5dec4e740 1 --2- 172.21.15.90:0/1685940385 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601adc2f090 0x5601adc3e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.711+0000 7fc5dec4e740 1 -- 172.21.15.90:0/1685940385 >> 172.21.15.90:0/1685940385 conn(0x5601adc2a700 msgr2=0x5601adc36fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:41.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.711+0000 7fc5baa00640 0 client.25246 ms_handle_reset on v2:172.21.15.149:6838/652734263 2024-04-24T09:21:41.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.712+0000 7fc5dec4e740 1 -- 172.21.15.90:0/1685940385 shutdown_connections 2024-04-24T09:21:41.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.712+0000 7fc5dec4e740 1 -- 172.21.15.90:0/1685940385 wait complete. 2024-04-24T09:21:41.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.712+0000 7fc5dec4e740 20 client.25246 trim_cache size 0 max 16384 2024-04-24T09:21:41.717 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T09:21:41.717 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "30ee9be2-021a-11ef-bc93-c7b262605968", 2024-04-24T09:21:41.717 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 1, 2024-04-24T09:21:41.717 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 24547, 2024-04-24T09:21:41.717 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-04-24T09:21:41.717 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-04-24T09:21:41.717 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-04-24T09:21:41.717 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 148.44762240399999, 2024-04-24T09:21:41.717 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 27, 2024-04-24T09:21:41.717 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 88, 2024-04-24T09:21:41.717 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 86, 2024-04-24T09:21:41.717 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 169.25837872100001 2024-04-24T09:21:41.717 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T09:21:41.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.712+0000 7fc5dbe00640 1 -- 172.21.15.90:0/1816581313 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fc5bc0a3bb0 msgr2=0x7fc5bc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:41.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.712+0000 7fc5dbe00640 1 --2- 172.21.15.90:0/1816581313 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fc5bc0a3bb0 0x7fc5bc0a6070 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x5601adc302e0 tx=0x7fc5cc009500 comp rx=0 tx=0).stop 2024-04-24T09:21:41.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.712+0000 7fc5dbe00640 1 -- 172.21.15.90:0/1816581313 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5d4108660 msgr2=0x7fc5d419e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:41.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.712+0000 7fc5dbe00640 1 --2- 172.21.15.90:0/1816581313 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5d4108660 0x7fc5d419e960 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800e9c0 tx=0x7fc5c800ee80 comp rx=0 tx=0).stop 2024-04-24T09:21:41.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.712+0000 7fc5dbe00640 1 -- 172.21.15.90:0/1816581313 shutdown_connections 2024-04-24T09:21:41.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.712+0000 7fc5dbe00640 1 --2- 172.21.15.90:0/1816581313 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fc5bc0a3bb0 0x7fc5bc0a6070 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.712+0000 7fc5dbe00640 1 --2- 172.21.15.90:0/1816581313 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5d410a210 0x7fc5d41a5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.712+0000 7fc5dbe00640 1 --2- 172.21.15.90:0/1816581313 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc5d4109850 0x7fc5d419eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.712+0000 7fc5dbe00640 1 --2- 172.21.15.90:0/1816581313 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5d4108660 0x7fc5d419e960 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.712+0000 7fc5dbe00640 1 -- 172.21.15.90:0/1816581313 >> 172.21.15.90:0/1816581313 conn(0x7fc5d4077600 msgr2=0x7fc5d410cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:41.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.712+0000 7fc5dbe00640 1 -- 172.21.15.90:0/1816581313 shutdown_connections 2024-04-24T09:21:41.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.712+0000 7fc5dbe00640 1 -- 172.21.15.90:0/1816581313 wait complete. 2024-04-24T09:21:41.730 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-24T09:21:41.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.930+0000 7f844a600640 1 -- 172.21.15.90:0/2068564230 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8444105450 msgr2=0x7f84441058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:41.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.930+0000 7f844a600640 1 --2- 172.21.15.90:0/2068564230 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8444105450 0x7f84441058d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8430009900 tx=0x7f843002a3f0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.930+0000 7f844a600640 1 -- 172.21.15.90:0/2068564230 shutdown_connections 2024-04-24T09:21:41.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.930+0000 7f844a600640 1 --2- 172.21.15.90:0/2068564230 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8444105e10 0x7f844410c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.930+0000 7f844a600640 1 --2- 172.21.15.90:0/2068564230 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8444105450 0x7f84441058d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.930+0000 7f844a600640 1 --2- 172.21.15.90:0/2068564230 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8444104260 0x7f8444104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.930+0000 7f844a600640 1 -- 172.21.15.90:0/2068564230 >> 172.21.15.90:0/2068564230 conn(0x7f84440ff7b0 msgr2=0x7f8444101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:41.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.930+0000 7f844a600640 1 -- 172.21.15.90:0/2068564230 shutdown_connections 2024-04-24T09:21:41.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.930+0000 7f844a600640 1 -- 172.21.15.90:0/2068564230 wait complete. 2024-04-24T09:21:41.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.931+0000 7f844a600640 1 Processor -- start 2024-04-24T09:21:41.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.931+0000 7f844a600640 1 -- start start 2024-04-24T09:21:41.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.931+0000 7f844a600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8444104260 0x7f844419a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.931+0000 7f844a600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8444105450 0x7f844419aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.931+0000 7f844a600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8444105e10 0x7f84441a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.931+0000 7f844a600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f844410ff40 con 0x7f8444104260 2024-04-24T09:21:41.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.931+0000 7f844a600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f844410fdc0 con 0x7f8444105e10 2024-04-24T09:21:41.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.931+0000 7f844a600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f84441100c0 con 0x7f8444105450 2024-04-24T09:21:41.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.932+0000 7f8443400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8444104260 0x7f844419a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.932+0000 7f8443e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8444105e10 0x7f84441a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.932+0000 7f8442a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8444105450 0x7f844419aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.932+0000 7f8443e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8444105e10 0x7f84441a1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.90:56070/0 (socket says 172.21.15.90:56070) 2024-04-24T09:21:41.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.932+0000 7f8443e00640 1 -- 172.21.15.90:0/1649245321 learned_addr learned my addr 172.21.15.90:0/1649245321 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:41.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.932+0000 7f8443400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8444104260 0x7f844419a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42458/0 (socket says 172.21.15.90:42458) 2024-04-24T09:21:41.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.932+0000 7f8443e00640 1 -- 172.21.15.90:0/1649245321 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8444105450 msgr2=0x7f844419aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:41.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.932+0000 7f8443e00640 1 --2- 172.21.15.90:0/1649245321 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8444105450 0x7f844419aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.932+0000 7f8443e00640 1 -- 172.21.15.90:0/1649245321 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8444104260 msgr2=0x7f844419a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:41.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.932+0000 7f8443e00640 1 --2- 172.21.15.90:0/1649245321 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8444104260 0x7f844419a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.932+0000 7f8443e00640 1 -- 172.21.15.90:0/1649245321 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84441a2180 con 0x7f8444105e10 2024-04-24T09:21:41.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.932+0000 7f8443e00640 1 --2- 172.21.15.90:0/1649245321 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8444105e10 0x7f84441a1b10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f843c00eb10 tx=0x7f843c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:41.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.933+0000 7f843be00640 1 -- 172.21.15.90:0/1649245321 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c004b20 con 0x7f8444105e10 2024-04-24T09:21:41.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.933+0000 7f844a600640 1 -- 172.21.15.90:0/1649245321 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84441a2470 con 0x7f8444105e10 2024-04-24T09:21:41.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.933+0000 7f843be00640 1 -- 172.21.15.90:0/1649245321 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f843c004cc0 con 0x7f8444105e10 2024-04-24T09:21:41.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.933+0000 7f843be00640 1 -- 172.21.15.90:0/1649245321 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c010760 con 0x7f8444105e10 2024-04-24T09:21:41.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.933+0000 7f844a600640 1 -- 172.21.15.90:0/1649245321 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84441a2990 con 0x7f8444105e10 2024-04-24T09:21:41.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.936+0000 7f844d3ce740 1 Processor -- start 2024-04-24T09:21:41.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.936+0000 7f843be00640 1 -- 172.21.15.90:0/1649245321 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f843c005020 con 0x7f8444105e10 2024-04-24T09:21:41.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.936+0000 7f844d3ce740 1 -- start start 2024-04-24T09:21:41.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.937+0000 7f844d3ce740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a081065090 0x55a0810654b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.937+0000 7f844d3ce740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a081065a80 0x55a08106a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.937+0000 7f844d3ce740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a08106abf0 0x55a08106d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.937+0000 7f844d3ce740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55a080fcab10 con 0x55a081065a80 2024-04-24T09:21:41.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.937+0000 7f844d3ce740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55a080f712b0 con 0x55a08106abf0 2024-04-24T09:21:41.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.937+0000 7f844d3ce740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a080fc6ad0 con 0x55a081065090 2024-04-24T09:21:41.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.937+0000 7f8443400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a081065a80 0x55a08106a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.937+0000 7f8442a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a08106abf0 0x55a08106d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.937+0000 7f8443400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a081065a80 0x55a08106a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42474/0 (socket says 172.21.15.90:42474) 2024-04-24T09:21:41.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.937+0000 7f8443400640 1 -- 172.21.15.90:0/754859148 learned_addr learned my addr 172.21.15.90:0/754859148 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:41.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.937+0000 7f8442a00640 1 --2- 172.21.15.90:0/754859148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a081065090 0x55a0810654b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.937+0000 7f8443400640 1 -- 172.21.15.90:0/754859148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a081065090 msgr2=0x55a0810654b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:41.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.937+0000 7f8443400640 1 --2- 172.21.15.90:0/754859148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a081065090 0x55a0810654b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.937+0000 7f8443400640 1 -- 172.21.15.90:0/754859148 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a08106abf0 msgr2=0x55a08106d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:41.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.937+0000 7f8443400640 1 --2- 172.21.15.90:0/754859148 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a08106abf0 0x55a08106d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.937+0000 7f8443400640 1 -- 172.21.15.90:0/754859148 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a080cdd2d0 con 0x55a081065a80 2024-04-24T09:21:41.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.937+0000 7f8442a00640 1 --2- 172.21.15.90:0/754859148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a081065090 0x55a0810654b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:21:41.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.938+0000 7f843be00640 1 --2- 172.21.15.90:0/1649245321 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f84280a3bb0 0x7f84280a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.938+0000 7f8443400640 1 --2- 172.21.15.90:0/754859148 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a081065a80 0x55a08106a580 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8434009ca0 tx=0x7f8434011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a773eafcb8c3db71 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:41.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.938+0000 7f843be00640 1 -- 172.21.15.90:0/1649245321 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f843c0d0a90 con 0x7f8444105e10 2024-04-24T09:21:41.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.938+0000 7f8442a00640 1 --2- 172.21.15.90:0/1649245321 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f84280a3bb0 0x7f84280a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.938+0000 7f8438c00640 1 -- 172.21.15.90:0/754859148 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8434004220 con 0x55a081065a80 2024-04-24T09:21:41.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.938+0000 7f8438c00640 1 -- 172.21.15.90:0/754859148 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84340043c0 con 0x55a081065a80 2024-04-24T09:21:41.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.938+0000 7f8438c00640 1 -- 172.21.15.90:0/754859148 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8434019930 con 0x55a081065a80 2024-04-24T09:21:41.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.939+0000 7f844d3ce740 1 -- 172.21.15.90:0/754859148 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a081065a80 msgr2=0x55a08106a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:41.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.939+0000 7f8442a00640 1 --2- 172.21.15.90:0/1649245321 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f84280a3bb0 0x7f84280a6070 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8430031440 tx=0x7f8430041040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:41.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.939+0000 7f844d3ce740 1 --2- 172.21.15.90:0/754859148 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a081065a80 0x55a08106a580 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8434009ca0 tx=0x7f8434011810 comp rx=0 tx=0).stop 2024-04-24T09:21:41.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.939+0000 7f844d3ce740 1 -- 172.21.15.90:0/754859148 shutdown_connections 2024-04-24T09:21:41.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.939+0000 7f844d3ce740 1 --2- 172.21.15.90:0/754859148 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a08106abf0 0x55a08106d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.939+0000 7f844d3ce740 1 --2- 172.21.15.90:0/754859148 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a081065a80 0x55a08106a580 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.939+0000 7f844d3ce740 1 --2- 172.21.15.90:0/754859148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a081065090 0x55a0810654b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.939+0000 7f844d3ce740 1 -- 172.21.15.90:0/754859148 >> 172.21.15.90:0/754859148 conn(0x55a081060700 msgr2=0x55a081060b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:41.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.939+0000 7f844d3ce740 1 -- 172.21.15.90:0/754859148 shutdown_connections 2024-04-24T09:21:41.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.939+0000 7f844d3ce740 1 -- 172.21.15.90:0/754859148 wait complete. 2024-04-24T09:21:41.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.940+0000 7f844d3ce740 1 -- start start 2024-04-24T09:21:41.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.940+0000 7f844d3ce740 1 Processor -- start 2024-04-24T09:21:41.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.940+0000 7f844d3ce740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a081065090 0x55a08106f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.940+0000 7f844d3ce740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a081065a80 0x55a08106f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.940+0000 7f844d3ce740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a08106abf0 0x55a081076a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.940+0000 7f844d3ce740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55a08106dba0 con 0x55a081065090 2024-04-24T09:21:41.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.940+0000 7f844d3ce740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55a080f84590 con 0x55a081065a80 2024-04-24T09:21:41.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.940+0000 7f844d3ce740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a08106dd20 con 0x55a08106abf0 2024-04-24T09:21:41.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.941+0000 7f844d3ce740 10 client.0 _ll_register_callbacks cb 0x55a080f673a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:21:41.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.941+0000 7f8443400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a081065090 0x55a08106f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.941+0000 7f8442a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a081065a80 0x55a08106f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.941+0000 7f8443400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a081065090 0x55a08106f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42488/0 (socket says 172.21.15.90:42488) 2024-04-24T09:21:41.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.941+0000 7f8443400640 1 -- 172.21.15.90:0/3409326598 learned_addr learned my addr 172.21.15.90:0/3409326598 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:41.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.941+0000 7f8442a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a081065a80 0x55a08106f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.90:56090/0 (socket says 172.21.15.90:56090) 2024-04-24T09:21:41.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.941+0000 7f8443e00640 1 --2- 172.21.15.90:0/3409326598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a08106abf0 0x55a081076a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.941+0000 7f8443400640 1 -- 172.21.15.90:0/3409326598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a08106abf0 msgr2=0x55a081076a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:41.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.941+0000 7f8443400640 1 --2- 172.21.15.90:0/3409326598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a08106abf0 0x55a081076a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.941+0000 7f8443400640 1 -- 172.21.15.90:0/3409326598 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a081065a80 msgr2=0x55a08106f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:41.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.941+0000 7f8443400640 1 --2- 172.21.15.90:0/3409326598 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a081065a80 0x55a08106f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.941+0000 7f8443400640 1 -- 172.21.15.90:0/3409326598 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a080da76b0 con 0x55a081065090 2024-04-24T09:21:41.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.941+0000 7f8443400640 1 --2- 172.21.15.90:0/3409326598 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a081065090 0x55a08106f4b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8434011d10 tx=0x7f8434003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10c9718fef57d67f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:41.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.941+0000 7f842d600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-24T09:21:41.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.942+0000 7f842d600640 1 -- 172.21.15.90:0/3409326598 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8434028070 con 0x55a081065090 2024-04-24T09:21:41.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.942+0000 7f842d600640 1 -- 172.21.15.90:0/3409326598 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f8434005040 con 0x55a081065090 2024-04-24T09:21:41.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.942+0000 7f844d3ce740 1 -- 172.21.15.90:0/3409326598 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84340043c0 con 0x55a081065090 2024-04-24T09:21:41.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.942+0000 7f842d600640 1 -- 172.21.15.90:0/3409326598 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8434034050 con 0x55a081065090 2024-04-24T09:21:41.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.942+0000 7f844d3ce740 10 client.15150 fetch_fsmap learned FSMap version 27 2024-04-24T09:21:41.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.942+0000 7f844d3ce740 1 -- 172.21.15.90:0/3409326598 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a081079da0 con 0x55a081065090 2024-04-24T09:21:41.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.942+0000 7f842d600640 1 -- 172.21.15.90:0/3409326598 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84340043c0 con 0x55a081065090 2024-04-24T09:21:41.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.942+0000 7f842d600640 1 -- 172.21.15.90:0/3409326598 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8434004a80 con 0x55a081065090 2024-04-24T09:21:41.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.943+0000 7f844d3ce740 10 client.15150 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T09:21:41.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.943+0000 7f844d3ce740 10 client.15150 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T09:21:41.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.943+0000 7f844d3ce740 20 client.15150 populate_metadata read hostname 'smithi090' 2024-04-24T09:21:41.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.943+0000 7f844d3ce740 1 --2- 172.21.15.90:0/3409326598 >> [v2:172.21.15.114:6838/1025403710,v1:172.21.15.114:6839/1025403710] conn(0x55a08107c1b0 0x55a08107d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.943+0000 7f844d3ce740 4 client.15150 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:21:41.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.943+0000 7f844d3ce740 1 -- 172.21.15.90:0/3409326598 --> [v2:172.21.15.114:6838/1025403710,v1:172.21.15.114:6839/1025403710] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a080fd8530 con 0x55a08107c1b0 2024-04-24T09:21:41.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.943+0000 7f8442a00640 1 --2- 172.21.15.90:0/3409326598 >> [v2:172.21.15.114:6838/1025403710,v1:172.21.15.114:6839/1025403710] conn(0x55a08107c1b0 0x55a08107d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.944+0000 7f8442a00640 1 --2- 172.21.15.90:0/3409326598 >> [v2:172.21.15.114:6838/1025403710,v1:172.21.15.114:6839/1025403710] conn(0x55a08107c1b0 0x55a08107d620 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f9510358863caaa server_cookie=50d39c702639e1e1 in_seq=0 out_seq=0 2024-04-24T09:21:41.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.944+0000 7f842d600640 10 client.15150 ms_handle_connect on v2:172.21.15.114:6838/1025403710 2024-04-24T09:21:41.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.944+0000 7f842d600640 1 -- 172.21.15.90:0/3409326598 <== mds.0 v2:172.21.15.114:6838/1025403710 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a080fd8530 con 0x55a08107c1b0 2024-04-24T09:21:41.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.944+0000 7f842d600640 10 client.15150 handle_command_reply: tid=0 2024-04-24T09:21:41.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.945+0000 7f844d3ce740 1 client.15150 shutdown 2024-04-24T09:21:41.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.945+0000 7f844d3ce740 1 -- 172.21.15.90:0/3409326598 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a081065090 msgr2=0x55a08106f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:41.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.945+0000 7f844d3ce740 1 --2- 172.21.15.90:0/3409326598 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a081065090 0x55a08106f4b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8434011d10 tx=0x7f8434003820 comp rx=0 tx=0).stop 2024-04-24T09:21:41.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.945+0000 7f844d3ce740 1 -- 172.21.15.90:0/3409326598 shutdown_connections 2024-04-24T09:21:41.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.945+0000 7f844d3ce740 1 --2- 172.21.15.90:0/3409326598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a08106abf0 0x55a081076a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.945+0000 7f844d3ce740 1 --2- 172.21.15.90:0/3409326598 >> [v2:172.21.15.114:6838/1025403710,v1:172.21.15.114:6839/1025403710] conn(0x55a08107c1b0 0x55a08107d620 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.945+0000 7f844d3ce740 1 --2- 172.21.15.90:0/3409326598 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a081065a80 0x55a08106f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.945+0000 7f844d3ce740 1 --2- 172.21.15.90:0/3409326598 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a081065090 0x55a08106f4b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.945+0000 7f842d600640 0 client.15150 ms_handle_reset on v2:172.21.15.114:6838/1025403710 2024-04-24T09:21:41.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.945+0000 7f844d3ce740 1 -- 172.21.15.90:0/3409326598 >> 172.21.15.90:0/3409326598 conn(0x55a081060700 msgr2=0x55a08105f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:41.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.945+0000 7f844d3ce740 1 -- 172.21.15.90:0/3409326598 shutdown_connections 2024-04-24T09:21:41.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.945+0000 7f844d3ce740 1 -- 172.21.15.90:0/3409326598 wait complete. 2024-04-24T09:21:41.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.945+0000 7f844d3ce740 20 client.15150 trim_cache size 0 max 16384 2024-04-24T09:21:41.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.965+0000 7f844d3ce740 1 Processor -- start 2024-04-24T09:21:41.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.965+0000 7f844d3ce740 1 -- start start 2024-04-24T09:21:41.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.965+0000 7f844d3ce740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a081065090 0x55a08106cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.965+0000 7f844d3ce740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a081065a80 0x55a0810769f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.965+0000 7f844d3ce740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a08106abf0 0x55a08105b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.965+0000 7f844d3ce740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55a08107b960 con 0x55a08106abf0 2024-04-24T09:21:41.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.965+0000 7f844d3ce740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55a08107c030 con 0x55a081065a80 2024-04-24T09:21:41.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.965+0000 7f844d3ce740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a08107c700 con 0x55a081065090 2024-04-24T09:21:41.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.966+0000 7f8443e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a08106abf0 0x55a08105b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.966+0000 7f8442a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a081065a80 0x55a0810769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.966+0000 7f8443400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a081065090 0x55a08106cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.966+0000 7f8443e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a08106abf0 0x55a08105b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42494/0 (socket says 172.21.15.90:42494) 2024-04-24T09:21:41.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.966+0000 7f8443e00640 1 -- 172.21.15.90:0/4281724417 learned_addr learned my addr 172.21.15.90:0/4281724417 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:41.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.966+0000 7f8443e00640 1 -- 172.21.15.90:0/4281724417 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a081065090 msgr2=0x55a08106cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:41.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.966+0000 7f8443e00640 1 --2- 172.21.15.90:0/4281724417 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a081065090 0x55a08106cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.966+0000 7f8443e00640 1 -- 172.21.15.90:0/4281724417 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a081065a80 msgr2=0x55a0810769f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:41.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.966+0000 7f8443e00640 1 --2- 172.21.15.90:0/4281724417 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a081065a80 0x55a0810769f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.966+0000 7f8443e00640 1 -- 172.21.15.90:0/4281724417 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a081065e60 con 0x55a08106abf0 2024-04-24T09:21:41.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.966+0000 7f8443400640 1 --2- 172.21.15.90:0/4281724417 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a081065090 0x55a08106cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:21:41.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.966+0000 7f8442a00640 1 --2- 172.21.15.90:0/4281724417 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a081065a80 0x55a0810769f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:21:41.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.966+0000 7f8443e00640 1 --2- 172.21.15.90:0/4281724417 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a08106abf0 0x55a08105b750 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f843c09cd00 tx=0x7f843c09cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a21324c6a57c74 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:41.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.967+0000 7f842fe00640 1 -- 172.21.15.90:0/4281724417 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c09cd70 con 0x55a08106abf0 2024-04-24T09:21:41.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.967+0000 7f842fe00640 1 -- 172.21.15.90:0/4281724417 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f843c0b6070 con 0x55a08106abf0 2024-04-24T09:21:41.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.967+0000 7f842fe00640 1 -- 172.21.15.90:0/4281724417 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c005310 con 0x55a08106abf0 2024-04-24T09:21:41.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.967+0000 7f844d3ce740 1 -- 172.21.15.90:0/4281724417 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a08106abf0 msgr2=0x55a08105b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:41.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.967+0000 7f844d3ce740 1 --2- 172.21.15.90:0/4281724417 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a08106abf0 0x55a08105b750 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f843c09cd00 tx=0x7f843c09cc10 comp rx=0 tx=0).stop 2024-04-24T09:21:41.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.967+0000 7f844d3ce740 1 -- 172.21.15.90:0/4281724417 shutdown_connections 2024-04-24T09:21:41.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.967+0000 7f844d3ce740 1 --2- 172.21.15.90:0/4281724417 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a08106abf0 0x55a08105b750 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.967+0000 7f844d3ce740 1 --2- 172.21.15.90:0/4281724417 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a081065a80 0x55a0810769f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.967+0000 7f844d3ce740 1 --2- 172.21.15.90:0/4281724417 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a081065090 0x55a08106cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.967+0000 7f844d3ce740 1 -- 172.21.15.90:0/4281724417 >> 172.21.15.90:0/4281724417 conn(0x55a081060700 msgr2=0x55a08107b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:41.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.968+0000 7f844d3ce740 1 -- 172.21.15.90:0/4281724417 shutdown_connections 2024-04-24T09:21:41.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.968+0000 7f844d3ce740 1 -- 172.21.15.90:0/4281724417 wait complete. 2024-04-24T09:21:41.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.968+0000 7f844d3ce740 1 -- start start 2024-04-24T09:21:41.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.968+0000 7f844d3ce740 1 Processor -- start 2024-04-24T09:21:41.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.968+0000 7f844d3ce740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a081065090 0x55a081072b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.968+0000 7f844d3ce740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a081065a80 0x55a0810730c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.968+0000 7f844d3ce740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a08106abf0 0x55a081075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.969+0000 7f844d3ce740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55a08107d4a0 con 0x55a081065a80 2024-04-24T09:21:41.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.969+0000 7f844d3ce740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55a08107cdd0 con 0x55a081065090 2024-04-24T09:21:41.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.969+0000 7f844d3ce740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a08107db70 con 0x55a08106abf0 2024-04-24T09:21:41.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.969+0000 7f844d3ce740 10 client.0 _ll_register_callbacks cb 0x7f8434004230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:21:41.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.969+0000 7f8442a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a081065a80 0x55a0810730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.969+0000 7f8442a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a081065a80 0x55a0810730c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42504/0 (socket says 172.21.15.90:42504) 2024-04-24T09:21:41.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.969+0000 7f8442a00640 1 -- 172.21.15.90:0/1364091321 learned_addr learned my addr 172.21.15.90:0/1364091321 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:41.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.969+0000 7f8443e00640 1 --2- 172.21.15.90:0/1364091321 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a08106abf0 0x55a081075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.969+0000 7f8443400640 1 --2- 172.21.15.90:0/1364091321 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a081065090 0x55a081072b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.969+0000 7f8442a00640 1 -- 172.21.15.90:0/1364091321 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a08106abf0 msgr2=0x55a081075bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:41.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.969+0000 7f8442a00640 1 --2- 172.21.15.90:0/1364091321 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a08106abf0 0x55a081075bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.969+0000 7f8442a00640 1 -- 172.21.15.90:0/1364091321 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a081065090 msgr2=0x55a081072b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:21:41.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.969+0000 7f8442a00640 1 --2- 172.21.15.90:0/1364091321 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a081065090 0x55a081072b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.969+0000 7f8442a00640 1 -- 172.21.15.90:0/1364091321 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a081076200 con 0x55a081065a80 2024-04-24T09:21:41.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.970+0000 7f8442a00640 1 --2- 172.21.15.90:0/1364091321 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a081065a80 0x55a0810730c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8430009900 tx=0x7f8430009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52f1abbba4bdf260 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:41.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.970+0000 7f842d600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-24T09:21:41.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.970+0000 7f842d600640 1 -- 172.21.15.90:0/1364091321 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8430046070 con 0x55a081065a80 2024-04-24T09:21:41.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.970+0000 7f842d600640 1 -- 172.21.15.90:0/1364091321 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f8430056870 con 0x55a081065a80 2024-04-24T09:21:41.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.970+0000 7f844d3ce740 1 -- 172.21.15.90:0/1364091321 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f843c0b6070 con 0x55a081065a80 2024-04-24T09:21:41.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.970+0000 7f842d600640 1 -- 172.21.15.90:0/1364091321 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f84300035f0 con 0x55a081065a80 2024-04-24T09:21:41.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.970+0000 7f844d3ce740 10 client.15156 fetch_fsmap learned FSMap version 27 2024-04-24T09:21:41.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.970+0000 7f844d3ce740 1 -- 172.21.15.90:0/1364091321 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a0810e2740 con 0x55a081065a80 2024-04-24T09:21:41.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.970+0000 7f842d600640 1 -- 172.21.15.90:0/1364091321 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f843c0b6070 con 0x55a081065a80 2024-04-24T09:21:41.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.971+0000 7f842d600640 1 -- 172.21.15.90:0/1364091321 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f843003d7e0 con 0x55a081065a80 2024-04-24T09:21:41.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.971+0000 7f844d3ce740 10 client.15156 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T09:21:41.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.971+0000 7f844d3ce740 10 client.15156 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T09:21:41.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.971+0000 7f844d3ce740 20 client.15156 populate_metadata read hostname 'smithi090' 2024-04-24T09:21:41.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.971+0000 7f844d3ce740 1 --2- 172.21.15.90:0/1364091321 >> [v2:172.21.15.114:6838/1025403710,v1:172.21.15.114:6839/1025403710] conn(0x55a0810e7350 0x55a0810e6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:41.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.971+0000 7f844d3ce740 4 client.15156 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T09:21:41.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.971+0000 7f844d3ce740 1 -- 172.21.15.90:0/1364091321 --> [v2:172.21.15.114:6838/1025403710,v1:172.21.15.114:6839/1025403710] -- command(tid 0: {"prefix": "status"}) -- 0x55a080fd23a0 con 0x55a0810e7350 2024-04-24T09:21:41.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.971+0000 7f8443400640 1 --2- 172.21.15.90:0/1364091321 >> [v2:172.21.15.114:6838/1025403710,v1:172.21.15.114:6839/1025403710] conn(0x55a0810e7350 0x55a0810e6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:41.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.972+0000 7f842d600640 10 client.15156 ms_handle_connect on v2:172.21.15.114:6838/1025403710 2024-04-24T09:21:41.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.972+0000 7f8443400640 1 --2- 172.21.15.90:0/1364091321 >> [v2:172.21.15.114:6838/1025403710,v1:172.21.15.114:6839/1025403710] conn(0x55a0810e7350 0x55a0810e6060 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c322fd38cfb4747b server_cookie=352cbfe1c520d72c in_seq=0 out_seq=0 2024-04-24T09:21:41.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.972+0000 7f842d600640 1 -- 172.21.15.90:0/1364091321 <== mds.0 v2:172.21.15.114:6838/1025403710 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a080fd23a0 con 0x55a0810e7350 2024-04-24T09:21:41.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.972+0000 7f842d600640 10 client.15156 handle_command_reply: tid=0 2024-04-24T09:21:41.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.972+0000 7f844d3ce740 1 client.15156 shutdown 2024-04-24T09:21:41.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.972+0000 7f844d3ce740 1 -- 172.21.15.90:0/1364091321 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a081065a80 msgr2=0x55a0810730c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:41.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.972+0000 7f844d3ce740 1 --2- 172.21.15.90:0/1364091321 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a081065a80 0x55a0810730c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8430009900 tx=0x7f8430009260 comp rx=0 tx=0).stop 2024-04-24T09:21:41.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.972+0000 7f844d3ce740 1 -- 172.21.15.90:0/1364091321 shutdown_connections 2024-04-24T09:21:41.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.972+0000 7f844d3ce740 1 --2- 172.21.15.90:0/1364091321 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a08106abf0 0x55a081075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.972+0000 7f844d3ce740 1 --2- 172.21.15.90:0/1364091321 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a081065a80 0x55a0810730c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.972+0000 7f844d3ce740 1 --2- 172.21.15.90:0/1364091321 >> [v2:172.21.15.114:6838/1025403710,v1:172.21.15.114:6839/1025403710] conn(0x55a0810e7350 0x55a0810e6060 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.972+0000 7f844d3ce740 1 --2- 172.21.15.90:0/1364091321 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a081065090 0x55a081072b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.972+0000 7f842d600640 0 client.15156 ms_handle_reset on v2:172.21.15.114:6838/1025403710 2024-04-24T09:21:41.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.972+0000 7f844d3ce740 1 -- 172.21.15.90:0/1364091321 >> 172.21.15.90:0/1364091321 conn(0x55a081060700 msgr2=0x55a08105b750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:41.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.972+0000 7f844d3ce740 1 -- 172.21.15.90:0/1364091321 shutdown_connections 2024-04-24T09:21:41.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.972+0000 7f844d3ce740 1 -- 172.21.15.90:0/1364091321 wait complete. 2024-04-24T09:21:41.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.973+0000 7f844d3ce740 20 client.15156 trim_cache size 0 max 16384 2024-04-24T09:21:41.976 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T09:21:41.976 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "30ee9be2-021a-11ef-bc93-c7b262605968", 2024-04-24T09:21:41.976 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 0, 2024-04-24T09:21:41.976 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 24557, 2024-04-24T09:21:41.976 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-04-24T09:21:41.977 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-04-24T09:21:41.977 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-04-24T09:21:41.977 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 152.02870518700001, 2024-04-24T09:21:41.977 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 27, 2024-04-24T09:21:41.977 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 88, 2024-04-24T09:21:41.977 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 86, 2024-04-24T09:21:41.977 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 158.171709892 2024-04-24T09:21:41.977 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T09:21:41.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.973+0000 7f844a600640 1 -- 172.21.15.90:0/1649245321 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f84280a3bb0 msgr2=0x7f84280a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:41.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.973+0000 7f844a600640 1 --2- 172.21.15.90:0/1649245321 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f84280a3bb0 0x7f84280a6070 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8430031440 tx=0x7f8430041040 comp rx=0 tx=0).stop 2024-04-24T09:21:41.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.973+0000 7f844a600640 1 -- 172.21.15.90:0/1649245321 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8444105e10 msgr2=0x7f84441a1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:41.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.973+0000 7f844a600640 1 --2- 172.21.15.90:0/1649245321 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8444105e10 0x7f84441a1b10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f843c00eb10 tx=0x7f843c00c2d0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.973+0000 7f844a600640 1 -- 172.21.15.90:0/1649245321 shutdown_connections 2024-04-24T09:21:41.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.973+0000 7f844a600640 1 --2- 172.21.15.90:0/1649245321 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f84280a3bb0 0x7f84280a6070 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.973+0000 7f844a600640 1 --2- 172.21.15.90:0/1649245321 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8444105e10 0x7f84441a1b10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.973+0000 7f844a600640 1 --2- 172.21.15.90:0/1649245321 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8444105450 0x7f844419aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.973+0000 7f844a600640 1 --2- 172.21.15.90:0/1649245321 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8444104260 0x7f844419a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:41.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.973+0000 7f844a600640 1 -- 172.21.15.90:0/1649245321 >> 172.21.15.90:0/1649245321 conn(0x7f84440ff7b0 msgr2=0x7f8444108750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:41.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.973+0000 7f844a600640 1 -- 172.21.15.90:0/1649245321 shutdown_connections 2024-04-24T09:21:41.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:41.973+0000 7f844a600640 1 -- 172.21.15.90:0/1649245321 wait complete. 2024-04-24T09:21:41.990 INFO:teuthology.run_tasks:Running task sequential... 2024-04-24T09:21:42.004 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-24T09:21:42.004 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-24T09:21:42.004 INFO:teuthology.task.print:Enabling mgr modules 2024-04-24T09:21:42.004 INFO:teuthology.run_tasks:Running task kclient... 2024-04-24T09:21:42.036 INFO:tasks.kclient:Mounting kernel clients... 2024-04-24T09:21:42.036 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-24T09:21:42.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:21:42.037 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list 2024-04-24T09:21:42.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:21:42.093 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link delete ceph-brx 2024-04-24T09:21:42.181 INFO:teuthology.orchestra.run.smithi090.stderr:Cannot find device "ceph-brx" 2024-04-24T09:21:42.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T09:21:42.185 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-04-24T09:21:42.185 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-24T09:21:42.186 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T09:21:42.186 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-24T09:21:42.616 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:21:42.948 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:42 smithi090 ceph-mon[27825]: pgmap v400: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T09:21:43.114 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:42 smithi149 ceph-mon[32592]: pgmap v400: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T09:21:43.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:42 smithi114 ceph-mon[32501]: pgmap v400: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T09:21:44.906 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:44 smithi090 ceph-mon[27825]: pgmap v401: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:21:45.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:44 smithi149 ceph-mon[32592]: pgmap v401: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:21:45.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:44 smithi114 ceph-mon[32501]: pgmap v401: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:21:45.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.190+0000 7f4a04800700 1 -- 172.21.15.90:0/4120412601 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49fc104f70 msgr2=0x7f49fc105350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:45.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.190+0000 7f4a04800700 1 --2- 172.21.15.90:0/4120412601 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49fc104f70 0x7f49fc105350 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f49f00099c0 tx=0x7f49f0009cd0 comp rx=0 tx=0).stop 2024-04-24T09:21:45.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.190+0000 7f4a04800700 1 -- 172.21.15.90:0/4120412601 shutdown_connections 2024-04-24T09:21:45.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.190+0000 7f4a04800700 1 --2- 172.21.15.90:0/4120412601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49fc10a050 0x7f49fc111c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:45.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.190+0000 7f4a04800700 1 --2- 172.21.15.90:0/4120412601 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49fc104f70 0x7f49fc105350 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:45.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.190+0000 7f4a04800700 1 --2- 172.21.15.90:0/4120412601 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f49fc105920 0x7f49fc109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:45.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.190+0000 7f4a04800700 1 -- 172.21.15.90:0/4120412601 >> 172.21.15.90:0/4120412601 conn(0x7f49fc100800 msgr2=0x7f49fc102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:45.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.190+0000 7f4a04800700 1 -- 172.21.15.90:0/4120412601 shutdown_connections 2024-04-24T09:21:45.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.190+0000 7f4a04800700 1 -- 172.21.15.90:0/4120412601 wait complete. 2024-04-24T09:21:45.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.191+0000 7f4a04800700 1 Processor -- start 2024-04-24T09:21:45.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.192+0000 7f4a04800700 1 -- start start 2024-04-24T09:21:45.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.192+0000 7f4a04800700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49fc104f70 0x7f49fc19b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:45.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.192+0000 7f4a04800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f49fc105920 0x7f49fc19ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:45.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.192+0000 7f4a04800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49fc10a050 0x7f49fc19fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:45.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.192+0000 7f4a04800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f49fc115030 con 0x7f49fc10a050 2024-04-24T09:21:45.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.192+0000 7f4a04800700 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f49fc114eb0 con 0x7f49fc104f70 2024-04-24T09:21:45.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.192+0000 7f4a04800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f49fc1151b0 con 0x7f49fc105920 2024-04-24T09:21:45.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.192+0000 7f4a02800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49fc10a050 0x7f49fc19fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:45.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.192+0000 7f4a01400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f49fc105920 0x7f49fc19ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:45.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.192+0000 7f4a01400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f49fc105920 0x7f49fc19ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:54706/0 (socket says 172.21.15.90:54706) 2024-04-24T09:21:45.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.192+0000 7f4a01400700 1 -- 172.21.15.90:0/380340518 learned_addr learned my addr 172.21.15.90:0/380340518 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:45.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.192+0000 7f4a01e00700 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49fc104f70 0x7f49fc19b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:45.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.193+0000 7f4a01400700 1 -- 172.21.15.90:0/380340518 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49fc104f70 msgr2=0x7f49fc19b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:45.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.193+0000 7f4a01400700 1 --2- 172.21.15.90:0/380340518 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49fc104f70 0x7f49fc19b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:45.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.193+0000 7f4a01400700 1 -- 172.21.15.90:0/380340518 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49fc10a050 msgr2=0x7f49fc19fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:45.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.193+0000 7f4a01400700 1 --2- 172.21.15.90:0/380340518 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49fc10a050 0x7f49fc19fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:45.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.193+0000 7f4a01400700 1 -- 172.21.15.90:0/380340518 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49fc1a0510 con 0x7f49fc105920 2024-04-24T09:21:45.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.193+0000 7f4a02800700 1 --2- 172.21.15.90:0/380340518 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49fc10a050 0x7f49fc19fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:21:45.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.193+0000 7f4a01e00700 1 --2- 172.21.15.90:0/380340518 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49fc104f70 0x7f49fc19b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:21:45.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.193+0000 7f4a01400700 1 --2- 172.21.15.90:0/380340518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f49fc105920 0x7f49fc19ba40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f49ec00e970 tx=0x7f49ec00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:45.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.193+0000 7f49f6a00700 1 -- 172.21.15.90:0/380340518 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ec004a30 con 0x7f49fc105920 2024-04-24T09:21:45.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.194+0000 7f49f6a00700 1 -- 172.21.15.90:0/380340518 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49ec00cca0 con 0x7f49fc105920 2024-04-24T09:21:45.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.194+0000 7f49f6a00700 1 -- 172.21.15.90:0/380340518 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ec005680 con 0x7f49fc105920 2024-04-24T09:21:45.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.194+0000 7f4a04800700 1 -- 172.21.15.90:0/380340518 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49fc1a0800 con 0x7f49fc105920 2024-04-24T09:21:45.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.194+0000 7f4a04800700 1 -- 172.21.15.90:0/380340518 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49fc1a8080 con 0x7f49fc105920 2024-04-24T09:21:45.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.195+0000 7f4a04800700 1 -- 172.21.15.90:0/380340518 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49fc0697e0 con 0x7f49fc105920 2024-04-24T09:21:45.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.197+0000 7f49f6a00700 1 -- 172.21.15.90:0/380340518 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f49ec00c7c0 con 0x7f49fc105920 2024-04-24T09:21:45.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.198+0000 7f49f6a00700 1 --2- 172.21.15.90:0/380340518 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f49e40a3c40 0x7f49e40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:45.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.198+0000 7f49f6a00700 1 -- 172.21.15.90:0/380340518 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f49ec0d0810 con 0x7f49fc105920 2024-04-24T09:21:45.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.198+0000 7f4a01e00700 1 --2- 172.21.15.90:0/380340518 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f49e40a3c40 0x7f49e40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:45.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.199+0000 7f4a01e00700 1 --2- 172.21.15.90:0/380340518 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f49e40a3c40 0x7f49e40a6100 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f49f0000c00 tx=0x7f49f0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:45.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.200+0000 7f49f6a00700 1 -- 172.21.15.90:0/380340518 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f49ec053ff0 con 0x7f49fc105920 2024-04-24T09:21:45.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.475+0000 7f4a04800700 1 -- 172.21.15.90:0/380340518 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f49fc02dfe0 con 0x7f49fc105920 2024-04-24T09:21:45.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.476+0000 7f49f6a00700 1 -- 172.21.15.90:0/380340518 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7f49ec09d1b0 con 0x7f49fc105920 2024-04-24T09:21:45.476 INFO:teuthology.orchestra.run.smithi090.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-24T09:21:45.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.479+0000 7f4a04800700 1 -- 172.21.15.90:0/380340518 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f49e40a3c40 msgr2=0x7f49e40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:45.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.479+0000 7f4a04800700 1 --2- 172.21.15.90:0/380340518 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f49e40a3c40 0x7f49e40a6100 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f49f0000c00 tx=0x7f49f0005fb0 comp rx=0 tx=0).stop 2024-04-24T09:21:45.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.479+0000 7f4a04800700 1 -- 172.21.15.90:0/380340518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f49fc105920 msgr2=0x7f49fc19ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:45.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.479+0000 7f4a04800700 1 --2- 172.21.15.90:0/380340518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f49fc105920 0x7f49fc19ba40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f49ec00e970 tx=0x7f49ec00ec80 comp rx=0 tx=0).stop 2024-04-24T09:21:45.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.479+0000 7f4a04800700 1 -- 172.21.15.90:0/380340518 shutdown_connections 2024-04-24T09:21:45.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.479+0000 7f4a04800700 1 --2- 172.21.15.90:0/380340518 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f49e40a3c40 0x7f49e40a6100 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:45.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.479+0000 7f4a04800700 1 --2- 172.21.15.90:0/380340518 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49fc10a050 0x7f49fc19fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:45.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.479+0000 7f4a04800700 1 --2- 172.21.15.90:0/380340518 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f49fc104f70 0x7f49fc19b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:45.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.479+0000 7f4a04800700 1 --2- 172.21.15.90:0/380340518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f49fc105920 0x7f49fc19ba40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:45.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.479+0000 7f4a04800700 1 -- 172.21.15.90:0/380340518 >> 172.21.15.90:0/380340518 conn(0x7f49fc100800 msgr2=0x7f49fc102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:45.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.480+0000 7f4a04800700 1 -- 172.21.15.90:0/380340518 shutdown_connections 2024-04-24T09:21:45.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:45.480+0000 7f4a04800700 1 -- 172.21.15.90:0/380340518 wait complete. 2024-04-24T09:21:45.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:45 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/380340518' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T09:21:46.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:45 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/380340518' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T09:21:46.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:45 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/380340518' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T09:21:46.455 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T09:21:46.455 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T09:21:46.455 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi090.front.sepia.ceph.com 2024-04-24T09:21:46.455 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T09:21:46.455 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T09:21:46.455 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T09:21:46.455 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T09:21:46.455 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T09:21:46.455 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T09:21:46.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:21:46.456 DEBUG:teuthology.orchestra.run.smithi090:> ip addr 2024-04-24T09:21:46.510 INFO:teuthology.orchestra.run.smithi090.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T09:21:46.510 INFO:teuthology.orchestra.run.smithi090.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T09:21:46.511 INFO:teuthology.orchestra.run.smithi090.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T09:21:46.511 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-04-24T09:21:46.511 INFO:teuthology.orchestra.run.smithi090.stdout: inet6 ::1/128 scope host 2024-04-24T09:21:46.511 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-04-24T09:21:46.511 INFO:teuthology.orchestra.run.smithi090.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T09:21:46.511 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:d8:a1:a8 brd ff:ff:ff:ff:ff:ff 2024-04-24T09:21:46.511 INFO:teuthology.orchestra.run.smithi090.stdout: altname enp7s0f0 2024-04-24T09:21:46.511 INFO:teuthology.orchestra.run.smithi090.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T09:21:46.511 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:d8:a1:a9 brd ff:ff:ff:ff:ff:ff 2024-04-24T09:21:46.511 INFO:teuthology.orchestra.run.smithi090.stdout: altname enp7s0f1 2024-04-24T09:21:46.511 INFO:teuthology.orchestra.run.smithi090.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T09:21:46.511 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:8f:d7:66 brd ff:ff:ff:ff:ff:ff 2024-04-24T09:21:46.511 INFO:teuthology.orchestra.run.smithi090.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T09:21:46.511 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:8f:d7:67 brd ff:ff:ff:ff:ff:ff 2024-04-24T09:21:46.512 INFO:teuthology.orchestra.run.smithi090.stdout: inet 172.21.15.90/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-24T09:21:46.512 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft 41931sec preferred_lft 41931sec 2024-04-24T09:21:46.512 INFO:teuthology.orchestra.run.smithi090.stdout: inet6 fe80::ec4:7aff:fe8f:d767/64 scope link 2024-04-24T09:21:46.512 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-04-24T09:21:46.513 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-24T09:21:46.513 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T09:21:46.513 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-04-24T09:21:46.513 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link add name ceph-brx type bridge 2024-04-24T09:21:46.513 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip addr flush dev ceph-brx 2024-04-24T09:21:46.513 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link set ceph-brx up 2024-04-24T09:21:46.513 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-24T09:21:46.513 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-04-24T09:21:46.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T09:21:46.963 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:46 smithi090 ceph-mon[27825]: pgmap v402: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-24T09:21:47.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T09:21:47.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:21:47.021 DEBUG:teuthology.orchestra.run.smithi090:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T09:21:47.084 INFO:teuthology.orchestra.run.smithi090.stdout:1 2024-04-24T09:21:47.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:21:47.088 DEBUG:teuthology.orchestra.run.smithi090:> route 2024-04-24T09:21:47.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:46 smithi149 ceph-mon[32592]: pgmap v402: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-24T09:21:47.145 INFO:teuthology.orchestra.run.smithi090.stdout:Kernel IP routing table 2024-04-24T09:21:47.145 INFO:teuthology.orchestra.run.smithi090.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T09:21:47.145 INFO:teuthology.orchestra.run.smithi090.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-24T09:21:47.145 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-24T09:21:47.146 INFO:teuthology.orchestra.run.smithi090.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T09:21:47.146 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T09:21:47.146 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-04-24T09:21:47.146 DEBUG:teuthology.orchestra.run.smithi090:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-24T09:21:47.146 DEBUG:teuthology.orchestra.run.smithi090:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-24T09:21:47.146 DEBUG:teuthology.orchestra.run.smithi090:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-24T09:21:47.146 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-04-24T09:21:47.175 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:46 smithi114 ceph-mon[32501]: pgmap v402: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-24T09:21:47.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T09:21:47.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:47 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:21:47.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:47 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:21:47.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:47 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:21:47.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:47 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:21:47.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:47 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:21:47.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T09:21:47.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:21:47.981 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list 2024-04-24T09:21:48.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:21:48.025 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list-id 2024-04-24T09:21:48.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:47 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:21:48.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:47 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:21:48.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:47 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:21:48.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:47 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:21:48.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:47 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:21:48.116 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T09:21:48.116 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-04-24T09:21:48.116 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T09:21:48.116 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-24T09:21:48.116 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-04-24T09:21:48.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:47 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:21:48.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:47 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:21:48.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:47 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:21:48.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:47 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:21:48.177 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:47 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:21:48.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T09:21:48.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T09:21:48.284 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-24T09:21:48.285 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T09:21:48.285 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-04-24T09:21:48.285 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-24T09:21:48.285 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-24T09:21:48.285 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-24T09:21:48.285 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-24T09:21:48.285 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-24T09:21:48.285 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-04-24T09:21:48.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T09:21:48.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T09:21:48.588 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T09:21:48.588 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-04-24T09:21:48.588 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link set brx.0 up 2024-04-24T09:21:48.588 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link set dev brx.0 master ceph-brx 2024-04-24T09:21:48.588 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-04-24T09:21:48.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T09:21:48.742 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:48 smithi090 ceph-mon[27825]: pgmap v403: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:21:48.742 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:48 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:21:48.742 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:48 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:21:48.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T09:21:48.776 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T09:21:48.776 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T09:21:48.814 INFO:teuthology.orchestra.run.smithi090.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T09:21:48.814 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T09:21:48.814 DEBUG:teuthology.orchestra.run.smithi090:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T09:21:48.901 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-24T09:21:48.901 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-24T09:21:48.901 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/af9733a5-fcb7-4ed2-92a8-e15d22975ff5 2024-04-24T09:21:48.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:21:48.901 DEBUG:teuthology.orchestra.run.smithi090:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/af9733a5-fcb7-4ed2-92a8-e15d22975ff5 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,nofallback 2024-04-24T09:21:49.095 INFO:teuthology.orchestra.run.smithi090.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,nofallback 2024-04-24T09:21:49.095 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync". 2024-04-24T09:21:49.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:48 smithi149 ceph-mon[32592]: pgmap v403: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:21:49.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:48 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:21:49.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:48 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:21:49.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:48 smithi114 ceph-mon[32501]: pgmap v403: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:21:49.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:48 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:21:49.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:48 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:21:50.132 INFO:teuthology.orchestra.run.smithi090.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,nofallback 2024-04-24T09:21:50.132 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync". 2024-04-24T09:21:50.132 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: resolved to: "172.21.15.90:3300,172.21.15.114:3300,172.21.15.149:3300" 2024-04-24T09:21:50.132 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: trying mount with new device syntax: 0@30ee9be2-021a-11ef-bc93-c7b262605968.cephfs=/volumes/qa/sv_1/af9733a5-fcb7-4ed2-92a8-e15d22975ff5 2024-04-24T09:21:50.132 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync,name=0,key=0,mon_addr=172.21.15.90:3300/172.21.15.114:3300/172.21.15.149:3300" will pass to kernel 2024-04-24T09:21:50.137 INFO:teuthology.orchestra.run.smithi090.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-24T09:21:50.137 INFO:teuthology.orchestra.run.smithi090.stdout: You just mounted a file system that supports labels which does not 2024-04-24T09:21:50.137 INFO:teuthology.orchestra.run.smithi090.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-24T09:21:50.137 INFO:teuthology.orchestra.run.smithi090.stdout: applications will generate AVC messages and not be allowed access to 2024-04-24T09:21:50.137 INFO:teuthology.orchestra.run.smithi090.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-24T09:21:50.140 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-24T09:21:50.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:21:50.141 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T09:21:50.214 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T09:21:50.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T09:21:50.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T09:21:50.327 INFO:teuthology.orchestra.run.smithi090.stdout:client25247 2024-04-24T09:21:50.327 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-24T09:21:50.327 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T09:21:50.371 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T09:21:50.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T09:21:50.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T09:21:50.519 INFO:teuthology.orchestra.run.smithi090.stdout:client25247 2024-04-24T09:21:50.519 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T09:21:50.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:50 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:21:50.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:50 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:21:50.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:50 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:21:50.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:50 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:21:50.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:50 smithi090 ceph-mon[27825]: pgmap v404: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-24T09:21:50.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:50 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:21:50.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:50 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:21:50.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:50 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:21:50.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:50 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:21:50.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:50 smithi149 ceph-mon[32592]: pgmap v404: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-24T09:21:50.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T09:21:50.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T09:21:50.628 INFO:teuthology.orchestra.run.smithi090.stdout:client25247 2024-04-24T09:21:50.629 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/30ee9be2-021a-11ef-bc93-c7b262605968.client25247/status') 2024-04-24T09:21:50.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:50 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:21:50.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:50 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:21:50.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:50 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:21:50.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:50 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:21:50.676 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:50 smithi114 ceph-mon[32501]: pgmap v404: 97 pgs: 97 active+clean; 643 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-24T09:21:50.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T09:21:50.757 INFO:teuthology.orchestra.run.smithi090.stdout:instance: client.25247 (3)172.21.15.90:0/3290654476 2024-04-24T09:21:50.757 INFO:teuthology.orchestra.run.smithi090.stdout:blocklisted: false 2024-04-24T09:21:50.757 INFO:teuthology.orchestra.run.smithi090.stderr:0+1 records in 2024-04-24T09:21:50.757 INFO:teuthology.orchestra.run.smithi090.stderr:0+1 records out 2024-04-24T09:21:50.757 INFO:teuthology.orchestra.run.smithi090.stderr:71 bytes copied, 5.9746e-05 s, 1.2 MB/s 2024-04-24T09:21:50.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T09:21:50.769 INFO:teuthology.run_tasks:Running task exec... 2024-04-24T09:21:50.780 INFO:teuthology.task.exec:Executing custom commands... 2024-04-24T09:21:50.780 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi090.front.sepia.ceph.com 2024-04-24T09:21:50.780 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-04-24T09:21:50.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:50.955+0000 7f6125400640 1 -- 172.21.15.90:0/4294881654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6120102ae0 msgr2=0x7f6120104ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:50.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:50.955+0000 7f6125400640 1 --2- 172.21.15.90:0/4294881654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6120102ae0 0x7f6120104ed0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6110009ca0 tx=0x7f611002a8c0 comp rx=0 tx=0).stop 2024-04-24T09:21:50.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:50.955+0000 7f6125400640 1 -- 172.21.15.90:0/4294881654 shutdown_connections 2024-04-24T09:21:50.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:50.955+0000 7f6125400640 1 --2- 172.21.15.90:0/4294881654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6120107d40 0x7f612010a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:50.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:50.955+0000 7f6125400640 1 --2- 172.21.15.90:0/4294881654 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6120105410 0x7f6120107800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:50.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:50.955+0000 7f6125400640 1 --2- 172.21.15.90:0/4294881654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6120102ae0 0x7f6120104ed0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:50.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:50.955+0000 7f6125400640 1 -- 172.21.15.90:0/4294881654 >> 172.21.15.90:0/4294881654 conn(0x7f61200fc4d0 msgr2=0x7f61200fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:50.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:50.955+0000 7f6125400640 1 -- 172.21.15.90:0/4294881654 shutdown_connections 2024-04-24T09:21:50.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:50.955+0000 7f6125400640 1 -- 172.21.15.90:0/4294881654 wait complete. 2024-04-24T09:21:50.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:50.956+0000 7f6125400640 1 Processor -- start 2024-04-24T09:21:50.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:50.956+0000 7f6125400640 1 -- start start 2024-04-24T09:21:50.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:50.957+0000 7f6125400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6120102ae0 0x7f612019a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:50.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:50.957+0000 7f6125400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6120105410 0x7f612019aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:50.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:50.957+0000 7f6125400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6120107d40 0x7f61201a1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:50.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:50.957+0000 7f6125400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f612010dd40 con 0x7f6120105410 2024-04-24T09:21:50.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:50.957+0000 7f6125400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f612010dbc0 con 0x7f6120102ae0 2024-04-24T09:21:50.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:50.957+0000 7f6125400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f612010dec0 con 0x7f6120107d40 2024-04-24T09:21:50.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:50.957+0000 7f6115e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6120105410 0x7f612019aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:50.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:50.957+0000 7f6115e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6120105410 0x7f612019aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48008/0 (socket says 172.21.15.90:48008) 2024-04-24T09:21:50.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:50.957+0000 7f6115e00640 1 -- 172.21.15.90:0/3588380684 learned_addr learned my addr 172.21.15.90:0/3588380684 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:50.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:50.957+0000 7f611e800640 1 --2- 172.21.15.90:0/3588380684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6120102ae0 0x7f612019a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:50.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:50.957+0000 7f611f200640 1 --2- 172.21.15.90:0/3588380684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6120107d40 0x7f61201a1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:50.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:50.958+0000 7f611f200640 1 -- 172.21.15.90:0/3588380684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6120102ae0 msgr2=0x7f612019a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:50.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:50.958+0000 7f611f200640 1 --2- 172.21.15.90:0/3588380684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6120102ae0 0x7f612019a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:50.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:50.958+0000 7f611f200640 1 -- 172.21.15.90:0/3588380684 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6120105410 msgr2=0x7f612019aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:50.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:50.958+0000 7f611f200640 1 --2- 172.21.15.90:0/3588380684 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6120105410 0x7f612019aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:50.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:50.958+0000 7f611f200640 1 -- 172.21.15.90:0/3588380684 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61201a20f0 con 0x7f6120107d40 2024-04-24T09:21:50.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:50.958+0000 7f6115e00640 1 --2- 172.21.15.90:0/3588380684 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6120105410 0x7f612019aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:21:50.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:50.958+0000 7f611f200640 1 --2- 172.21.15.90:0/3588380684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6120107d40 0x7f61201a1a80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f611800e970 tx=0x7f611800ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:50.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:50.958+0000 7f6117e00640 1 -- 172.21.15.90:0/3588380684 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611800ccc0 con 0x7f6120107d40 2024-04-24T09:21:50.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:50.958+0000 7f6125400640 1 -- 172.21.15.90:0/3588380684 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61201a23e0 con 0x7f6120107d40 2024-04-24T09:21:50.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:50.958+0000 7f6117e00640 1 -- 172.21.15.90:0/3588380684 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6118005020 con 0x7f6120107d40 2024-04-24T09:21:50.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:50.958+0000 7f6117e00640 1 -- 172.21.15.90:0/3588380684 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611801c070 con 0x7f6120107d40 2024-04-24T09:21:50.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:50.959+0000 7f6125400640 1 -- 172.21.15.90:0/3588380684 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61201a2980 con 0x7f6120107d40 2024-04-24T09:21:50.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:50.960+0000 7f6125400640 1 -- 172.21.15.90:0/3588380684 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6120109890 con 0x7f6120107d40 2024-04-24T09:21:50.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:50.962+0000 7f6117e00640 1 -- 172.21.15.90:0/3588380684 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6118005540 con 0x7f6120107d40 2024-04-24T09:21:50.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:50.963+0000 7f6117e00640 1 --2- 172.21.15.90:0/3588380684 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f61040a3cd0 0x7f61040a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:50.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:50.963+0000 7f6117e00640 1 -- 172.21.15.90:0/3588380684 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f61180d0310 con 0x7f6120107d40 2024-04-24T09:21:50.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:50.963+0000 7f611e800640 1 --2- 172.21.15.90:0/3588380684 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f61040a3cd0 0x7f61040a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:50.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:50.964+0000 7f611e800640 1 --2- 172.21.15.90:0/3588380684 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f61040a3cd0 0x7f61040a6190 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6110005e30 tx=0x7f6110005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:50.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:50.968+0000 7f6117e00640 1 -- 172.21.15.90:0/3588380684 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f611809ccb0 con 0x7f6120107d40 2024-04-24T09:21:51.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.215+0000 7f6125400640 1 -- 172.21.15.90:0/3588380684 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f6120062c00 con 0x7f6120107d40 2024-04-24T09:21:51.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.684+0000 7f6117e00640 1 -- 172.21.15.90:0/3588380684 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v28) ==== 118+0+0 (secure 0 0 0) 0x7f611809c5b0 con 0x7f6120107d40 2024-04-24T09:21:51.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.688+0000 7f6125400640 1 -- 172.21.15.90:0/3588380684 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f61040a3cd0 msgr2=0x7f61040a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:51.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.688+0000 7f6125400640 1 --2- 172.21.15.90:0/3588380684 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f61040a3cd0 0x7f61040a6190 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6110005e30 tx=0x7f6110005f50 comp rx=0 tx=0).stop 2024-04-24T09:21:51.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.688+0000 7f6125400640 1 -- 172.21.15.90:0/3588380684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6120107d40 msgr2=0x7f61201a1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:51.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.688+0000 7f6125400640 1 --2- 172.21.15.90:0/3588380684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6120107d40 0x7f61201a1a80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f611800e970 tx=0x7f611800ee30 comp rx=0 tx=0).stop 2024-04-24T09:21:51.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.688+0000 7f6125400640 1 -- 172.21.15.90:0/3588380684 shutdown_connections 2024-04-24T09:21:51.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.688+0000 7f6125400640 1 --2- 172.21.15.90:0/3588380684 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f61040a3cd0 0x7f61040a6190 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:51.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.688+0000 7f6125400640 1 --2- 172.21.15.90:0/3588380684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6120107d40 0x7f61201a1a80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:51.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.688+0000 7f6125400640 1 --2- 172.21.15.90:0/3588380684 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6120105410 0x7f612019aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:51.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.688+0000 7f6125400640 1 --2- 172.21.15.90:0/3588380684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6120102ae0 0x7f612019a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:51.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.688+0000 7f6125400640 1 -- 172.21.15.90:0/3588380684 >> 172.21.15.90:0/3588380684 conn(0x7f61200fc4d0 msgr2=0x7f6120102160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:51.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.689+0000 7f6125400640 1 -- 172.21.15.90:0/3588380684 shutdown_connections 2024-04-24T09:21:51.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.689+0000 7f6125400640 1 -- 172.21.15.90:0/3588380684 wait complete. 2024-04-24T09:21:51.718 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-04-24T09:21:51.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.922+0000 7f3e66a00640 1 -- 172.21.15.90:0/1355817557 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e601052e0 msgr2=0x7f3e601076d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:51.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.922+0000 7f3e66a00640 1 --2- 172.21.15.90:0/1355817557 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e601052e0 0x7f3e601076d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c009900 tx=0x7f3e4c02a3f0 comp rx=0 tx=0).stop 2024-04-24T09:21:51.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.922+0000 7f3e66a00640 1 -- 172.21.15.90:0/1355817557 shutdown_connections 2024-04-24T09:21:51.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.922+0000 7f3e66a00640 1 --2- 172.21.15.90:0/1355817557 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e60107c10 0x7f3e6010a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:51.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.922+0000 7f3e66a00640 1 --2- 172.21.15.90:0/1355817557 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e601052e0 0x7f3e601076d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:51.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.922+0000 7f3e66a00640 1 --2- 172.21.15.90:0/1355817557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e601029b0 0x7f3e60104da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:51.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.922+0000 7f3e66a00640 1 -- 172.21.15.90:0/1355817557 >> 172.21.15.90:0/1355817557 conn(0x7f3e600fc340 msgr2=0x7f3e600fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:51.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.922+0000 7f3e66a00640 1 -- 172.21.15.90:0/1355817557 shutdown_connections 2024-04-24T09:21:51.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.922+0000 7f3e66a00640 1 -- 172.21.15.90:0/1355817557 wait complete. 2024-04-24T09:21:51.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.923+0000 7f3e66a00640 1 Processor -- start 2024-04-24T09:21:51.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.923+0000 7f3e66a00640 1 -- start start 2024-04-24T09:21:51.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.923+0000 7f3e66a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e601029b0 0x7f3e6019a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:51.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.923+0000 7f3e66a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e601052e0 0x7f3e6019a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:51.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.923+0000 7f3e66a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e60107c10 0x7f3e601a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:51.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.923+0000 7f3e66a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6010db50 con 0x7f3e60107c10 2024-04-24T09:21:51.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.924+0000 7f3e66a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6010d9d0 con 0x7f3e601052e0 2024-04-24T09:21:51.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.924+0000 7f3e66a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6010dcd0 con 0x7f3e601029b0 2024-04-24T09:21:51.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.924+0000 7f3e66000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e60107c10 0x7f3e601a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:51.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.924+0000 7f3e66000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e60107c10 0x7f3e601a1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48016/0 (socket says 172.21.15.90:48016) 2024-04-24T09:21:51.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.924+0000 7f3e66000640 1 -- 172.21.15.90:0/755553885 learned_addr learned my addr 172.21.15.90:0/755553885 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:21:51.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.924+0000 7f3e64c00640 1 --2- 172.21.15.90:0/755553885 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e601052e0 0x7f3e6019a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:51.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.924+0000 7f3e65600640 1 --2- 172.21.15.90:0/755553885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e601029b0 0x7f3e6019a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:51.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.924+0000 7f3e66000640 1 -- 172.21.15.90:0/755553885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e601029b0 msgr2=0x7f3e6019a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:51.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.924+0000 7f3e66000640 1 --2- 172.21.15.90:0/755553885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e601029b0 0x7f3e6019a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:51.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.924+0000 7f3e66000640 1 -- 172.21.15.90:0/755553885 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e601052e0 msgr2=0x7f3e6019a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:51.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.924+0000 7f3e66000640 1 --2- 172.21.15.90:0/755553885 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e601052e0 0x7f3e6019a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:51.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.924+0000 7f3e66000640 1 -- 172.21.15.90:0/755553885 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e601a1f80 con 0x7f3e60107c10 2024-04-24T09:21:51.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.924+0000 7f3e65600640 1 --2- 172.21.15.90:0/755553885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e601029b0 0x7f3e6019a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:21:51.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.925+0000 7f3e66000640 1 --2- 172.21.15.90:0/755553885 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e60107c10 0x7f3e601a1910 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3e5800a5c0 tx=0x7f3e5800aa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:51.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.925+0000 7f3e5e000640 1 -- 172.21.15.90:0/755553885 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5800b9d0 con 0x7f3e60107c10 2024-04-24T09:21:51.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.925+0000 7f3e5e000640 1 -- 172.21.15.90:0/755553885 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e5800bb70 con 0x7f3e60107c10 2024-04-24T09:21:51.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.925+0000 7f3e5e000640 1 -- 172.21.15.90:0/755553885 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e58014350 con 0x7f3e60107c10 2024-04-24T09:21:51.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.925+0000 7f3e66a00640 1 -- 172.21.15.90:0/755553885 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e60193290 con 0x7f3e60107c10 2024-04-24T09:21:51.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.925+0000 7f3e66a00640 1 -- 172.21.15.90:0/755553885 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e60193760 con 0x7f3e60107c10 2024-04-24T09:21:51.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:51 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:21:51.927 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:51 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:21:51.928 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:51 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/3588380684' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-24T09:21:51.928 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:51 smithi114 ceph-mon[32501]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-24T09:21:51.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.926+0000 7f3e66a00640 1 -- 172.21.15.90:0/755553885 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e60109700 con 0x7f3e60107c10 2024-04-24T09:21:51.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.929+0000 7f3e5e000640 1 -- 172.21.15.90:0/755553885 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3e58013440 con 0x7f3e60107c10 2024-04-24T09:21:51.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.930+0000 7f3e5e000640 1 --2- 172.21.15.90:0/755553885 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f3e480a3ba0 0x7f3e480a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:21:51.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.930+0000 7f3e5e000640 1 -- 172.21.15.90:0/755553885 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f3e580e11d0 con 0x7f3e60107c10 2024-04-24T09:21:51.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.930+0000 7f3e65600640 1 --2- 172.21.15.90:0/755553885 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f3e480a3ba0 0x7f3e480a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:21:51.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.931+0000 7f3e65600640 1 --2- 172.21.15.90:0/755553885 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f3e480a3ba0 0x7f3e480a6060 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3e540095f0 tx=0x7f3e54005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:21:51.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:51.936+0000 7f3e5e000640 1 -- 172.21.15.90:0/755553885 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3e580ad520 con 0x7f3e60107c10 2024-04-24T09:21:52.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:51 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:21:52.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:51 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:21:52.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:51 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/3588380684' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-24T09:21:52.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:51 smithi090 ceph-mon[27825]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-24T09:21:52.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:51 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:21:52.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:51 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:21:52.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:51 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/3588380684' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-24T09:21:52.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:51 smithi149 ceph-mon[32592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-24T09:21:52.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:52.173+0000 7f3e66a00640 1 -- 172.21.15.90:0/755553885 --> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f3e600625a0 con 0x7f3e480a3ba0 2024-04-24T09:21:52.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:52.202+0000 7f3e5e000640 1 -- 172.21.15.90:0/755553885 <== mgr.14152 v2:172.21.15.90:6800/4261662956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f3e600625a0 con 0x7f3e480a3ba0 2024-04-24T09:21:52.203 INFO:teuthology.orchestra.run.smithi090.stdout:{} 2024-04-24T09:21:52.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:52.205+0000 7f3e66a00640 1 -- 172.21.15.90:0/755553885 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f3e480a3ba0 msgr2=0x7f3e480a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:52.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:52.205+0000 7f3e66a00640 1 --2- 172.21.15.90:0/755553885 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f3e480a3ba0 0x7f3e480a6060 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3e540095f0 tx=0x7f3e54005f90 comp rx=0 tx=0).stop 2024-04-24T09:21:52.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:52.205+0000 7f3e66a00640 1 -- 172.21.15.90:0/755553885 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e60107c10 msgr2=0x7f3e601a1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:21:52.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:52.205+0000 7f3e66a00640 1 --2- 172.21.15.90:0/755553885 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e60107c10 0x7f3e601a1910 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3e5800a5c0 tx=0x7f3e5800aa80 comp rx=0 tx=0).stop 2024-04-24T09:21:52.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:52.205+0000 7f3e66a00640 1 -- 172.21.15.90:0/755553885 shutdown_connections 2024-04-24T09:21:52.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:52.205+0000 7f3e66a00640 1 --2- 172.21.15.90:0/755553885 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f3e480a3ba0 0x7f3e480a6060 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:52.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:52.205+0000 7f3e66a00640 1 --2- 172.21.15.90:0/755553885 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e60107c10 0x7f3e601a1910 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:52.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:52.205+0000 7f3e66a00640 1 --2- 172.21.15.90:0/755553885 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3e601052e0 0x7f3e6019a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:52.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:52.205+0000 7f3e66a00640 1 --2- 172.21.15.90:0/755553885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e601029b0 0x7f3e6019a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:21:52.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:52.205+0000 7f3e66a00640 1 -- 172.21.15.90:0/755553885 >> 172.21.15.90:0/755553885 conn(0x7f3e600fc340 msgr2=0x7f3e601020d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:21:52.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:52.205+0000 7f3e66a00640 1 -- 172.21.15.90:0/755553885 shutdown_connections 2024-04-24T09:21:52.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:21:52.206+0000 7f3e66a00640 1 -- 172.21.15.90:0/755553885 wait complete. 2024-04-24T09:21:52.223 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-24T09:21:52.235 INFO:teuthology.run_tasks:Running task workunit... 2024-04-24T09:21:52.285 INFO:tasks.workunit:Pulling workunits from ref 1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T09:21:52.285 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-24T09:21:52.285 INFO:tasks.workunit:timeout=3h 2024-04-24T09:21:52.285 INFO:tasks.workunit:cleanup=True 2024-04-24T09:21:52.286 DEBUG:teuthology.orchestra.run.smithi090:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-24T09:21:52.341 INFO:teuthology.orchestra.run.smithi090.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-24T09:21:52.341 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-24T09:21:52.341 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 72h/114d Inode: 1099511628283 Links: 2 2024-04-24T09:21:52.341 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T09:21:52.341 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-24T09:21:52.341 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-04-24 09:19:17.677315069 +0000 2024-04-24T09:21:52.341 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-04-24 09:19:17.677315069 +0000 2024-04-24T09:21:52.341 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-04-24 09:21:50.207071782 +0000 2024-04-24T09:21:52.341 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-04-24 09:19:17.677315069 +0000 2024-04-24T09:21:52.341 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-24T09:21:52.341 DEBUG:teuthology.orchestra.run.smithi090:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-24T09:21:52.434 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:21:52 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:21:52.180+0000 7f2d4fe00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T09:21:52.434 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:21:52 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:21:52.180+0000 7f2d4fe00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T09:21:52.435 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:21:52 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:21:52.180+0000 7f2d4fe00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T09:21:52.435 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:21:52 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:21:52.180+0000 7f2d4fe00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T09:21:52.435 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:21:52 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:21:52.180+0000 7f2d4fe00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T09:21:52.437 DEBUG:teuthology.orchestra.run.smithi090:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T09:21:52.496 INFO:tasks.workunit.client.0.smithi090.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-24T09:21:53.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:52 smithi090 ceph-mon[27825]: pgmap v405: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-24T09:21:53.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:52 smithi090 ceph-mon[27825]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-24T09:21:53.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:52 smithi090 ceph-mon[27825]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:21:53.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:52 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:21:53.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:52 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:21:53.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:52 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:21:53.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:52 smithi149 ceph-mon[32592]: pgmap v405: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-24T09:21:53.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:52 smithi149 ceph-mon[32592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-24T09:21:53.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:52 smithi149 ceph-mon[32592]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:21:53.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:52 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:21:53.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:52 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:21:53.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:52 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:21:53.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:52 smithi114 ceph-mon[32501]: pgmap v405: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-24T09:21:53.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:52 smithi114 ceph-mon[32501]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-24T09:21:53.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:52 smithi114 ceph-mon[32501]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:21:53.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:52 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:21:53.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:52 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:21:53.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:52 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:21:54.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:53 smithi090 ceph-mon[27825]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:21:54.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:53 smithi149 ceph-mon[32592]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:21:54.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:53 smithi114 ceph-mon[32501]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:21:55.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:54 smithi090 ceph-mon[27825]: pgmap v406: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-24T09:21:55.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:54 smithi149 ceph-mon[32592]: pgmap v406: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-24T09:21:55.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:54 smithi114 ceph-mon[32501]: pgmap v406: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-24T09:21:56.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:55 smithi090 ceph-mon[27825]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-04-24T09:21:56.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:55 smithi149 ceph-mon[32592]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-04-24T09:21:56.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:55 smithi114 ceph-mon[32501]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-04-24T09:21:57.086 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:56 smithi090 ceph-mon[27825]: pgmap v407: 97 pgs: 97 active+clean; 657 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-24T09:21:57.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:56 smithi149 ceph-mon[32592]: pgmap v407: 97 pgs: 97 active+clean; 657 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-24T09:21:57.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:56 smithi114 ceph-mon[32501]: pgmap v407: 97 pgs: 97 active+clean; 657 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-24T09:21:59.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:21:58 smithi090 ceph-mon[27825]: pgmap v408: 97 pgs: 97 active+clean; 658 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.3 KiB/s wr, 11 op/s 2024-04-24T09:21:59.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:21:58 smithi149 ceph-mon[32592]: pgmap v408: 97 pgs: 97 active+clean; 658 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.3 KiB/s wr, 11 op/s 2024-04-24T09:21:59.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:21:58 smithi114 ceph-mon[32501]: pgmap v408: 97 pgs: 97 active+clean; 658 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.3 KiB/s wr, 11 op/s 2024-04-24T09:22:01.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:22:00 smithi090 ceph-mon[27825]: pgmap v409: 97 pgs: 97 active+clean; 660 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2024-04-24T09:22:01.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:22:00 smithi149 ceph-mon[32592]: pgmap v409: 97 pgs: 97 active+clean; 660 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2024-04-24T09:22:01.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:22:00 smithi114 ceph-mon[32501]: pgmap v409: 97 pgs: 97 active+clean; 660 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2024-04-24T09:22:03.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:22:02 smithi090 ceph-mon[27825]: pgmap v410: 97 pgs: 97 active+clean; 660 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.5 KiB/s wr, 11 op/s 2024-04-24T09:22:03.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:22:02 smithi149 ceph-mon[32592]: pgmap v410: 97 pgs: 97 active+clean; 660 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.5 KiB/s wr, 11 op/s 2024-04-24T09:22:03.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:22:02 smithi114 ceph-mon[32501]: pgmap v410: 97 pgs: 97 active+clean; 660 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.5 KiB/s wr, 11 op/s 2024-04-24T09:22:04.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:22:03 smithi090 ceph-mon[27825]: pgmap v411: 97 pgs: 97 active+clean; 660 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.4 KiB/s wr, 11 op/s 2024-04-24T09:22:04.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:22:03 smithi149 ceph-mon[32592]: pgmap v411: 97 pgs: 97 active+clean; 660 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.4 KiB/s wr, 11 op/s 2024-04-24T09:22:04.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:22:03 smithi114 ceph-mon[32501]: pgmap v411: 97 pgs: 97 active+clean; 660 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.4 KiB/s wr, 11 op/s 2024-04-24T09:22:06.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:22:06 smithi114 ceph-mon[32501]: pgmap v412: 97 pgs: 97 active+clean; 660 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.3 KiB/s wr, 13 op/s 2024-04-24T09:22:07.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:22:06 smithi090 ceph-mon[27825]: pgmap v412: 97 pgs: 97 active+clean; 660 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.3 KiB/s wr, 13 op/s 2024-04-24T09:22:07.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:22:06 smithi149 ceph-mon[32592]: pgmap v412: 97 pgs: 97 active+clean; 660 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.3 KiB/s wr, 13 op/s 2024-04-24T09:22:08.906 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:22:08 smithi090 ceph-mon[27825]: pgmap v413: 97 pgs: 97 active+clean; 660 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-24T09:22:08.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:22:08 smithi114 ceph-mon[32501]: pgmap v413: 97 pgs: 97 active+clean; 660 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-24T09:22:09.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:22:08 smithi149 ceph-mon[32592]: pgmap v413: 97 pgs: 97 active+clean; 660 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-24T09:22:10.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:22:10 smithi090 ceph-mon[27825]: pgmap v414: 97 pgs: 97 active+clean; 660 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-24T09:22:10.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:22:10 smithi114 ceph-mon[32501]: pgmap v414: 97 pgs: 97 active+clean; 660 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-24T09:22:11.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:22:10 smithi149 ceph-mon[32592]: pgmap v414: 97 pgs: 97 active+clean; 660 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-24T09:22:12.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:22:12 smithi114 ceph-mon[32501]: pgmap v415: 97 pgs: 97 active+clean; 660 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-04-24T09:22:13.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:22:12 smithi090 ceph-mon[27825]: pgmap v415: 97 pgs: 97 active+clean; 660 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-04-24T09:22:13.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:22:12 smithi149 ceph-mon[32592]: pgmap v415: 97 pgs: 97 active+clean; 660 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-04-24T09:22:14.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:22:14 smithi114 ceph-mon[32501]: pgmap v416: 97 pgs: 97 active+clean; 660 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-24T09:22:14.929 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:22:14 smithi149 ceph-mon[32592]: pgmap v416: 97 pgs: 97 active+clean; 660 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-24T09:22:15.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:22:14 smithi090 ceph-mon[27825]: pgmap v416: 97 pgs: 97 active+clean; 660 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-24T09:22:17.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:22:16 smithi090 ceph-mon[27825]: pgmap v417: 97 pgs: 97 active+clean; 660 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-04-24T09:22:17.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:22:16 smithi149 ceph-mon[32592]: pgmap v417: 97 pgs: 97 active+clean; 660 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-04-24T09:22:17.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:22:16 smithi114 ceph-mon[32501]: pgmap v417: 97 pgs: 97 active+clean; 660 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-04-24T09:22:19.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:22:18 smithi090 ceph-mon[27825]: pgmap v418: 97 pgs: 97 active+clean; 660 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:22:19.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:22:18 smithi149 ceph-mon[32592]: pgmap v418: 97 pgs: 97 active+clean; 660 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:22:19.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:22:18 smithi114 ceph-mon[32501]: pgmap v418: 97 pgs: 97 active+clean; 660 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:22:21.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:22:20 smithi090 ceph-mon[27825]: pgmap v419: 97 pgs: 97 active+clean; 660 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-24T09:22:21.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:22:20 smithi149 ceph-mon[32592]: pgmap v419: 97 pgs: 97 active+clean; 660 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-24T09:22:21.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:22:20 smithi114 ceph-mon[32501]: pgmap v419: 97 pgs: 97 active+clean; 660 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-24T09:22:23.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:22:22 smithi090 ceph-mon[27825]: pgmap v420: 97 pgs: 97 active+clean; 660 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T09:22:23.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:22:22 smithi149 ceph-mon[32592]: pgmap v420: 97 pgs: 97 active+clean; 660 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T09:22:23.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:22:22 smithi114 ceph-mon[32501]: pgmap v420: 97 pgs: 97 active+clean; 660 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T09:22:25.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:22:24 smithi090 ceph-mon[27825]: pgmap v421: 97 pgs: 97 active+clean; 660 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:22:25.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:22:24 smithi149 ceph-mon[32592]: pgmap v421: 97 pgs: 97 active+clean; 660 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:22:25.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:22:24 smithi114 ceph-mon[32501]: pgmap v421: 97 pgs: 97 active+clean; 660 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:22:27.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:22:26 smithi090 ceph-mon[27825]: pgmap v422: 97 pgs: 97 active+clean; 660 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-24T09:22:27.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:22:26 smithi149 ceph-mon[32592]: pgmap v422: 97 pgs: 97 active+clean; 660 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-24T09:22:27.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:22:26 smithi114 ceph-mon[32501]: pgmap v422: 97 pgs: 97 active+clean; 660 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-24T09:22:29.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:22:28 smithi090 ceph-mon[27825]: pgmap v423: 97 pgs: 97 active+clean; 660 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-24T09:22:29.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:22:28 smithi149 ceph-mon[32592]: pgmap v423: 97 pgs: 97 active+clean; 660 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-24T09:22:29.175 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:22:28 smithi114 ceph-mon[32501]: pgmap v423: 97 pgs: 97 active+clean; 660 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-24T09:22:30.978 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:22:30 smithi114 ceph-mon[32501]: pgmap v424: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-24T09:22:31.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:22:30 smithi090 ceph-mon[27825]: pgmap v424: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-24T09:22:31.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:22:30 smithi149 ceph-mon[32592]: pgmap v424: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-24T09:22:33.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:22:32 smithi090 ceph-mon[27825]: pgmap v425: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-24T09:22:33.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:22:32 smithi149 ceph-mon[32592]: pgmap v425: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-24T09:22:33.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:22:32 smithi114 ceph-mon[32501]: pgmap v425: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-24T09:22:35.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:22:34 smithi090 ceph-mon[27825]: pgmap v426: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-24T09:22:35.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:22:34 smithi149 ceph-mon[32592]: pgmap v426: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-24T09:22:35.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:22:34 smithi114 ceph-mon[32501]: pgmap v426: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-24T09:22:37.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:22:36 smithi090 ceph-mon[27825]: pgmap v427: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 682 B/s wr, 11 op/s 2024-04-24T09:22:37.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:22:36 smithi149 ceph-mon[32592]: pgmap v427: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 682 B/s wr, 11 op/s 2024-04-24T09:22:37.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:22:36 smithi114 ceph-mon[32501]: pgmap v427: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 682 B/s wr, 11 op/s 2024-04-24T09:22:39.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:22:38 smithi090 ceph-mon[27825]: pgmap v428: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-24T09:22:39.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:22:38 smithi149 ceph-mon[32592]: pgmap v428: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-24T09:22:39.175 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:22:38 smithi114 ceph-mon[32501]: pgmap v428: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-24T09:22:41.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:22:40 smithi090 ceph-mon[27825]: pgmap v429: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-24T09:22:41.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:22:40 smithi149 ceph-mon[32592]: pgmap v429: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-24T09:22:41.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:22:40 smithi114 ceph-mon[32501]: pgmap v429: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-24T09:22:43.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:22:42 smithi090 ceph-mon[27825]: pgmap v430: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-24T09:22:43.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:22:42 smithi149 ceph-mon[32592]: pgmap v430: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-24T09:22:43.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:22:42 smithi114 ceph-mon[32501]: pgmap v430: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-24T09:22:45.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:22:44 smithi090 ceph-mon[27825]: pgmap v431: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:22:45.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:22:44 smithi149 ceph-mon[32592]: pgmap v431: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:22:45.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:22:44 smithi114 ceph-mon[32501]: pgmap v431: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:22:47.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:22:46 smithi090 ceph-mon[27825]: pgmap v432: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-24T09:22:47.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:22:46 smithi149 ceph-mon[32592]: pgmap v432: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-24T09:22:47.175 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:22:46 smithi114 ceph-mon[32501]: pgmap v432: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-24T09:22:49.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:22:48 smithi090 ceph-mon[27825]: pgmap v433: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-24T09:22:49.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:22:48 smithi149 ceph-mon[32592]: pgmap v433: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-24T09:22:49.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:22:48 smithi114 ceph-mon[32501]: pgmap v433: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-24T09:22:51.038 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:22:50 smithi090 ceph-mon[27825]: pgmap v434: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-24T09:22:51.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:22:50 smithi149 ceph-mon[32592]: pgmap v434: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-24T09:22:51.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:22:50 smithi114 ceph-mon[32501]: pgmap v434: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-24T09:22:53.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:22:52 smithi090 ceph-mon[27825]: pgmap v435: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-24T09:22:53.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:22:52 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:22:53.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:22:52 smithi149 ceph-mon[32592]: pgmap v435: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-24T09:22:53.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:22:52 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:22:53.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:22:52 smithi114 ceph-mon[32501]: pgmap v435: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-24T09:22:53.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:22:52 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:22:54.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:22:53 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:22:54.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:22:53 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:22:54.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:22:53 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:22:54.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:22:53 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:22:54.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:22:53 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:22:54.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:22:53 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:22:54.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:22:53 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:22:54.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:22:53 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:22:54.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:22:53 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:22:55.101 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:22:54 smithi090 ceph-mon[27825]: pgmap v436: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-24T09:22:55.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:22:54 smithi149 ceph-mon[32592]: pgmap v436: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-24T09:22:55.175 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:22:54 smithi114 ceph-mon[32501]: pgmap v436: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-24T09:22:56.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:22:55 smithi090 ceph-mon[27825]: pgmap v437: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-04-24T09:22:56.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:22:55 smithi149 ceph-mon[32592]: pgmap v437: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-04-24T09:22:56.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:22:55 smithi114 ceph-mon[32501]: pgmap v437: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-04-24T09:22:58.926 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:22:58 smithi114 ceph-mon[32501]: pgmap v438: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-24T09:22:59.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:22:58 smithi090 ceph-mon[27825]: pgmap v438: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-24T09:22:59.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:22:58 smithi149 ceph-mon[32592]: pgmap v438: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-24T09:23:00.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:22:59 smithi090 ceph-mon[27825]: mgrmap e20: x(active, since 12m), standbys: z, y 2024-04-24T09:23:00.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:22:59 smithi149 ceph-mon[32592]: mgrmap e20: x(active, since 12m), standbys: z, y 2024-04-24T09:23:00.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:22:59 smithi114 ceph-mon[32501]: mgrmap e20: x(active, since 12m), standbys: z, y 2024-04-24T09:23:01.077 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:23:00 smithi114 ceph-mon[32501]: pgmap v439: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-24T09:23:01.101 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:23:00 smithi090 ceph-mon[27825]: pgmap v439: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-24T09:23:01.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:23:00 smithi149 ceph-mon[32592]: pgmap v439: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-24T09:23:03.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:23:02 smithi090 ceph-mon[27825]: pgmap v440: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 682 B/s wr, 11 op/s 2024-04-24T09:23:03.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:23:02 smithi149 ceph-mon[32592]: pgmap v440: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 682 B/s wr, 11 op/s 2024-04-24T09:23:03.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:23:02 smithi114 ceph-mon[32501]: pgmap v440: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 682 B/s wr, 11 op/s 2024-04-24T09:23:05.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:23:04 smithi090 ceph-mon[27825]: pgmap v441: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 10 op/s 2024-04-24T09:23:05.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:23:04 smithi149 ceph-mon[32592]: pgmap v441: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 10 op/s 2024-04-24T09:23:05.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:23:04 smithi114 ceph-mon[32501]: pgmap v441: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 10 op/s 2024-04-24T09:23:07.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:23:06 smithi090 ceph-mon[27825]: pgmap v442: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 12 op/s 2024-04-24T09:23:07.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:23:06 smithi149 ceph-mon[32592]: pgmap v442: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 12 op/s 2024-04-24T09:23:07.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:23:06 smithi114 ceph-mon[32501]: pgmap v442: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 12 op/s 2024-04-24T09:23:09.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:23:08 smithi090 ceph-mon[27825]: pgmap v443: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 767 B/s wr, 10 op/s 2024-04-24T09:23:09.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:23:08 smithi149 ceph-mon[32592]: pgmap v443: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 767 B/s wr, 10 op/s 2024-04-24T09:23:09.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:23:08 smithi114 ceph-mon[32501]: pgmap v443: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 767 B/s wr, 10 op/s 2024-04-24T09:23:11.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:23:10 smithi090 ceph-mon[27825]: pgmap v444: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-04-24T09:23:11.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:23:10 smithi149 ceph-mon[32592]: pgmap v444: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-04-24T09:23:11.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:23:10 smithi114 ceph-mon[32501]: pgmap v444: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-04-24T09:23:13.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:23:12 smithi090 ceph-mon[27825]: pgmap v445: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-24T09:23:13.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:23:12 smithi149 ceph-mon[32592]: pgmap v445: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-24T09:23:13.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:23:12 smithi114 ceph-mon[32501]: pgmap v445: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-24T09:23:15.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:23:14 smithi090 ceph-mon[27825]: pgmap v446: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-24T09:23:15.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:23:14 smithi149 ceph-mon[32592]: pgmap v446: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-24T09:23:15.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:23:14 smithi114 ceph-mon[32501]: pgmap v446: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-24T09:23:17.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:23:16 smithi090 ceph-mon[27825]: pgmap v447: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-24T09:23:17.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:23:16 smithi149 ceph-mon[32592]: pgmap v447: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-24T09:23:17.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:23:16 smithi114 ceph-mon[32501]: pgmap v447: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-24T09:23:19.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:23:18 smithi090 ceph-mon[27825]: pgmap v448: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:23:19.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:23:18 smithi149 ceph-mon[32592]: pgmap v448: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:23:19.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:23:18 smithi114 ceph-mon[32501]: pgmap v448: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:23:21.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:23:20 smithi090 ceph-mon[27825]: pgmap v449: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-24T09:23:21.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:23:20 smithi149 ceph-mon[32592]: pgmap v449: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-24T09:23:21.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:23:20 smithi114 ceph-mon[32501]: pgmap v449: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-24T09:23:23.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:23:22 smithi090 ceph-mon[27825]: pgmap v450: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-24T09:23:23.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:23:22 smithi149 ceph-mon[32592]: pgmap v450: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-24T09:23:23.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:23:22 smithi114 ceph-mon[32501]: pgmap v450: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-24T09:23:25.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:23:24 smithi090 ceph-mon[27825]: pgmap v451: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-24T09:23:25.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:23:24 smithi149 ceph-mon[32592]: pgmap v451: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-24T09:23:25.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:23:24 smithi114 ceph-mon[32501]: pgmap v451: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-24T09:23:27.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:23:26 smithi090 ceph-mon[27825]: pgmap v452: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-24T09:23:27.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:23:26 smithi149 ceph-mon[32592]: pgmap v452: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-24T09:23:27.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:23:26 smithi114 ceph-mon[32501]: pgmap v452: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-24T09:23:28.348 INFO:tasks.workunit.client.0.smithi090.stderr:Updating files: 61% (7533/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-04-24T09:23:29.015 INFO:tasks.workunit.client.0.smithi090.stderr:Note: switching to '1f81fb397ae98da7563d451a78c61574c8f4e6e0'. 2024-04-24T09:23:29.015 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-04-24T09:23:29.015 INFO:tasks.workunit.client.0.smithi090.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-24T09:23:29.015 INFO:tasks.workunit.client.0.smithi090.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-24T09:23:29.016 INFO:tasks.workunit.client.0.smithi090.stderr:state without impacting any branches by switching back to a branch. 2024-04-24T09:23:29.016 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-04-24T09:23:29.016 INFO:tasks.workunit.client.0.smithi090.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-24T09:23:29.016 INFO:tasks.workunit.client.0.smithi090.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-24T09:23:29.016 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-04-24T09:23:29.016 INFO:tasks.workunit.client.0.smithi090.stderr: git switch -c 2024-04-24T09:23:29.016 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-04-24T09:23:29.016 INFO:tasks.workunit.client.0.smithi090.stderr:Or undo this operation with: 2024-04-24T09:23:29.016 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-04-24T09:23:29.016 INFO:tasks.workunit.client.0.smithi090.stderr: git switch - 2024-04-24T09:23:29.016 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-04-24T09:23:29.016 INFO:tasks.workunit.client.0.smithi090.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-24T09:23:29.016 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-04-24T09:23:29.016 INFO:tasks.workunit.client.0.smithi090.stderr:HEAD is now at 1f81fb397ae DNM: trigger builds 2024-04-24T09:23:29.030 DEBUG:teuthology.orchestra.run.smithi090:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-24T09:23:29.092 INFO:tasks.workunit.client.0.smithi090.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-24T09:23:29.100 INFO:tasks.workunit.client.0.smithi090.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-24T09:23:29.100 INFO:tasks.workunit.client.0.smithi090.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-24T09:23:29.101 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:23:28 smithi090 ceph-mon[27825]: pgmap v453: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:23:29.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:23:28 smithi149 ceph-mon[32592]: pgmap v453: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:23:29.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:23:28 smithi114 ceph-mon[32501]: pgmap v453: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:23:30.146 INFO:tasks.workunit.client.0.smithi090.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-24T09:23:30.288 INFO:tasks.workunit.client.0.smithi090.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-24T09:23:30.371 INFO:tasks.workunit.client.0.smithi090.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-24T09:23:30.377 INFO:tasks.workunit.client.0.smithi090.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-24T09:23:30.377 INFO:tasks.workunit.client.0.smithi090.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-24T09:23:30.452 INFO:tasks.workunit.client.0.smithi090.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-24T09:23:30.463 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-24T09:23:30.463 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-24T09:23:30.509 INFO:tasks.workunit:Running workunits matching suites/fsx.sh on client.0... 2024-04-24T09:23:30.510 INFO:tasks.workunit:Running workunit suites/fsx.sh... 2024-04-24T09:23:30.511 DEBUG:teuthology.orchestra.run.smithi090:workunit test suites/fsx.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1f81fb397ae98da7563d451a78c61574c8f4e6e0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh 2024-04-24T09:23:30.625 INFO:tasks.workunit.client.0.smithi090.stderr:+ set -e 2024-04-24T09:23:30.625 INFO:tasks.workunit.client.0.smithi090.stderr:+ git clone https://git.ceph.com/xfstests-dev.git 2024-04-24T09:23:30.631 INFO:tasks.workunit.client.0.smithi090.stderr:Cloning into 'xfstests-dev'... 2024-04-24T09:23:31.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:23:30 smithi090 ceph-mon[27825]: pgmap v454: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-24T09:23:31.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:23:30 smithi149 ceph-mon[32592]: pgmap v454: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-24T09:23:31.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:23:30 smithi114 ceph-mon[32501]: pgmap v454: 97 pgs: 97 active+clean; 665 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-24T09:23:33.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:23:32 smithi090 ceph-mon[27825]: pgmap v455: 97 pgs: 97 active+clean; 666 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 511 B/s wr, 10 op/s 2024-04-24T09:23:33.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:23:32 smithi149 ceph-mon[32592]: pgmap v455: 97 pgs: 97 active+clean; 666 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 511 B/s wr, 10 op/s 2024-04-24T09:23:33.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:23:32 smithi114 ceph-mon[32501]: pgmap v455: 97 pgs: 97 active+clean; 666 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 511 B/s wr, 10 op/s 2024-04-24T09:23:35.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:23:34 smithi090 ceph-mon[27825]: pgmap v456: 97 pgs: 97 active+clean; 666 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-24T09:23:35.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:23:34 smithi149 ceph-mon[32592]: pgmap v456: 97 pgs: 97 active+clean; 666 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-24T09:23:35.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:23:34 smithi114 ceph-mon[32501]: pgmap v456: 97 pgs: 97 active+clean; 666 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-24T09:23:36.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:23:35 smithi090 ceph-mon[27825]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:23:36.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:23:35 smithi149 ceph-mon[32592]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:23:36.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:23:35 smithi114 ceph-mon[32501]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:23:37.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:23:36 smithi090 ceph-mon[27825]: pgmap v457: 97 pgs: 97 active+clean; 2.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 164 KiB/s wr, 14 op/s 2024-04-24T09:23:37.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:23:36 smithi149 ceph-mon[32592]: pgmap v457: 97 pgs: 97 active+clean; 2.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 164 KiB/s wr, 14 op/s 2024-04-24T09:23:37.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:23:36 smithi114 ceph-mon[32501]: pgmap v457: 97 pgs: 97 active+clean; 2.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 164 KiB/s wr, 14 op/s 2024-04-24T09:23:38.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:23:37 smithi090 ceph-mon[27825]: pgmap v458: 97 pgs: 97 active+clean; 14 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 1.1 MiB/s wr, 16 op/s 2024-04-24T09:23:38.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:23:37 smithi149 ceph-mon[32592]: pgmap v458: 97 pgs: 97 active+clean; 14 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 1.1 MiB/s wr, 16 op/s 2024-04-24T09:23:38.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:23:37 smithi114 ceph-mon[32501]: pgmap v458: 97 pgs: 97 active+clean; 14 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 1.1 MiB/s wr, 16 op/s 2024-04-24T09:23:40.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:23:39 smithi090 ceph-mon[27825]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:23:40.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:23:39 smithi090 ceph-mon[27825]: pgmap v459: 97 pgs: 97 active+clean; 15 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 19 op/s 2024-04-24T09:23:40.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:23:39 smithi149 ceph-mon[32592]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:23:40.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:23:39 smithi149 ceph-mon[32592]: pgmap v459: 97 pgs: 97 active+clean; 15 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 19 op/s 2024-04-24T09:23:40.175 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:23:39 smithi114 ceph-mon[32501]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:23:40.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:23:39 smithi114 ceph-mon[32501]: pgmap v459: 97 pgs: 97 active+clean; 15 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 19 op/s 2024-04-24T09:23:43.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:23:42 smithi090 ceph-mon[27825]: pgmap v460: 97 pgs: 97 active+clean; 15 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 21 op/s 2024-04-24T09:23:43.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:23:42 smithi149 ceph-mon[32592]: pgmap v460: 97 pgs: 97 active+clean; 15 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 21 op/s 2024-04-24T09:23:43.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:23:42 smithi114 ceph-mon[32501]: pgmap v460: 97 pgs: 97 active+clean; 15 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 21 op/s 2024-04-24T09:23:45.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:23:44 smithi090 ceph-mon[27825]: pgmap v461: 97 pgs: 97 active+clean; 15 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 20 op/s 2024-04-24T09:23:45.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:23:44 smithi149 ceph-mon[32592]: pgmap v461: 97 pgs: 97 active+clean; 15 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 20 op/s 2024-04-24T09:23:45.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:23:44 smithi114 ceph-mon[32501]: pgmap v461: 97 pgs: 97 active+clean; 15 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 20 op/s 2024-04-24T09:23:47.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:23:46 smithi090 ceph-mon[27825]: pgmap v462: 97 pgs: 97 active+clean; 17 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 23 op/s 2024-04-24T09:23:47.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:23:46 smithi149 ceph-mon[32592]: pgmap v462: 97 pgs: 97 active+clean; 17 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 23 op/s 2024-04-24T09:23:47.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:23:46 smithi114 ceph-mon[32501]: pgmap v462: 97 pgs: 97 active+clean; 17 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 23 op/s 2024-04-24T09:23:49.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:23:48 smithi090 ceph-mon[27825]: pgmap v463: 97 pgs: 97 active+clean; 18 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 23 op/s 2024-04-24T09:23:49.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:23:48 smithi149 ceph-mon[32592]: pgmap v463: 97 pgs: 97 active+clean; 18 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 23 op/s 2024-04-24T09:23:49.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:23:48 smithi114 ceph-mon[32501]: pgmap v463: 97 pgs: 97 active+clean; 18 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 23 op/s 2024-04-24T09:23:51.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:23:50 smithi090 ceph-mon[27825]: pgmap v464: 97 pgs: 97 active+clean; 18 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 478 KiB/s wr, 27 op/s 2024-04-24T09:23:51.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:23:50 smithi149 ceph-mon[32592]: pgmap v464: 97 pgs: 97 active+clean; 18 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 478 KiB/s wr, 27 op/s 2024-04-24T09:23:51.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:23:50 smithi114 ceph-mon[32501]: pgmap v464: 97 pgs: 97 active+clean; 18 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 478 KiB/s wr, 27 op/s 2024-04-24T09:23:53.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:23:52 smithi090 ceph-mon[27825]: pgmap v465: 97 pgs: 97 active+clean; 20 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 524 KiB/s wr, 33 op/s 2024-04-24T09:23:53.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:23:52 smithi149 ceph-mon[32592]: pgmap v465: 97 pgs: 97 active+clean; 20 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 524 KiB/s wr, 33 op/s 2024-04-24T09:23:53.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:23:52 smithi114 ceph-mon[32501]: pgmap v465: 97 pgs: 97 active+clean; 20 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 524 KiB/s wr, 33 op/s 2024-04-24T09:23:54.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:23:53 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:23:54.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:23:53 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:23:54.175 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:23:53 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:23:55.101 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:23:54 smithi090 ceph-mon[27825]: pgmap v466: 97 pgs: 97 active+clean; 20 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 516 KiB/s wr, 30 op/s 2024-04-24T09:23:55.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:23:54 smithi149 ceph-mon[32592]: pgmap v466: 97 pgs: 97 active+clean; 20 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 516 KiB/s wr, 30 op/s 2024-04-24T09:23:55.175 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:23:54 smithi114 ceph-mon[32501]: pgmap v466: 97 pgs: 97 active+clean; 20 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 516 KiB/s wr, 30 op/s 2024-04-24T09:23:56.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:23:55 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:23:56.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:23:55 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:23:56.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:23:55 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:23:56.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:23:55 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:23:56.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:23:55 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:23:56.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:23:55 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:23:56.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:23:55 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:23:56.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:23:55 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:23:56.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:23:55 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:23:57.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:23:56 smithi090 ceph-mon[27825]: pgmap v467: 97 pgs: 97 active+clean; 20 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 515 KiB/s wr, 31 op/s 2024-04-24T09:23:57.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:23:56 smithi149 ceph-mon[32592]: pgmap v467: 97 pgs: 97 active+clean; 20 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 515 KiB/s wr, 31 op/s 2024-04-24T09:23:57.167 INFO:tasks.workunit.client.0.smithi090.stderr:Updating files: 7% (321/4290) Updating files: 8% (344/4290) Updating files: 9% (387/4290) Updating files: 10% (429/4290) Updating files: 11% (472/4290) Updating files: 11% (490/4290) Updating files: 12% (515/4290) Updating files: 13% (558/4290) Updating files: 14% (601/4290) Updating files: 15% (644/4290) Updating files: 16% (687/4290) Updating files: 17% (730/4290) Updating files: 18% (773/4290) Updating files: 19% (816/4290) Updating files: 20% (858/4290) Updating files: 21% (901/4290) Updating files: 21% (913/4290) Updating files: 22% (944/4290) Updating files: 23% (987/4290) Updating files: 24% (1030/4290) Updating files: 25% (1073/4290) Updating files: 26% (1116/4290) Updating files: 27% (1159/4290) Updating files: 28% (1202/4290) Updating files: 29% (1245/4290) Updating files: 30% (1287/4290) Updating files: 30% (1288/4290) Updating files: 31% (1330/4290) Updating files: 32% (1373/4290) Updating files: 33% (1416/4290) Updating files: 34% (1459/4290) Updating files: 35% (1502/4290) Updating files: 35% (1523/4290) Updating files: 36% (1545/4290) Updating files: 37% (1588/4290) Updating files: 38% (1631/4290) Updating files: 39% (1674/4290) Updating files: 40% (1716/4290) Updating files: 41% (1759/4290) Updating files: 42% (1802/4290) Updating files: 43% (1845/4290) Updating files: 44% (1888/4290) Updating files: 45% (1931/4290) Updating files: 46% (1974/4290) Updating files: 47% (2017/4290) Updating files: 47% (2054/4290) Updating files: 48% (2060/4290) Updating files: 49% (2103/4290) Updating files: 50% (2145/4290) Updating files: 51% (2188/4290) Updating files: 52% (2231/4290) Updating files: 53% (2274/4290) Updating files: 54% (2317/4290) Updating files: 55% (2360/4290) Updating files: 56% (2403/4290) Updating files: 56% (2420/4290) Updating files: 57% (2446/4290) Updating files: 58% (2489/4290) Updating files: 59% (2532/4290) Updating files: 60% (2574/4290) Updating files: 61% (2617/4290) Updating files: 62% (2660/4290) Updating files: 63% (2703/4290) Updating files: 64% (2746/4290) Updating files: 64% (2784/4290) Updating files: 65% (2789/4290) Updating files: 66% (2832/4290) Updating files: 67% (2875/4290) Updating files: 68% (2918/4290) Updating files: 69% (2961/4290) Updating files: 70% (3003/4290) Updating files: 70% (3022/4290) Updating files: 71% (3046/4290) Updating files: 72% (3089/4290) Updating files: 73% (3132/4290) Updating files: 74% (3175/4290) Updating files: 75% (3218/4290) Updating files: 76% (3261/4290) Updating files: 77% (3304/4290) Updating files: 78% (3347/4290) Updating files: 79% (3390/4290) Updating files: 79% (3406/4290) Updating files: 80% (3432/4290) Updating files: 81% (3475/4290) Updating files: 82% (3518/4290) Updating files: 83% (3561/4290) Updating files: 84% (3604/4290) Updating files: 85% (3647/4290) Updating files: 86% (3690/4290) Updating files: 87% (3733/4290) Updating files: 88% (3776/4290) Updating files: 89% (3819/4290) Updating files: 90% (3861/4290) Updating files: 91% (3904/4290) Updating files: 91% (3940/4290) Updating files: 92% (3947/4290) Updating files: 93% (3990/4290) Updating files: 94% (4033/4290) Updating files: 95% (4076/4290) Updating files: 96% (4119/4290) Updating files: 97% (4162/4290) Updating files: 98% (4205/4290) Updating files: 98% (4220/4290) Updating files: 99% (4248/4290) Updating files: 100% (4290/4290) Updating files: 100% (4290/4290), done. 2024-04-24T09:23:57.175 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:23:56 smithi114 ceph-mon[32501]: pgmap v467: 97 pgs: 97 active+clean; 20 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 515 KiB/s wr, 31 op/s 2024-04-24T09:23:57.186 INFO:tasks.workunit.client.0.smithi090.stderr:+ cd xfstests-dev 2024-04-24T09:23:57.187 INFO:tasks.workunit.client.0.smithi090.stderr:+ git checkout 12973fc04fd10d4af086901e10ffa8e48866b735 2024-04-24T09:23:59.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:23:58 smithi090 ceph-mon[27825]: pgmap v468: 97 pgs: 97 active+clean; 22 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 661 KiB/s wr, 61 op/s 2024-04-24T09:23:59.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:23:58 smithi149 ceph-mon[32592]: pgmap v468: 97 pgs: 97 active+clean; 22 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 661 KiB/s wr, 61 op/s 2024-04-24T09:23:59.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:23:58 smithi114 ceph-mon[32501]: pgmap v468: 97 pgs: 97 active+clean; 22 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 661 KiB/s wr, 61 op/s 2024-04-24T09:24:01.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:24:00 smithi090 ceph-mon[27825]: pgmap v469: 97 pgs: 97 active+clean; 28 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 1.0 MiB/s wr, 98 op/s 2024-04-24T09:24:01.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:24:00 smithi149 ceph-mon[32592]: pgmap v469: 97 pgs: 97 active+clean; 28 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 1.0 MiB/s wr, 98 op/s 2024-04-24T09:24:01.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:24:00 smithi114 ceph-mon[32501]: pgmap v469: 97 pgs: 97 active+clean; 28 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 1.0 MiB/s wr, 98 op/s 2024-04-24T09:24:03.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:24:02 smithi090 ceph-mon[27825]: pgmap v470: 97 pgs: 97 active+clean; 29 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 1.1 MiB/s wr, 145 op/s 2024-04-24T09:24:03.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:24:02 smithi149 ceph-mon[32592]: pgmap v470: 97 pgs: 97 active+clean; 29 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 1.1 MiB/s wr, 145 op/s 2024-04-24T09:24:03.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:24:02 smithi114 ceph-mon[32501]: pgmap v470: 97 pgs: 97 active+clean; 29 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 1.1 MiB/s wr, 145 op/s 2024-04-24T09:24:05.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:24:04 smithi090 ceph-mon[27825]: pgmap v471: 97 pgs: 97 active+clean; 34 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 1.5 MiB/s wr, 237 op/s 2024-04-24T09:24:05.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:24:04 smithi149 ceph-mon[32592]: pgmap v471: 97 pgs: 97 active+clean; 34 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 1.5 MiB/s wr, 237 op/s 2024-04-24T09:24:05.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:24:04 smithi114 ceph-mon[32501]: pgmap v471: 97 pgs: 97 active+clean; 34 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 1.5 MiB/s wr, 237 op/s 2024-04-24T09:24:06.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:24:05 smithi090 ceph-mon[27825]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:24:06.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:24:05 smithi149 ceph-mon[32592]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:24:06.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:24:05 smithi114 ceph-mon[32501]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:24:07.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:24:06 smithi090 ceph-mon[27825]: pgmap v472: 97 pgs: 97 active+clean; 40 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 350 op/s 2024-04-24T09:24:07.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:24:06 smithi149 ceph-mon[32592]: pgmap v472: 97 pgs: 97 active+clean; 40 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 350 op/s 2024-04-24T09:24:07.175 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:24:06 smithi114 ceph-mon[32501]: pgmap v472: 97 pgs: 97 active+clean; 40 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 350 op/s 2024-04-24T09:24:09.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:24:08 smithi090 ceph-mon[27825]: pgmap v473: 97 pgs: 97 active+clean; 44 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 494 op/s 2024-04-24T09:24:09.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:24:08 smithi149 ceph-mon[32592]: pgmap v473: 97 pgs: 97 active+clean; 44 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 494 op/s 2024-04-24T09:24:09.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:24:08 smithi114 ceph-mon[32501]: pgmap v473: 97 pgs: 97 active+clean; 44 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 494 op/s 2024-04-24T09:24:10.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:24:09 smithi090 ceph-mon[27825]: pgmap v474: 97 pgs: 97 active+clean; 50 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 503 op/s 2024-04-24T09:24:10.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:24:09 smithi149 ceph-mon[32592]: pgmap v474: 97 pgs: 97 active+clean; 50 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 503 op/s 2024-04-24T09:24:10.175 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:24:09 smithi114 ceph-mon[32501]: pgmap v474: 97 pgs: 97 active+clean; 50 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 503 op/s 2024-04-24T09:24:10.620 INFO:tasks.workunit.client.0.smithi090.stderr:Note: switching to '12973fc04fd10d4af086901e10ffa8e48866b735'. 2024-04-24T09:24:10.620 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-04-24T09:24:10.620 INFO:tasks.workunit.client.0.smithi090.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-24T09:24:10.620 INFO:tasks.workunit.client.0.smithi090.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-24T09:24:10.620 INFO:tasks.workunit.client.0.smithi090.stderr:state without impacting any branches by switching back to a branch. 2024-04-24T09:24:10.620 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-04-24T09:24:10.620 INFO:tasks.workunit.client.0.smithi090.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-24T09:24:10.620 INFO:tasks.workunit.client.0.smithi090.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-24T09:24:10.620 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-04-24T09:24:10.621 INFO:tasks.workunit.client.0.smithi090.stderr: git switch -c 2024-04-24T09:24:10.621 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-04-24T09:24:10.621 INFO:tasks.workunit.client.0.smithi090.stderr:Or undo this operation with: 2024-04-24T09:24:10.621 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-04-24T09:24:10.621 INFO:tasks.workunit.client.0.smithi090.stderr: git switch - 2024-04-24T09:24:10.621 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-04-24T09:24:10.621 INFO:tasks.workunit.client.0.smithi090.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-24T09:24:10.621 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-04-24T09:24:10.621 INFO:tasks.workunit.client.0.smithi090.stderr:HEAD is now at 12973fc0 btrfs/022: Correct seq number in golden output 2024-04-24T09:24:10.633 INFO:tasks.workunit.client.0.smithi090.stderr:+ make -j4 2024-04-24T09:24:10.659 INFO:tasks.workunit.client.0.smithi090.stdout:libtoolize -cfi 2024-04-24T09:24:11.395 INFO:tasks.workunit.client.0.smithi090.stdout:libtoolize: putting auxiliary files in '.'. 2024-04-24T09:24:11.395 INFO:tasks.workunit.client.0.smithi090.stdout:libtoolize: copying file './config.guess' 2024-04-24T09:24:11.408 INFO:tasks.workunit.client.0.smithi090.stdout:libtoolize: copying file './config.sub' 2024-04-24T09:24:11.423 INFO:tasks.workunit.client.0.smithi090.stdout:libtoolize: copying file './install-sh' 2024-04-24T09:24:11.440 INFO:tasks.workunit.client.0.smithi090.stdout:libtoolize: copying file './ltmain.sh' 2024-04-24T09:24:11.521 INFO:tasks.workunit.client.0.smithi090.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-04-24T09:24:11.521 INFO:tasks.workunit.client.0.smithi090.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-04-24T09:24:11.540 INFO:tasks.workunit.client.0.smithi090.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-04-24T09:24:11.559 INFO:tasks.workunit.client.0.smithi090.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-04-24T09:24:11.578 INFO:tasks.workunit.client.0.smithi090.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-04-24T09:24:11.597 INFO:tasks.workunit.client.0.smithi090.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-04-24T09:24:11.622 INFO:tasks.workunit.client.0.smithi090.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-04-24T09:24:11.627 INFO:tasks.workunit.client.0.smithi090.stdout:cp include/install-sh . 2024-04-24T09:24:11.654 INFO:tasks.workunit.client.0.smithi090.stdout:aclocal -I m4 2024-04-24T09:24:13.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:24:12 smithi090 ceph-mon[27825]: pgmap v475: 97 pgs: 97 active+clean; 49 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 494 op/s 2024-04-24T09:24:13.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:24:12 smithi149 ceph-mon[32592]: pgmap v475: 97 pgs: 97 active+clean; 49 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 494 op/s 2024-04-24T09:24:13.175 INFO:tasks.workunit.client.0.smithi090.stdout:autoheader 2024-04-24T09:24:13.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:24:12 smithi114 ceph-mon[32501]: pgmap v475: 97 pgs: 97 active+clean; 49 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 494 op/s 2024-04-24T09:24:13.607 INFO:tasks.workunit.client.0.smithi090.stdout:autoconf 2024-04-24T09:24:13.885 INFO:tasks.workunit.client.0.smithi090.stdout:./configure \ 2024-04-24T09:24:13.886 INFO:tasks.workunit.client.0.smithi090.stdout: --libexecdir=/usr/lib \ 2024-04-24T09:24:13.886 INFO:tasks.workunit.client.0.smithi090.stdout: --exec_prefix=/var/lib 2024-04-24T09:24:14.263 INFO:tasks.workunit.client.0.smithi090.stdout:checking build system type... x86_64-pc-linux-gnu 2024-04-24T09:24:14.264 INFO:tasks.workunit.client.0.smithi090.stdout:checking host system type... x86_64-pc-linux-gnu 2024-04-24T09:24:14.268 INFO:tasks.workunit.client.0.smithi090.stdout:checking how to print strings... printf 2024-04-24T09:24:14.269 INFO:tasks.workunit.client.0.smithi090.stdout:checking for gcc... gcc 2024-04-24T09:24:14.455 INFO:tasks.workunit.client.0.smithi090.stdout:checking whether the C compiler works... yes 2024-04-24T09:24:14.455 INFO:tasks.workunit.client.0.smithi090.stdout:checking for C compiler default output file name... a.out 2024-04-24T09:24:14.538 INFO:tasks.workunit.client.0.smithi090.stdout:checking for suffix of executables... 2024-04-24T09:24:14.627 INFO:tasks.workunit.client.0.smithi090.stdout:checking whether we are cross compiling... no 2024-04-24T09:24:14.709 INFO:tasks.workunit.client.0.smithi090.stdout:checking for suffix of object files... o 2024-04-24T09:24:14.774 INFO:tasks.workunit.client.0.smithi090.stdout:checking whether we are using the GNU C compiler... yes 2024-04-24T09:24:14.835 INFO:tasks.workunit.client.0.smithi090.stdout:checking whether gcc accepts -g... yes 2024-04-24T09:24:14.972 INFO:tasks.workunit.client.0.smithi090.stdout:checking for gcc option to accept ISO C89... none needed 2024-04-24T09:24:14.985 INFO:tasks.workunit.client.0.smithi090.stdout:checking for a sed that does not truncate output... /usr/bin/sed 2024-04-24T09:24:14.996 INFO:tasks.workunit.client.0.smithi090.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-04-24T09:24:15.002 INFO:tasks.workunit.client.0.smithi090.stdout:checking for egrep... /usr/bin/grep -E 2024-04-24T09:24:15.007 INFO:tasks.workunit.client.0.smithi090.stdout:checking for fgrep... /usr/bin/grep -F 2024-04-24T09:24:15.022 INFO:tasks.workunit.client.0.smithi090.stdout:checking for ld used by gcc... /usr/bin/ld 2024-04-24T09:24:15.029 INFO:tasks.workunit.client.0.smithi090.stdout:checking if the linker (/usr/bin/ld) is GNU ld... yes 2024-04-24T09:24:15.042 INFO:tasks.workunit.client.0.smithi090.stdout:checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B 2024-04-24T09:24:15.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:24:14 smithi090 ceph-mon[27825]: pgmap v476: 97 pgs: 97 active+clean; 52 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 476 op/s 2024-04-24T09:24:15.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:24:14 smithi149 ceph-mon[32592]: pgmap v476: 97 pgs: 97 active+clean; 52 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 476 op/s 2024-04-24T09:24:15.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:24:14 smithi114 ceph-mon[32501]: pgmap v476: 97 pgs: 97 active+clean; 52 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 476 op/s 2024-04-24T09:24:16.736 INFO:tasks.workunit.client.0.smithi090.stdout:checking the name lister (/usr/bin/nm -B) interface... BSD nm 2024-04-24T09:24:16.736 INFO:tasks.workunit.client.0.smithi090.stdout:checking whether ln -s works... yes 2024-04-24T09:24:16.753 INFO:tasks.workunit.client.0.smithi090.stdout:checking the maximum length of command line arguments... 1572864 2024-04-24T09:24:16.763 INFO:tasks.workunit.client.0.smithi090.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-04-24T09:24:16.763 INFO:tasks.workunit.client.0.smithi090.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-04-24T09:24:16.763 INFO:tasks.workunit.client.0.smithi090.stdout:checking for /usr/bin/ld option to reload object files... -r 2024-04-24T09:24:16.765 INFO:tasks.workunit.client.0.smithi090.stdout:checking for objdump... objdump 2024-04-24T09:24:16.765 INFO:tasks.workunit.client.0.smithi090.stdout:checking how to recognize dependent libraries... pass_all 2024-04-24T09:24:16.767 INFO:tasks.workunit.client.0.smithi090.stdout:checking for dlltool... no 2024-04-24T09:24:16.767 INFO:tasks.workunit.client.0.smithi090.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-04-24T09:24:16.768 INFO:tasks.workunit.client.0.smithi090.stdout:checking for ar... ar 2024-04-24T09:24:16.931 INFO:tasks.workunit.client.0.smithi090.stdout:checking for archiver @FILE support... @ 2024-04-24T09:24:16.933 INFO:tasks.workunit.client.0.smithi090.stdout:checking for strip... strip 2024-04-24T09:24:16.934 INFO:tasks.workunit.client.0.smithi090.stdout:checking for ranlib... ranlib 2024-04-24T09:24:16.934 INFO:tasks.workunit.client.0.smithi090.stdout:checking for gawk... gawk 2024-04-24T09:24:17.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:24:16 smithi090 ceph-mon[27825]: pgmap v477: 97 pgs: 97 active+clean; 52 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 390 op/s 2024-04-24T09:24:17.135 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:24:16 smithi149 ceph-mon[32592]: pgmap v477: 97 pgs: 97 active+clean; 52 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 390 op/s 2024-04-24T09:24:17.170 INFO:tasks.workunit.client.0.smithi090.stdout:checking command to parse /usr/bin/nm -B output from gcc object... ok 2024-04-24T09:24:17.174 INFO:tasks.workunit.client.0.smithi090.stdout:checking for sysroot... no 2024-04-24T09:24:17.175 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:24:16 smithi114 ceph-mon[32501]: pgmap v477: 97 pgs: 97 active+clean; 52 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 390 op/s 2024-04-24T09:24:17.195 INFO:tasks.workunit.client.0.smithi090.stdout:checking for a working dd... /usr/bin/dd 2024-04-24T09:24:17.214 INFO:tasks.workunit.client.0.smithi090.stdout:checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1 2024-04-24T09:24:17.273 INFO:tasks.workunit.client.0.smithi090.stdout:checking for mt... no 2024-04-24T09:24:17.299 INFO:tasks.workunit.client.0.smithi090.stdout:checking if : is a manifest tool... no 2024-04-24T09:24:17.440 INFO:tasks.workunit.client.0.smithi090.stdout:checking how to run the C preprocessor... gcc -E 2024-04-24T09:24:17.845 INFO:tasks.workunit.client.0.smithi090.stdout:checking for ANSI C header files... yes 2024-04-24T09:24:17.926 INFO:tasks.workunit.client.0.smithi090.stdout:checking for sys/types.h... yes 2024-04-24T09:24:18.019 INFO:tasks.workunit.client.0.smithi090.stdout:checking for sys/stat.h... yes 2024-04-24T09:24:18.108 INFO:tasks.workunit.client.0.smithi090.stdout:checking for stdlib.h... yes 2024-04-24T09:24:18.198 INFO:tasks.workunit.client.0.smithi090.stdout:checking for string.h... yes 2024-04-24T09:24:18.288 INFO:tasks.workunit.client.0.smithi090.stdout:checking for memory.h... yes 2024-04-24T09:24:18.381 INFO:tasks.workunit.client.0.smithi090.stdout:checking for strings.h... yes 2024-04-24T09:24:18.473 INFO:tasks.workunit.client.0.smithi090.stdout:checking for inttypes.h... yes 2024-04-24T09:24:18.591 INFO:tasks.workunit.client.0.smithi090.stdout:checking for stdint.h... yes 2024-04-24T09:24:18.691 INFO:tasks.workunit.client.0.smithi090.stdout:checking for unistd.h... yes 2024-04-24T09:24:18.784 INFO:tasks.workunit.client.0.smithi090.stdout:checking for dlfcn.h... yes 2024-04-24T09:24:18.801 INFO:tasks.workunit.client.0.smithi090.stdout:checking for objdir... .libs 2024-04-24T09:24:19.048 INFO:tasks.workunit.client.0.smithi090.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-04-24T09:24:19.048 INFO:tasks.workunit.client.0.smithi090.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-04-24T09:24:19.101 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:24:18 smithi090 ceph-mon[27825]: pgmap v478: 97 pgs: 97 active+clean; 52 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 289 op/s 2024-04-24T09:24:19.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:24:18 smithi149 ceph-mon[32592]: pgmap v478: 97 pgs: 97 active+clean; 52 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 289 op/s 2024-04-24T09:24:19.131 INFO:tasks.workunit.client.0.smithi090.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-04-24T09:24:19.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:24:18 smithi114 ceph-mon[32501]: pgmap v478: 97 pgs: 97 active+clean; 52 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 289 op/s 2024-04-24T09:24:19.215 INFO:tasks.workunit.client.0.smithi090.stdout:checking if gcc static flag -static works... no 2024-04-24T09:24:19.349 INFO:tasks.workunit.client.0.smithi090.stdout:checking if gcc supports -c -o file.o... yes 2024-04-24T09:24:19.349 INFO:tasks.workunit.client.0.smithi090.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-04-24T09:24:19.374 INFO:tasks.workunit.client.0.smithi090.stdout:checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-04-24T09:24:19.465 INFO:tasks.workunit.client.0.smithi090.stdout:checking whether -lc should be explicitly linked in... no 2024-04-24T09:24:19.630 INFO:tasks.workunit.client.0.smithi090.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-04-24T09:24:19.630 INFO:tasks.workunit.client.0.smithi090.stdout:checking how to hardcode library paths into programs... immediate 2024-04-24T09:24:19.634 INFO:tasks.workunit.client.0.smithi090.stdout:checking whether stripping libraries is possible... yes 2024-04-24T09:24:19.634 INFO:tasks.workunit.client.0.smithi090.stdout:checking if libtool supports shared libraries... yes 2024-04-24T09:24:19.634 INFO:tasks.workunit.client.0.smithi090.stdout:checking whether to build shared libraries... yes 2024-04-24T09:24:19.634 INFO:tasks.workunit.client.0.smithi090.stdout:checking whether to build static libraries... yes 2024-04-24T09:24:19.656 INFO:tasks.workunit.client.0.smithi090.stdout:checking for gcc... (cached) gcc 2024-04-24T09:24:19.740 INFO:tasks.workunit.client.0.smithi090.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-04-24T09:24:19.741 INFO:tasks.workunit.client.0.smithi090.stdout:checking whether gcc accepts -g... (cached) yes 2024-04-24T09:24:19.743 INFO:tasks.workunit.client.0.smithi090.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-04-24T09:24:19.745 INFO:tasks.workunit.client.0.smithi090.stdout:checking for gmake... /usr/bin/gmake 2024-04-24T09:24:19.745 INFO:tasks.workunit.client.0.smithi090.stderr:./configure: line 11968: test: $(SHELL): binary operator expected 2024-04-24T09:24:19.746 INFO:tasks.workunit.client.0.smithi090.stdout:checking for tar... /bin/tar 2024-04-24T09:24:19.747 INFO:tasks.workunit.client.0.smithi090.stdout:checking for gzip... /bin/gzip 2024-04-24T09:24:19.836 INFO:tasks.workunit.client.0.smithi090.stdout:checking whether gcc -MM is supported... yes 2024-04-24T09:24:19.838 INFO:tasks.workunit.client.0.smithi090.stdout:checking for sort... /bin/sort 2024-04-24T09:24:19.838 INFO:tasks.workunit.client.0.smithi090.stdout:checking whether ln -s works... yes 2024-04-24T09:24:19.840 INFO:tasks.workunit.client.0.smithi090.stdout:checking for rpm... /bin/rpm 2024-04-24T09:24:19.853 INFO:tasks.workunit.client.0.smithi090.stdout:checking for rpmbuild... no 2024-04-24T09:24:19.853 INFO:tasks.workunit.client.0.smithi090.stdout:checking for ANSI C header files... (cached) yes 2024-04-24T09:24:19.959 INFO:tasks.workunit.client.0.smithi090.stdout:checking assert.h usability... yes 2024-04-24T09:24:19.997 INFO:tasks.workunit.client.0.smithi090.stdout:checking assert.h presence... yes 2024-04-24T09:24:19.997 INFO:tasks.workunit.client.0.smithi090.stdout:checking for assert.h... yes 2024-04-24T09:24:20.117 INFO:tasks.workunit.client.0.smithi090.stdout:checking bstring.h usability... no 2024-04-24T09:24:20.186 INFO:tasks.workunit.client.0.smithi090.stdout:checking bstring.h presence... no 2024-04-24T09:24:20.186 INFO:tasks.workunit.client.0.smithi090.stdout:checking for bstring.h... no 2024-04-24T09:24:20.278 INFO:tasks.workunit.client.0.smithi090.stdout:checking libgen.h usability... yes 2024-04-24T09:24:20.317 INFO:tasks.workunit.client.0.smithi090.stdout:checking libgen.h presence... yes 2024-04-24T09:24:20.317 INFO:tasks.workunit.client.0.smithi090.stdout:checking for libgen.h... yes 2024-04-24T09:24:20.435 INFO:tasks.workunit.client.0.smithi090.stdout:checking dirent.h usability... yes 2024-04-24T09:24:20.476 INFO:tasks.workunit.client.0.smithi090.stdout:checking dirent.h presence... yes 2024-04-24T09:24:20.476 INFO:tasks.workunit.client.0.smithi090.stdout:checking for dirent.h... yes 2024-04-24T09:24:20.587 INFO:tasks.workunit.client.0.smithi090.stdout:checking errno.h usability... yes 2024-04-24T09:24:20.623 INFO:tasks.workunit.client.0.smithi090.stdout:checking errno.h presence... yes 2024-04-24T09:24:20.623 INFO:tasks.workunit.client.0.smithi090.stdout:checking for errno.h... yes 2024-04-24T09:24:20.754 INFO:tasks.workunit.client.0.smithi090.stdout:checking malloc.h usability... yes 2024-04-24T09:24:20.796 INFO:tasks.workunit.client.0.smithi090.stdout:checking malloc.h presence... yes 2024-04-24T09:24:20.797 INFO:tasks.workunit.client.0.smithi090.stdout:checking for malloc.h... yes 2024-04-24T09:24:20.909 INFO:tasks.workunit.client.0.smithi090.stdout:checking uuid.h usability... no 2024-04-24T09:24:20.976 INFO:tasks.workunit.client.0.smithi090.stdout:checking uuid.h presence... no 2024-04-24T09:24:20.976 INFO:tasks.workunit.client.0.smithi090.stdout:checking for uuid.h... no 2024-04-24T09:24:21.076 INFO:tasks.workunit.client.0.smithi090.stdout:checking uuid/uuid.h usability... yes 2024-04-24T09:24:21.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:24:20 smithi090 ceph-mon[27825]: pgmap v479: 97 pgs: 97 active+clean; 53 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1023 KiB/s wr, 157 op/s 2024-04-24T09:24:21.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:24:20 smithi149 ceph-mon[32592]: pgmap v479: 97 pgs: 97 active+clean; 53 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1023 KiB/s wr, 157 op/s 2024-04-24T09:24:21.123 INFO:tasks.workunit.client.0.smithi090.stdout:checking uuid/uuid.h presence... yes 2024-04-24T09:24:21.124 INFO:tasks.workunit.client.0.smithi090.stdout:checking for uuid/uuid.h... yes 2024-04-24T09:24:21.175 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:24:20 smithi114 ceph-mon[32501]: pgmap v479: 97 pgs: 97 active+clean; 53 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1023 KiB/s wr, 157 op/s 2024-04-24T09:24:21.237 INFO:tasks.workunit.client.0.smithi090.stdout:checking sys/uuid.h usability... no 2024-04-24T09:24:21.300 INFO:tasks.workunit.client.0.smithi090.stdout:checking sys/uuid.h presence... no 2024-04-24T09:24:21.300 INFO:tasks.workunit.client.0.smithi090.stdout:checking for sys/uuid.h... no 2024-04-24T09:24:21.393 INFO:tasks.workunit.client.0.smithi090.stdout:checking sys/file.h usability... yes 2024-04-24T09:24:21.433 INFO:tasks.workunit.client.0.smithi090.stdout:checking sys/file.h presence... yes 2024-04-24T09:24:21.433 INFO:tasks.workunit.client.0.smithi090.stdout:checking for sys/file.h... yes 2024-04-24T09:24:21.541 INFO:tasks.workunit.client.0.smithi090.stdout:checking sys/fcntl.h usability... yes 2024-04-24T09:24:21.578 INFO:tasks.workunit.client.0.smithi090.stdout:checking sys/fcntl.h presence... yes 2024-04-24T09:24:21.578 INFO:tasks.workunit.client.0.smithi090.stdout:checking for sys/fcntl.h... yes 2024-04-24T09:24:21.778 INFO:tasks.workunit.client.0.smithi090.stdout:checking sys/syssgi.h usability... no 2024-04-24T09:24:21.842 INFO:tasks.workunit.client.0.smithi090.stdout:checking sys/syssgi.h presence... no 2024-04-24T09:24:21.842 INFO:tasks.workunit.client.0.smithi090.stdout:checking for sys/syssgi.h... no 2024-04-24T09:24:21.975 INFO:tasks.workunit.client.0.smithi090.stdout:checking sys/param.h usability... yes 2024-04-24T09:24:22.019 INFO:tasks.workunit.client.0.smithi090.stdout:checking sys/param.h presence... yes 2024-04-24T09:24:22.019 INFO:tasks.workunit.client.0.smithi090.stdout:checking for sys/param.h... yes 2024-04-24T09:24:22.054 INFO:tasks.workunit.client.0.smithi090.stdout:checking for sys/stat.h... (cached) yes 2024-04-24T09:24:22.141 INFO:tasks.workunit.client.0.smithi090.stdout:checking sys/statvfs.h usability... yes 2024-04-24T09:24:22.180 INFO:tasks.workunit.client.0.smithi090.stdout:checking sys/statvfs.h presence... yes 2024-04-24T09:24:22.180 INFO:tasks.workunit.client.0.smithi090.stdout:checking for sys/statvfs.h... yes 2024-04-24T09:24:22.287 INFO:tasks.workunit.client.0.smithi090.stdout:checking sys/time.h usability... yes 2024-04-24T09:24:22.327 INFO:tasks.workunit.client.0.smithi090.stdout:checking sys/time.h presence... yes 2024-04-24T09:24:22.328 INFO:tasks.workunit.client.0.smithi090.stdout:checking for sys/time.h... yes 2024-04-24T09:24:22.432 INFO:tasks.workunit.client.0.smithi090.stdout:checking sys/ioctl.h usability... yes 2024-04-24T09:24:22.471 INFO:tasks.workunit.client.0.smithi090.stdout:checking sys/ioctl.h presence... yes 2024-04-24T09:24:22.471 INFO:tasks.workunit.client.0.smithi090.stdout:checking for sys/ioctl.h... yes 2024-04-24T09:24:22.843 INFO:tasks.workunit.client.0.smithi090.stdout:checking sys/wait.h usability... yes 2024-04-24T09:24:23.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:24:22 smithi090 ceph-mon[27825]: pgmap v480: 97 pgs: 97 active+clean; 53 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 421 KiB/s wr, 115 op/s 2024-04-24T09:24:23.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:24:22 smithi149 ceph-mon[32592]: pgmap v480: 97 pgs: 97 active+clean; 53 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 421 KiB/s wr, 115 op/s 2024-04-24T09:24:23.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:24:22 smithi114 ceph-mon[32501]: pgmap v480: 97 pgs: 97 active+clean; 53 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 421 KiB/s wr, 115 op/s 2024-04-24T09:24:23.438 INFO:tasks.workunit.client.0.smithi090.stdout:checking sys/wait.h presence... yes 2024-04-24T09:24:23.438 INFO:tasks.workunit.client.0.smithi090.stdout:checking for sys/wait.h... yes 2024-04-24T09:24:23.759 INFO:tasks.workunit.client.0.smithi090.stdout:checking for sys/types.h... (cached) yes 2024-04-24T09:24:23.777 INFO:tasks.workunit.client.0.smithi090.stdout:checking for strings.h... (cached) yes 2024-04-24T09:24:23.867 INFO:tasks.workunit.client.0.smithi090.stdout:checking err.h usability... yes 2024-04-24T09:24:23.905 INFO:tasks.workunit.client.0.smithi090.stdout:checking err.h presence... yes 2024-04-24T09:24:23.905 INFO:tasks.workunit.client.0.smithi090.stdout:checking for err.h... yes 2024-04-24T09:24:24.003 INFO:tasks.workunit.client.0.smithi090.stdout:checking linux/falloc.h usability... yes 2024-04-24T09:24:24.041 INFO:tasks.workunit.client.0.smithi090.stdout:checking linux/falloc.h presence... yes 2024-04-24T09:24:24.041 INFO:tasks.workunit.client.0.smithi090.stdout:checking for linux/falloc.h... yes 2024-04-24T09:24:24.146 INFO:tasks.workunit.client.0.smithi090.stdout:checking sys/fs/xfs_fsops.h usability... no 2024-04-24T09:24:24.221 INFO:tasks.workunit.client.0.smithi090.stdout:checking sys/fs/xfs_fsops.h presence... no 2024-04-24T09:24:24.221 INFO:tasks.workunit.client.0.smithi090.stdout:checking for sys/fs/xfs_fsops.h... no 2024-04-24T09:24:24.306 INFO:tasks.workunit.client.0.smithi090.stdout:checking sys/fs/xfs_itable.h usability... no 2024-04-24T09:24:24.362 INFO:tasks.workunit.client.0.smithi090.stdout:checking sys/fs/xfs_itable.h presence... no 2024-04-24T09:24:24.363 INFO:tasks.workunit.client.0.smithi090.stdout:checking for sys/fs/xfs_itable.h... no 2024-04-24T09:24:24.459 INFO:tasks.workunit.client.0.smithi090.stdout:checking xfs/platform_defs.h usability... no 2024-04-24T09:24:24.521 INFO:tasks.workunit.client.0.smithi090.stdout:checking xfs/platform_defs.h presence... no 2024-04-24T09:24:24.522 INFO:tasks.workunit.client.0.smithi090.stdout:checking for xfs/platform_defs.h... no 2024-04-24T09:24:24.617 INFO:tasks.workunit.client.0.smithi090.stdout:checking btrfs/ioctl.h usability... no 2024-04-24T09:24:24.680 INFO:tasks.workunit.client.0.smithi090.stdout:checking btrfs/ioctl.h presence... no 2024-04-24T09:24:24.680 INFO:tasks.workunit.client.0.smithi090.stdout:checking for btrfs/ioctl.h... no 2024-04-24T09:24:24.777 INFO:tasks.workunit.client.0.smithi090.stdout:checking cifs/ioctl.h usability... no 2024-04-24T09:24:24.844 INFO:tasks.workunit.client.0.smithi090.stdout:checking cifs/ioctl.h presence... no 2024-04-24T09:24:24.845 INFO:tasks.workunit.client.0.smithi090.stdout:checking for cifs/ioctl.h... no 2024-04-24T09:24:24.965 INFO:tasks.workunit.client.0.smithi090.stdout:checking sys/mman.h usability... yes 2024-04-24T09:24:25.008 INFO:tasks.workunit.client.0.smithi090.stdout:checking sys/mman.h presence... yes 2024-04-24T09:24:25.008 INFO:tasks.workunit.client.0.smithi090.stdout:checking for sys/mman.h... yes 2024-04-24T09:24:25.095 INFO:tasks.workunit.client.0.smithi090.stdout:checking for xfs/xfs_log_format.h... no 2024-04-24T09:24:25.101 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:24:24 smithi090 ceph-mon[27825]: pgmap v481: 97 pgs: 97 active+clean; 54 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 552 KiB/s wr, 102 op/s 2024-04-24T09:24:25.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:24:24 smithi149 ceph-mon[32592]: pgmap v481: 97 pgs: 97 active+clean; 54 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 552 KiB/s wr, 102 op/s 2024-04-24T09:24:25.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:24:24 smithi114 ceph-mon[32501]: pgmap v481: 97 pgs: 97 active+clean; 54 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 552 KiB/s wr, 102 op/s 2024-04-24T09:24:25.225 INFO:tasks.workunit.client.0.smithi090.stdout:checking for uuid_compare... no 2024-04-24T09:24:25.320 INFO:tasks.workunit.client.0.smithi090.stdout:checking for uuid_compare in -luuid... yes 2024-04-24T09:24:25.491 INFO:tasks.workunit.client.0.smithi090.stdout:checking for xfs/xfs.h... yes 2024-04-24T09:24:25.570 INFO:tasks.workunit.client.0.smithi090.stdout:checking for xfs/libxfs.h... no 2024-04-24T09:24:25.651 INFO:tasks.workunit.client.0.smithi090.stdout:checking whether xlog_assign_lsn is declared... no 2024-04-24T09:24:25.737 INFO:tasks.workunit.client.0.smithi090.stdout:checking for xfs/xqm.h... yes 2024-04-24T09:24:25.855 INFO:tasks.workunit.client.0.smithi090.stdout:checking xfsctl from xfs/xfs.h... ok 2024-04-24T09:24:25.935 INFO:tasks.workunit.client.0.smithi090.stdout:checking xfs/handle.h usability... yes 2024-04-24T09:24:25.971 INFO:tasks.workunit.client.0.smithi090.stdout:checking xfs/handle.h presence... yes 2024-04-24T09:24:25.971 INFO:tasks.workunit.client.0.smithi090.stdout:checking for xfs/handle.h... yes 2024-04-24T09:24:26.078 INFO:tasks.workunit.client.0.smithi090.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-04-24T09:24:26.194 INFO:tasks.workunit.client.0.smithi090.stdout:checking attr/attributes.h usability... yes 2024-04-24T09:24:26.233 INFO:tasks.workunit.client.0.smithi090.stdout:checking attr/attributes.h presence... yes 2024-04-24T09:24:26.233 INFO:tasks.workunit.client.0.smithi090.stdout:checking for attr/attributes.h... yes 2024-04-24T09:24:26.340 INFO:tasks.workunit.client.0.smithi090.stdout:checking sys/attributes.h usability... no 2024-04-24T09:24:26.403 INFO:tasks.workunit.client.0.smithi090.stdout:checking sys/attributes.h presence... no 2024-04-24T09:24:26.403 INFO:tasks.workunit.client.0.smithi090.stdout:checking for sys/attributes.h... no 2024-04-24T09:24:26.497 INFO:tasks.workunit.client.0.smithi090.stdout:checking for attr_list in -lattr... yes 2024-04-24T09:24:26.588 INFO:tasks.workunit.client.0.smithi090.stdout:checking for attr_set in -lattr... yes 2024-04-24T09:24:26.668 INFO:tasks.workunit.client.0.smithi090.stdout:checking sys/acl.h usability... yes 2024-04-24T09:24:26.708 INFO:tasks.workunit.client.0.smithi090.stdout:checking sys/acl.h presence... yes 2024-04-24T09:24:26.708 INFO:tasks.workunit.client.0.smithi090.stdout:checking for sys/acl.h... yes 2024-04-24T09:24:26.819 INFO:tasks.workunit.client.0.smithi090.stdout:checking acl/libacl.h usability... yes 2024-04-24T09:24:26.860 INFO:tasks.workunit.client.0.smithi090.stdout:checking acl/libacl.h presence... yes 2024-04-24T09:24:26.860 INFO:tasks.workunit.client.0.smithi090.stdout:checking for acl/libacl.h... yes 2024-04-24T09:24:26.959 INFO:tasks.workunit.client.0.smithi090.stdout:checking for acl_init in -lacl... yes 2024-04-24T09:24:27.071 INFO:tasks.workunit.client.0.smithi090.stdout:checking gdbm-ndbm.h usability... no 2024-04-24T09:24:27.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:24:26 smithi090 ceph-mon[27825]: pgmap v482: 97 pgs: 97 active+clean; 60 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 708 KiB/s wr, 113 op/s 2024-04-24T09:24:27.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:24:26 smithi149 ceph-mon[32592]: pgmap v482: 97 pgs: 97 active+clean; 60 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 708 KiB/s wr, 113 op/s 2024-04-24T09:24:27.135 INFO:tasks.workunit.client.0.smithi090.stdout:checking gdbm-ndbm.h presence... no 2024-04-24T09:24:27.135 INFO:tasks.workunit.client.0.smithi090.stdout:checking for gdbm-ndbm.h... no 2024-04-24T09:24:27.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:24:26 smithi114 ceph-mon[32501]: pgmap v482: 97 pgs: 97 active+clean; 60 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 708 KiB/s wr, 113 op/s 2024-04-24T09:24:27.217 INFO:tasks.workunit.client.0.smithi090.stdout:checking gdbm/ndbm.h usability... no 2024-04-24T09:24:27.282 INFO:tasks.workunit.client.0.smithi090.stdout:checking gdbm/ndbm.h presence... no 2024-04-24T09:24:27.282 INFO:tasks.workunit.client.0.smithi090.stdout:checking for gdbm/ndbm.h... no 2024-04-24T09:24:27.369 INFO:tasks.workunit.client.0.smithi090.stdout:checking ndbm.h usability... no 2024-04-24T09:24:27.435 INFO:tasks.workunit.client.0.smithi090.stdout:checking ndbm.h presence... no 2024-04-24T09:24:27.435 INFO:tasks.workunit.client.0.smithi090.stdout:checking for ndbm.h... no 2024-04-24T09:24:27.510 INFO:tasks.workunit.client.0.smithi090.stdout:checking libaio.h usability... yes 2024-04-24T09:24:27.552 INFO:tasks.workunit.client.0.smithi090.stdout:checking libaio.h presence... yes 2024-04-24T09:24:27.552 INFO:tasks.workunit.client.0.smithi090.stdout:checking for libaio.h... yes 2024-04-24T09:24:27.682 INFO:tasks.workunit.client.0.smithi090.stdout:checking for dm_make_handle in -ldm... no 2024-04-24T09:24:27.682 INFO:tasks.workunit.client.0.smithi090.stdout: 2024-04-24T09:24:27.682 INFO:tasks.workunit.client.0.smithi090.stdout:WARNING: could not find a valid DMAPI base library. 2024-04-24T09:24:27.682 INFO:tasks.workunit.client.0.smithi090.stdout:If you want DMAPI support please install the data migration 2024-04-24T09:24:27.682 INFO:tasks.workunit.client.0.smithi090.stdout:API (dmapi) library package. Alternatively, run "make install" 2024-04-24T09:24:27.682 INFO:tasks.workunit.client.0.smithi090.stdout:from the dmapi source. 2024-04-24T09:24:27.682 INFO:tasks.workunit.client.0.smithi090.stdout: 2024-04-24T09:24:28.013 INFO:tasks.workunit.client.0.smithi090.stdout:checking linux/fiemap.h usability... yes 2024-04-24T09:24:28.053 INFO:tasks.workunit.client.0.smithi090.stdout:checking linux/fiemap.h presence... yes 2024-04-24T09:24:28.053 INFO:tasks.workunit.client.0.smithi090.stdout:checking for linux/fiemap.h... yes 2024-04-24T09:24:28.147 INFO:tasks.workunit.client.0.smithi090.stdout:checking for fallocate... yes 2024-04-24T09:24:28.282 INFO:tasks.workunit.client.0.smithi090.stdout:checking for open_by_handle_at... yes 2024-04-24T09:24:28.373 INFO:tasks.workunit.client.0.smithi090.stdout:checking sys/prctl.h usability... yes 2024-04-24T09:24:28.409 INFO:tasks.workunit.client.0.smithi090.stdout:checking sys/prctl.h presence... yes 2024-04-24T09:24:28.409 INFO:tasks.workunit.client.0.smithi090.stdout:checking for sys/prctl.h... yes 2024-04-24T09:24:28.488 INFO:tasks.workunit.client.0.smithi090.stdout:checking linux/fs.h usability... yes 2024-04-24T09:24:28.520 INFO:tasks.workunit.client.0.smithi090.stdout:checking linux/fs.h presence... yes 2024-04-24T09:24:28.520 INFO:tasks.workunit.client.0.smithi090.stdout:checking for linux/fs.h... yes 2024-04-24T09:24:28.647 INFO:tasks.workunit.client.0.smithi090.stdout:checking btrfsutil.h usability... no 2024-04-24T09:24:28.778 INFO:tasks.workunit.client.0.smithi090.stdout:checking btrfsutil.h presence... no 2024-04-24T09:24:28.779 INFO:tasks.workunit.client.0.smithi090.stdout:checking for btrfsutil.h... no 2024-04-24T09:24:28.888 INFO:tasks.workunit.client.0.smithi090.stdout:checking for copy_file_range... yes 2024-04-24T09:24:29.000 INFO:tasks.workunit.client.0.smithi090.stdout:checking for renameat2... yes 2024-04-24T09:24:29.042 INFO:tasks.workunit.client.0.smithi090.stdout:configure: creating ./config.status 2024-04-24T09:24:29.101 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:24:28 smithi090 ceph-mon[27825]: pgmap v483: 97 pgs: 97 active+clean; 63 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 1008 KiB/s wr, 176 op/s 2024-04-24T09:24:29.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:24:28 smithi149 ceph-mon[32592]: pgmap v483: 97 pgs: 97 active+clean; 63 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 1008 KiB/s wr, 176 op/s 2024-04-24T09:24:29.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:24:28 smithi114 ceph-mon[32501]: pgmap v483: 97 pgs: 97 active+clean; 63 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 1008 KiB/s wr, 176 op/s 2024-04-24T09:24:30.145 INFO:tasks.workunit.client.0.smithi090.stdout:config.status: creating include/builddefs 2024-04-24T09:24:30.198 INFO:tasks.workunit.client.0.smithi090.stdout:config.status: creating include/config.h 2024-04-24T09:24:30.235 INFO:tasks.workunit.client.0.smithi090.stdout:config.status: executing libtool commands 2024-04-24T09:24:30.405 INFO:tasks.workunit.client.0.smithi090.stdout:Building include 2024-04-24T09:24:30.405 INFO:tasks.workunit.client.0.smithi090.stdout:Building lib 2024-04-24T09:24:30.406 INFO:tasks.workunit.client.0.smithi090.stdout:Building tests 2024-04-24T09:24:30.417 INFO:tasks.workunit.client.0.smithi090.stdout: [LTDEP] 2024-04-24T09:24:30.842 INFO:tasks.workunit.client.0.smithi090.stdout: [CC] dataascii.lo 2024-04-24T09:24:30.842 INFO:tasks.workunit.client.0.smithi090.stdout: [CC] databin.lo 2024-04-24T09:24:30.843 INFO:tasks.workunit.client.0.smithi090.stdout: [CC] datapid.lo 2024-04-24T09:24:30.843 INFO:tasks.workunit.client.0.smithi090.stdout: [CC] file_lock.lo 2024-04-24T09:24:31.075 INFO:tasks.workunit.client.0.smithi090.stdout: [CC] forker.lo 2024-04-24T09:24:31.090 INFO:tasks.workunit.client.0.smithi090.stdout: [CC] pattern.lo 2024-04-24T09:24:31.101 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:24:30 smithi090 ceph-mon[27825]: pgmap v484: 97 pgs: 97 active+clean; 66 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 1.4 MiB/s wr, 189 op/s 2024-04-24T09:24:31.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:24:30 smithi149 ceph-mon[32592]: pgmap v484: 97 pgs: 97 active+clean; 66 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 1.4 MiB/s wr, 189 op/s 2024-04-24T09:24:31.140 INFO:tasks.workunit.client.0.smithi090.stdout: [CC] open_flags.lo 2024-04-24T09:24:31.158 INFO:tasks.workunit.client.0.smithi090.stdout: [CC] random_range.lo 2024-04-24T09:24:31.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:24:30 smithi114 ceph-mon[32501]: pgmap v484: 97 pgs: 97 active+clean; 66 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 1.4 MiB/s wr, 189 op/s 2024-04-24T09:24:31.337 INFO:tasks.workunit.client.0.smithi090.stdout: [CC] string_to_tokens.lo 2024-04-24T09:24:31.355 INFO:tasks.workunit.client.0.smithi090.stdout: [CC] str_to_bytes.lo 2024-04-24T09:24:31.425 INFO:tasks.workunit.client.0.smithi090.stdout: [CC] tlibio.lo 2024-04-24T09:24:31.557 INFO:tasks.workunit.client.0.smithi090.stdout: [CC] write_log.lo 2024-04-24T09:24:31.558 INFO:tasks.workunit.client.0.smithi090.stdout: [CC] random.lo 2024-04-24T09:24:31.925 INFO:tasks.workunit.client.0.smithi090.stdout: [LD] 2024-04-24T09:24:32.159 INFO:tasks.workunit.client.0.smithi090.stdout:Building ltp 2024-04-24T09:24:32.160 INFO:tasks.workunit.client.0.smithi090.stdout:Building src 2024-04-24T09:24:32.160 INFO:tasks.workunit.client.0.smithi090.stdout:Building m4 2024-04-24T09:24:32.162 INFO:tasks.workunit.client.0.smithi090.stdout:Building common 2024-04-24T09:24:32.185 INFO:tasks.workunit.client.0.smithi090.stdout: [DEP] 2024-04-24T09:24:32.195 INFO:tasks.workunit.client.0.smithi090.stdout: [DEP] 2024-04-24T09:24:32.630 INFO:tasks.workunit.client.0.smithi090.stdout: [CC] doio 2024-04-24T09:24:32.631 INFO:tasks.workunit.client.0.smithi090.stdout: [CC] fsstress 2024-04-24T09:24:32.635 INFO:tasks.workunit.client.0.smithi090.stdout: [CC] fsx 2024-04-24T09:24:32.931 INFO:tasks.workunit.client.0.smithi090.stderr:fsstress.c: In function 'attr_list_path': 2024-04-24T09:24:32.931 INFO:tasks.workunit.client.0.smithi090.stderr:fsstress.c:859:9: warning: 'attr_list' is deprecated: Use listxattr or llistxattr instead [-Wdeprecated-declarations] 2024-04-24T09:24:32.931 INFO:tasks.workunit.client.0.smithi090.stderr: 859 | rval = attr_list(name->path, buffer, buffersize, flags, cursor); 2024-04-24T09:24:32.931 INFO:tasks.workunit.client.0.smithi090.stderr: | ^~~~ 2024-04-24T09:24:32.931 INFO:tasks.workunit.client.0.smithi090.stderr:In file included from ../src/global.h:25, 2024-04-24T09:24:32.931 INFO:tasks.workunit.client.0.smithi090.stderr: from fsstress.c:12: 2024-04-24T09:24:32.931 INFO:tasks.workunit.client.0.smithi090.stderr:/usr/include/attr/attributes.h:165:12: note: declared here 2024-04-24T09:24:32.931 INFO:tasks.workunit.client.0.smithi090.stderr: 165 | extern int attr_list(const char *__path, char *__buffer, const int __buffersize, 2024-04-24T09:24:32.931 INFO:tasks.workunit.client.0.smithi090.stderr: | ^~~~~~~~~ 2024-04-24T09:24:32.931 INFO:tasks.workunit.client.0.smithi090.stderr:fsstress.c: In function 'attr_remove_path': 2024-04-24T09:24:32.932 INFO:tasks.workunit.client.0.smithi090.stderr:fsstress.c:878:9: warning: 'attr_remove' is deprecated: Use removexattr or lremovexattr instead [-Wdeprecated-declarations] 2024-04-24T09:24:32.932 INFO:tasks.workunit.client.0.smithi090.stderr: 878 | rval = attr_remove(name->path, attrname, flags); 2024-04-24T09:24:32.932 INFO:tasks.workunit.client.0.smithi090.stderr: | ^~~~ 2024-04-24T09:24:32.932 INFO:tasks.workunit.client.0.smithi090.stderr:In file included from ../src/global.h:25, 2024-04-24T09:24:32.932 INFO:tasks.workunit.client.0.smithi090.stderr: from fsstress.c:12: 2024-04-24T09:24:32.932 INFO:tasks.workunit.client.0.smithi090.stderr:/usr/include/attr/attributes.h:152:12: note: declared here 2024-04-24T09:24:32.932 INFO:tasks.workunit.client.0.smithi090.stderr: 152 | extern int attr_remove (const char *__path, const char *__attrname, 2024-04-24T09:24:32.932 INFO:tasks.workunit.client.0.smithi090.stderr: | ^~~~~~~~~~~ 2024-04-24T09:24:32.932 INFO:tasks.workunit.client.0.smithi090.stderr:fsstress.c: In function 'attr_set_path': 2024-04-24T09:24:32.932 INFO:tasks.workunit.client.0.smithi090.stderr:fsstress.c:898:9: warning: 'attr_set' is deprecated: Use setxattr or lsetxattr instead [-Wdeprecated-declarations] 2024-04-24T09:24:32.932 INFO:tasks.workunit.client.0.smithi090.stderr: 898 | rval = attr_set(name->path, attrname, attrvalue, valuelength, flags); 2024-04-24T09:24:32.932 INFO:tasks.workunit.client.0.smithi090.stderr: | ^~~~ 2024-04-24T09:24:32.932 INFO:tasks.workunit.client.0.smithi090.stderr:In file included from ../src/global.h:25, 2024-04-24T09:24:32.932 INFO:tasks.workunit.client.0.smithi090.stderr: from fsstress.c:12: 2024-04-24T09:24:32.932 INFO:tasks.workunit.client.0.smithi090.stderr:/usr/include/attr/attributes.h:139:12: note: declared here 2024-04-24T09:24:32.933 INFO:tasks.workunit.client.0.smithi090.stderr: 139 | extern int attr_set (const char *__path, const char *__attrname, 2024-04-24T09:24:32.933 INFO:tasks.workunit.client.0.smithi090.stderr: | ^~~~~~~~ 2024-04-24T09:24:32.942 INFO:tasks.workunit.client.0.smithi090.stderr:fsstress.c: In function 'allocsp_f': 2024-04-24T09:24:32.942 INFO:tasks.workunit.client.0.smithi090.stderr:fsstress.c:2047:32: error: 'XFS_IOC_ALLOCSP64' undeclared (first use in this function); did you mean 'XFS_IOC_RESVSP64'? 2024-04-24T09:24:32.942 INFO:tasks.workunit.client.0.smithi090.stderr: 2047 | e = xfsctl(f.path, fd, XFS_IOC_ALLOCSP64, &fl) < 0 ? errno : 0; 2024-04-24T09:24:32.942 INFO:tasks.workunit.client.0.smithi090.stderr: | ^~~~~~~~~~~~~~~~~ 2024-04-24T09:24:32.943 INFO:tasks.workunit.client.0.smithi090.stderr: | XFS_IOC_RESVSP64 2024-04-24T09:24:32.943 INFO:tasks.workunit.client.0.smithi090.stderr:fsstress.c:2047:32: note: each undeclared identifier is reported only once for each function it appears in 2024-04-24T09:24:32.961 INFO:tasks.workunit.client.0.smithi090.stderr:fsstress.c: In function 'freesp_f': 2024-04-24T09:24:32.962 INFO:tasks.workunit.client.0.smithi090.stderr:fsstress.c:3640:32: error: 'XFS_IOC_FREESP64' undeclared (first use in this function); did you mean 'XFS_IOC_RESVSP64'? 2024-04-24T09:24:32.962 INFO:tasks.workunit.client.0.smithi090.stderr: 3640 | e = xfsctl(f.path, fd, XFS_IOC_FREESP64, &fl) < 0 ? errno : 0; 2024-04-24T09:24:32.962 INFO:tasks.workunit.client.0.smithi090.stderr: | ^~~~~~~~~~~~~~~~ 2024-04-24T09:24:32.962 INFO:tasks.workunit.client.0.smithi090.stderr: | XFS_IOC_RESVSP64 2024-04-24T09:24:33.005 INFO:tasks.workunit.client.0.smithi090.stderr:gmake[3]: *** [Makefile:46: fsstress] Error 1 2024-04-24T09:24:33.005 INFO:tasks.workunit.client.0.smithi090.stderr:gmake[3]: *** Waiting for unfinished jobs.... 2024-04-24T09:24:33.101 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:24:32 smithi090 ceph-mon[27825]: pgmap v485: 97 pgs: 97 active+clean; 66 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 1.3 MiB/s wr, 203 op/s 2024-04-24T09:24:33.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:24:32 smithi149 ceph-mon[32592]: pgmap v485: 97 pgs: 97 active+clean; 66 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 1.3 MiB/s wr, 203 op/s 2024-04-24T09:24:33.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:24:32 smithi114 ceph-mon[32501]: pgmap v485: 97 pgs: 97 active+clean; 66 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 1.3 MiB/s wr, 203 op/s 2024-04-24T09:24:33.380 INFO:tasks.workunit.client.0.smithi090.stderr:doio.c: In function 'parse_cmdline': 2024-04-24T09:24:33.380 INFO:tasks.workunit.client.0.smithi090.stderr:doio.c:3113:33: warning: '%s' directive argument is null [-Wformat-overflow=] 2024-04-24T09:24:33.381 INFO:tasks.workunit.client.0.smithi090.stderr: 3113 | fprintf(stderr, 2024-04-24T09:24:33.381 INFO:tasks.workunit.client.0.smithi090.stderr: | ^~~~~~~~~~~~~~~ 2024-04-24T09:24:33.381 INFO:tasks.workunit.client.0.smithi090.stderr: 3114 | "%s%s: Illegal -C arg (%s). Must be one of: ", 2024-04-24T09:24:33.381 INFO:tasks.workunit.client.0.smithi090.stderr: | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-24T09:24:33.381 INFO:tasks.workunit.client.0.smithi090.stderr: 3115 | Prog, TagName, tok); 2024-04-24T09:24:33.381 INFO:tasks.workunit.client.0.smithi090.stderr: | ~~~~~~~~~~~~~~~~~~~ 2024-04-24T09:24:33.418 INFO:tasks.workunit.client.0.smithi090.stderr:fsx.c: In function 'main': 2024-04-24T09:24:33.418 INFO:tasks.workunit.client.0.smithi090.stderr:fsx.c:2662:33: warning: 'strncpy' specified bound 4096 equals destination size [-Wstringop-truncation] 2024-04-24T09:24:33.418 INFO:tasks.workunit.client.0.smithi090.stderr: 2662 | strncpy(opsfile, optarg, sizeof(opsfile)); 2024-04-24T09:24:33.418 INFO:tasks.workunit.client.0.smithi090.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-24T09:24:33.418 INFO:tasks.workunit.client.0.smithi090.stderr:fsx.c:2632:25: warning: 'strncpy' specified bound 1024 equals destination size [-Wstringop-truncation] 2024-04-24T09:24:33.419 INFO:tasks.workunit.client.0.smithi090.stderr: 2632 | strncpy(dname, optarg, sizeof(dname)); 2024-04-24T09:24:33.419 INFO:tasks.workunit.client.0.smithi090.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-24T09:24:33.561 INFO:tasks.workunit.client.0.smithi090.stderr:gmake[2]: *** [include/buildrules:30: ltp] Error 2 2024-04-24T09:24:33.561 INFO:tasks.workunit.client.0.smithi090.stderr:gmake[2]: *** Waiting for unfinished jobs.... 2024-04-24T09:24:35.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:24:34 smithi090 ceph-mon[27825]: pgmap v486: 97 pgs: 97 active+clean; 68 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 1.5 MiB/s wr, 224 op/s 2024-04-24T09:24:35.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:24:34 smithi149 ceph-mon[32592]: pgmap v486: 97 pgs: 97 active+clean; 68 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 1.5 MiB/s wr, 224 op/s 2024-04-24T09:24:35.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:24:34 smithi114 ceph-mon[32501]: pgmap v486: 97 pgs: 97 active+clean; 68 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 1.5 MiB/s wr, 224 op/s 2024-04-24T09:24:35.506 INFO:tasks.workunit.client.0.smithi090.stdout: [DEP] 2024-04-24T09:24:35.511 INFO:tasks.workunit.client.0.smithi090.stdout: [DEP] 2024-04-24T09:24:35.513 INFO:tasks.workunit.client.0.smithi090.stdout: [DEP] 2024-04-24T09:24:36.026 INFO:tasks.workunit.client.0.smithi090.stdout: [CC] dirstress 2024-04-24T09:24:36.032 INFO:tasks.workunit.client.0.smithi090.stdout: [CC] fill 2024-04-24T09:24:36.033 INFO:tasks.workunit.client.0.smithi090.stdout: [CC] fill2 2024-04-24T09:24:36.035 INFO:tasks.workunit.client.0.smithi090.stdout: [CC] getpagesize 2024-04-24T09:24:36.302 INFO:tasks.workunit.client.0.smithi090.stdout: [CC] holes 2024-04-24T09:24:36.334 INFO:tasks.workunit.client.0.smithi090.stdout: [CC] lstat64 2024-04-24T09:24:36.422 INFO:tasks.workunit.client.0.smithi090.stdout: [CC] nametest 2024-04-24T09:24:36.434 INFO:tasks.workunit.client.0.smithi090.stdout: [CC] permname 2024-04-24T09:24:36.665 INFO:tasks.workunit.client.0.smithi090.stdout: [CC] randholes 2024-04-24T09:24:36.668 INFO:tasks.workunit.client.0.smithi090.stdout: [CC] runas 2024-04-24T09:24:36.789 INFO:tasks.workunit.client.0.smithi090.stdout: [CC] truncfile 2024-04-24T09:24:36.816 INFO:tasks.workunit.client.0.smithi090.stdout: [CC] usemem 2024-04-24T09:24:36.998 INFO:tasks.workunit.client.0.smithi090.stdout: [CC] mmapcat 2024-04-24T09:24:37.085 INFO:tasks.workunit.client.0.smithi090.stdout: [CC] append_reader 2024-04-24T09:24:37.100 INFO:tasks.workunit.client.0.smithi090.stdout: [CC] append_writer 2024-04-24T09:24:37.101 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:24:36 smithi090 ceph-mon[27825]: pgmap v487: 97 pgs: 97 active+clean; 68 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 1.4 MiB/s wr, 206 op/s 2024-04-24T09:24:37.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:24:36 smithi149 ceph-mon[32592]: pgmap v487: 97 pgs: 97 active+clean; 68 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 1.4 MiB/s wr, 206 op/s 2024-04-24T09:24:37.135 INFO:tasks.workunit.client.0.smithi090.stdout: [CC] dirperf 2024-04-24T09:24:37.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:24:36 smithi114 ceph-mon[32501]: pgmap v487: 97 pgs: 97 active+clean; 68 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 1.4 MiB/s wr, 206 op/s 2024-04-24T09:24:37.295 INFO:tasks.workunit.client.0.smithi090.stdout: [CC] metaperf 2024-04-24T09:24:37.370 INFO:tasks.workunit.client.0.smithi090.stdout: [CC] devzero 2024-04-24T09:24:37.384 INFO:tasks.workunit.client.0.smithi090.stdout: [CC] feature 2024-04-24T09:24:37.543 INFO:tasks.workunit.client.0.smithi090.stdout: [CC] alloc 2024-04-24T09:24:37.699 INFO:tasks.workunit.client.0.smithi090.stdout: [CC] fault 2024-04-24T09:24:37.733 INFO:tasks.workunit.client.0.smithi090.stdout: [CC] fstest 2024-04-24T09:24:37.748 INFO:tasks.workunit.client.0.smithi090.stdout: [CC] t_access_root 2024-04-24T09:24:37.793 INFO:tasks.workunit.client.0.smithi090.stderr:alloc.c: In function 'main': 2024-04-24T09:24:37.793 INFO:tasks.workunit.client.0.smithi090.stderr:alloc.c:154:1: error: 'bozo' undeclared (first use in this function) 2024-04-24T09:24:37.793 INFO:tasks.workunit.client.0.smithi090.stderr: 154 | bozo! 2024-04-24T09:24:37.793 INFO:tasks.workunit.client.0.smithi090.stderr: | ^~~~ 2024-04-24T09:24:37.793 INFO:tasks.workunit.client.0.smithi090.stderr:alloc.c:154:1: note: each undeclared identifier is reported only once for each function it appears in 2024-04-24T09:24:37.793 INFO:tasks.workunit.client.0.smithi090.stderr:alloc.c:154:5: error: expected ';' before '!' token 2024-04-24T09:24:37.794 INFO:tasks.workunit.client.0.smithi090.stderr: 154 | bozo! 2024-04-24T09:24:37.794 INFO:tasks.workunit.client.0.smithi090.stderr: | ^ 2024-04-24T09:24:37.794 INFO:tasks.workunit.client.0.smithi090.stderr: | ; 2024-04-24T09:24:37.794 INFO:tasks.workunit.client.0.smithi090.stderr:alloc.c:184:25: error: 'rflag' undeclared (first use in this function); did you mean 'nflag'? 2024-04-24T09:24:37.794 INFO:tasks.workunit.client.0.smithi090.stderr: 184 | rflag = 1; 2024-04-24T09:24:37.794 INFO:tasks.workunit.client.0.smithi090.stderr: | ^~~~~ 2024-04-24T09:24:37.794 INFO:tasks.workunit.client.0.smithi090.stderr: | nflag 2024-04-24T09:24:37.795 INFO:tasks.workunit.client.0.smithi090.stderr:alloc.c:320:5: error: expected ';' before '!' token 2024-04-24T09:24:37.795 INFO:tasks.workunit.client.0.smithi090.stderr: 320 | bozo! 2024-04-24T09:24:37.795 INFO:tasks.workunit.client.0.smithi090.stderr: | ^ 2024-04-24T09:24:37.795 INFO:tasks.workunit.client.0.smithi090.stderr: | ; 2024-04-24T09:24:37.795 INFO:tasks.workunit.client.0.smithi090.stderr:alloc.c:128:25: warning: variable 'f' set but not used [-Wunused-but-set-variable] 2024-04-24T09:24:37.795 INFO:tasks.workunit.client.0.smithi090.stderr: 128 | struct flock64 f; 2024-04-24T09:24:37.795 INFO:tasks.workunit.client.0.smithi090.stderr: | ^ 2024-04-24T09:24:37.801 INFO:tasks.workunit.client.0.smithi090.stderr:gmake[3]: *** [Makefile:83: alloc] Error 1 2024-04-24T09:24:37.801 INFO:tasks.workunit.client.0.smithi090.stderr:gmake[3]: *** Waiting for unfinished jobs.... 2024-04-24T09:24:38.002 INFO:tasks.workunit.client.0.smithi090.stderr:fstest.c: In function 'run_child': 2024-04-24T09:24:38.002 INFO:tasks.workunit.client.0.smithi090.stderr:fstest.c:95:27: warning: '/file' directive writing 5 bytes into a region of size between 1 and 1024 [-Wformat-overflow=] 2024-04-24T09:24:38.002 INFO:tasks.workunit.client.0.smithi090.stderr: 95 | sprintf(fname, "%s/file%d", dir, fnum); 2024-04-24T09:24:38.002 INFO:tasks.workunit.client.0.smithi090.stderr: | ^~~~~ 2024-04-24T09:24:38.002 INFO:tasks.workunit.client.0.smithi090.stderr:fstest.c:95:24: note: directive argument in the range [0, 2147483647] 2024-04-24T09:24:38.002 INFO:tasks.workunit.client.0.smithi090.stderr: 95 | sprintf(fname, "%s/file%d", dir, fnum); 2024-04-24T09:24:38.002 INFO:tasks.workunit.client.0.smithi090.stderr: | ^~~~~~~~~~~ 2024-04-24T09:24:38.003 INFO:tasks.workunit.client.0.smithi090.stderr:fstest.c:95:9: note: 'sprintf' output between 7 and 1039 bytes into a destination of size 1024 2024-04-24T09:24:38.003 INFO:tasks.workunit.client.0.smithi090.stderr: 95 | sprintf(fname, "%s/file%d", dir, fnum); 2024-04-24T09:24:38.003 INFO:tasks.workunit.client.0.smithi090.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-24T09:24:38.003 INFO:tasks.workunit.client.0.smithi090.stderr:fstest.c:166:27: warning: '/file' directive writing 5 bytes into a region of size between 1 and 1024 [-Wformat-overflow=] 2024-04-24T09:24:38.003 INFO:tasks.workunit.client.0.smithi090.stderr: 166 | sprintf(fname, "%s/file%d", dir, fnum); 2024-04-24T09:24:38.003 INFO:tasks.workunit.client.0.smithi090.stderr: | ^~~~~ 2024-04-24T09:24:38.003 INFO:tasks.workunit.client.0.smithi090.stderr:fstest.c:166:24: note: directive argument in the range [0, 2147483647] 2024-04-24T09:24:38.003 INFO:tasks.workunit.client.0.smithi090.stderr: 166 | sprintf(fname, "%s/file%d", dir, fnum); 2024-04-24T09:24:38.003 INFO:tasks.workunit.client.0.smithi090.stderr: | ^~~~~~~~~~~ 2024-04-24T09:24:38.003 INFO:tasks.workunit.client.0.smithi090.stderr:fstest.c:166:9: note: 'sprintf' output between 7 and 1039 bytes into a destination of size 1024 2024-04-24T09:24:38.003 INFO:tasks.workunit.client.0.smithi090.stderr: 166 | sprintf(fname, "%s/file%d", dir, fnum); 2024-04-24T09:24:38.003 INFO:tasks.workunit.client.0.smithi090.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-24T09:24:38.105 INFO:tasks.workunit.client.0.smithi090.stderr:gmake[2]: *** [include/buildrules:30: src] Error 2 2024-04-24T09:24:38.106 INFO:tasks.workunit.client.0.smithi090.stderr:make[1]: *** [Makefile:53: default] Error 2 2024-04-24T09:24:38.106 INFO:tasks.workunit.client.0.smithi090.stderr:make: *** [Makefile:51: default] Error 2 2024-04-24T09:24:38.108 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-24T09:24:38.108 INFO:tasks.workunit:Stopping ['suites/fsx.sh'] on client.0... 2024-04-24T09:24:38.109 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-24T09:24:39.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:24:38 smithi090 ceph-mon[27825]: pgmap v488: 97 pgs: 97 active+clean; 68 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 919 KiB/s wr, 161 op/s 2024-04-24T09:24:39.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:24:38 smithi149 ceph-mon[32592]: pgmap v488: 97 pgs: 97 active+clean; 68 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 919 KiB/s wr, 161 op/s 2024-04-24T09:24:39.126 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_1f81fb397ae98da7563d451a78c61574c8f4e6e0/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_1f81fb397ae98da7563d451a78c61574c8f4e6e0/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_1f81fb397ae98da7563d451a78c61574c8f4e6e0/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/fsx.sh) on smithi090 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1f81fb397ae98da7563d451a78c61574c8f4e6e0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh' 2024-04-24T09:24:39.361 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=8fbee611dfa140be809da4c148483a7b Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_1f81fb397ae98da7563d451a78c61574c8f4e6e0/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_1f81fb397ae98da7563d451a78c61574c8f4e6e0/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_1f81fb397ae98da7563d451a78c61574c8f4e6e0/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/fsx.sh) on smithi090 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1f81fb397ae98da7563d451a78c61574c8f4e6e0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh' 2024-04-24T09:24:39.364 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-24T09:24:39.374 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T09:24:39.379 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:24:38 smithi114 ceph-mon[32501]: pgmap v488: 97 pgs: 97 active+clean; 68 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 919 KiB/s wr, 161 op/s 2024-04-24T09:24:39.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.595+0000 7ff515c00640 1 -- 172.21.15.90:0/2430168597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff510101ea0 msgr2=0x7ff5101022c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:24:39.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.595+0000 7ff515c00640 1 --2- 172.21.15.90:0/2430168597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff510101ea0 0x7ff5101022c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff500009860 tx=0x7ff50002a340 comp rx=0 tx=0).stop 2024-04-24T09:24:39.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.596+0000 7ff515c00640 1 -- 172.21.15.90:0/2430168597 shutdown_connections 2024-04-24T09:24:39.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.596+0000 7ff515c00640 1 --2- 172.21.15.90:0/2430168597 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5101039b0 0x7ff51010a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:24:39.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.596+0000 7ff515c00640 1 --2- 172.21.15.90:0/2430168597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff510102ff0 0x7ff510103470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:24:39.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.596+0000 7ff515c00640 1 --2- 172.21.15.90:0/2430168597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff510101ea0 0x7ff5101022c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:24:39.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.596+0000 7ff515c00640 1 -- 172.21.15.90:0/2430168597 >> 172.21.15.90:0/2430168597 conn(0x7ff5100fd410 msgr2=0x7ff5100ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:24:39.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.596+0000 7ff515c00640 1 -- 172.21.15.90:0/2430168597 shutdown_connections 2024-04-24T09:24:39.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.596+0000 7ff515c00640 1 -- 172.21.15.90:0/2430168597 wait complete. 2024-04-24T09:24:39.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.597+0000 7ff515c00640 1 Processor -- start 2024-04-24T09:24:39.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.597+0000 7ff515c00640 1 -- start start 2024-04-24T09:24:39.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.597+0000 7ff515c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff510101ea0 0x7ff51019a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:24:39.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.597+0000 7ff515c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff510102ff0 0x7ff51019a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:24:39.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.597+0000 7ff515c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5101039b0 0x7ff5101a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:24:39.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.597+0000 7ff515c00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff51010db20 con 0x7ff510101ea0 2024-04-24T09:24:39.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.597+0000 7ff515c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff51010d9a0 con 0x7ff5101039b0 2024-04-24T09:24:39.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.597+0000 7ff515c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff51010dca0 con 0x7ff510102ff0 2024-04-24T09:24:39.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.598+0000 7ff50fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff510101ea0 0x7ff51019a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:24:39.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.598+0000 7ff515200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5101039b0 0x7ff5101a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:24:39.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.598+0000 7ff50fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff510101ea0 0x7ff51019a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50392/0 (socket says 172.21.15.90:50392) 2024-04-24T09:24:39.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.598+0000 7ff50fe00640 1 -- 172.21.15.90:0/3510602559 learned_addr learned my addr 172.21.15.90:0/3510602559 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:24:39.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.598+0000 7ff507e00640 1 --2- 172.21.15.90:0/3510602559 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff510102ff0 0x7ff51019a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:24:39.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.598+0000 7ff50fe00640 1 -- 172.21.15.90:0/3510602559 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff510102ff0 msgr2=0x7ff51019a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:24:39.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.598+0000 7ff50fe00640 1 --2- 172.21.15.90:0/3510602559 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff510102ff0 0x7ff51019a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:24:39.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.598+0000 7ff50fe00640 1 -- 172.21.15.90:0/3510602559 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5101039b0 msgr2=0x7ff5101a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:24:39.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.598+0000 7ff50fe00640 1 --2- 172.21.15.90:0/3510602559 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5101039b0 0x7ff5101a1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:24:39.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.598+0000 7ff50fe00640 1 -- 172.21.15.90:0/3510602559 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5101a1fb0 con 0x7ff510101ea0 2024-04-24T09:24:39.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.598+0000 7ff515200640 1 --2- 172.21.15.90:0/3510602559 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5101039b0 0x7ff5101a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:24:39.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.598+0000 7ff50fe00640 1 --2- 172.21.15.90:0/3510602559 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff510101ea0 0x7ff51019a3e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff500035040 tx=0x7ff500033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:24:39.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.599+0000 7ff50d600640 1 -- 172.21.15.90:0/3510602559 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff500037040 con 0x7ff510101ea0 2024-04-24T09:24:39.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.599+0000 7ff50d600640 1 -- 172.21.15.90:0/3510602559 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff500030850 con 0x7ff510101ea0 2024-04-24T09:24:39.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.599+0000 7ff50d600640 1 -- 172.21.15.90:0/3510602559 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff500004920 con 0x7ff510101ea0 2024-04-24T09:24:39.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.599+0000 7ff515c00640 1 -- 172.21.15.90:0/3510602559 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5101a2240 con 0x7ff510101ea0 2024-04-24T09:24:39.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.599+0000 7ff515c00640 1 -- 172.21.15.90:0/3510602559 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5101a2730 con 0x7ff510101ea0 2024-04-24T09:24:39.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.600+0000 7ff515c00640 1 -- 172.21.15.90:0/3510602559 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5101a2990 con 0x7ff510101ea0 2024-04-24T09:24:39.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.602+0000 7ff50d600640 1 -- 172.21.15.90:0/3510602559 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7ff500004050 con 0x7ff510101ea0 2024-04-24T09:24:39.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.603+0000 7ff50d600640 1 --2- 172.21.15.90:0/3510602559 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7ff4f40a3bb0 0x7ff4f40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:24:39.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.603+0000 7ff50d600640 1 -- 172.21.15.90:0/3510602559 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7ff5000f0310 con 0x7ff510101ea0 2024-04-24T09:24:39.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.604+0000 7ff507e00640 1 --2- 172.21.15.90:0/3510602559 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7ff4f40a3bb0 0x7ff4f40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:24:39.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.604+0000 7ff507e00640 1 --2- 172.21.15.90:0/3510602559 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7ff4f40a3bb0 0x7ff4f40a6070 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff51019b8a0 tx=0x7ff4f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:24:39.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.608+0000 7ff50d600640 1 -- 172.21.15.90:0/3510602559 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff5000bccb0 con 0x7ff510101ea0 2024-04-24T09:24:39.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.808+0000 7ff515c00640 1 -- 172.21.15.90:0/3510602559 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff5101a2b60 con 0x7ff510101ea0 2024-04-24T09:24:39.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.812+0000 7ff50d600640 1 -- 172.21.15.90:0/3510602559 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22652 (secure 0 0 0) 0x7ff5000bc5b0 con 0x7ff510101ea0 2024-04-24T09:24:39.813 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:24:39.813 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":88,"fsid":"30ee9be2-021a-11ef-bc93-c7b262605968","created":"2024-04-24T09:09:49.496019+0000","modified":"2024-04-24T09:19:58.609640+0000","last_up_change":"2024-04-24T09:18:10.455632+0000","last_in_change":"2024-04-24T09:17:50.600366+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T09:13:45.548774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T09:19:07.185580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T09:19:08.374818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13a59b9f-b46e-4ec4-90af-bb7a3d5c753b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6803","nonce":2864288066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6805","nonce":2864288066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6809","nonce":2864288066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6807","nonce":2864288066}]},"public_addr":"172.21.15.90:6803/2864288066","cluster_addr":"172.21.15.90:6805/2864288066","heartbeat_back_addr":"172.21.15.90:6809/2864288066","heartbeat_front_addr":"172.21.15.90:6807/2864288066","state":["exists","up"]},{"osd":1,"uuid":"1567ce8f-24e6-42cf-bf64-f6f0c534a9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6801","nonce":344222414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6803","nonce":344222414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6807","nonce":344222414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6805","nonce":344222414}]},"public_addr":"172.21.15.114:6801/344222414","cluster_addr":"172.21.15.114:6803/344222414","heartbeat_back_addr":"172.21.15.114:6807/344222414","heartbeat_front_addr":"172.21.15.114:6805/344222414","state":["exists","up"]},{"osd":2,"uuid":"f861b304-9c7d-4629-8522-37f33a747945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6801","nonce":3257067419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6803","nonce":3257067419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6807","nonce":3257067419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6805","nonce":3257067419}]},"public_addr":"172.21.15.149:6801/3257067419","cluster_addr":"172.21.15.149:6803/3257067419","heartbeat_back_addr":"172.21.15.149:6807/3257067419","heartbeat_front_addr":"172.21.15.149:6805/3257067419","state":["exists","up"]},{"osd":3,"uuid":"9377ab6e-5996-45fb-93a9-475a2e5f17a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6811","nonce":3068524970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6813","nonce":3068524970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6817","nonce":3068524970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6815","nonce":3068524970}]},"public_addr":"172.21.15.90:6811/3068524970","cluster_addr":"172.21.15.90:6813/3068524970","heartbeat_back_addr":"172.21.15.90:6817/3068524970","heartbeat_front_addr":"172.21.15.90:6815/3068524970","state":["exists","up"]},{"osd":4,"uuid":"e2ddd19e-e641-4d61-b342-02b5f12b1c4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6809","nonce":2782015522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6811","nonce":2782015522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6815","nonce":2782015522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6813","nonce":2782015522}]},"public_addr":"172.21.15.114:6809/2782015522","cluster_addr":"172.21.15.114:6811/2782015522","heartbeat_back_addr":"172.21.15.114:6815/2782015522","heartbeat_front_addr":"172.21.15.114:6813/2782015522","state":["exists","up"]},{"osd":5,"uuid":"b6ff9d10-5210-4b46-a192-6dcbf9d1f156","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6809","nonce":4266460479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6811","nonce":4266460479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6815","nonce":4266460479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6813","nonce":4266460479}]},"public_addr":"172.21.15.149:6809/4266460479","cluster_addr":"172.21.15.149:6811/4266460479","heartbeat_back_addr":"172.21.15.149:6815/4266460479","heartbeat_front_addr":"172.21.15.149:6813/4266460479","state":["exists","up"]},{"osd":6,"uuid":"4287b5a1-f953-421c-bf62-5e390bee834d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6819","nonce":1121461276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6821","nonce":1121461276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6825","nonce":1121461276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6823","nonce":1121461276}]},"public_addr":"172.21.15.90:6819/1121461276","cluster_addr":"172.21.15.90:6821/1121461276","heartbeat_back_addr":"172.21.15.90:6825/1121461276","heartbeat_front_addr":"172.21.15.90:6823/1121461276","state":["exists","up"]},{"osd":7,"uuid":"a54e507d-d6ba-4bf2-9a7f-a7b95c3859bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6817","nonce":3222546280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6819","nonce":3222546280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6823","nonce":3222546280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6821","nonce":3222546280}]},"public_addr":"172.21.15.114:6817/3222546280","cluster_addr":"172.21.15.114:6819/3222546280","heartbeat_back_addr":"172.21.15.114:6823/3222546280","heartbeat_front_addr":"172.21.15.114:6821/3222546280","state":["exists","up"]},{"osd":8,"uuid":"9950b010-5865-4109-ae37-582b1dbd2f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6817","nonce":157615149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6819","nonce":157615149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6823","nonce":157615149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6821","nonce":157615149}]},"public_addr":"172.21.15.149:6817/157615149","cluster_addr":"172.21.15.149:6819/157615149","heartbeat_back_addr":"172.21.15.149:6823/157615149","heartbeat_front_addr":"172.21.15.149:6821/157615149","state":["exists","up"]},{"osd":9,"uuid":"31e2635c-e234-421f-9757-592fd673c936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6827","nonce":1307318076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6829","nonce":1307318076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6833","nonce":1307318076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6831","nonce":1307318076}]},"public_addr":"172.21.15.90:6827/1307318076","cluster_addr":"172.21.15.90:6829/1307318076","heartbeat_back_addr":"172.21.15.90:6833/1307318076","heartbeat_front_addr":"172.21.15.90:6831/1307318076","state":["exists","up"]},{"osd":10,"uuid":"46a3b2a6-e6e5-4578-b4bb-f3e998b9bc95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6825","nonce":1719802140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6827","nonce":1719802140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6831","nonce":1719802140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6829","nonce":1719802140}]},"public_addr":"172.21.15.114:6825/1719802140","cluster_addr":"172.21.15.114:6827/1719802140","heartbeat_back_addr":"172.21.15.114:6831/1719802140","heartbeat_front_addr":"172.21.15.114:6829/1719802140","state":["exists","up"]},{"osd":11,"uuid":"ccd4456b-e947-46cf-a779-148ffa289b58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6825","nonce":1737275166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6827","nonce":1737275166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6831","nonce":1737275166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6829","nonce":1737275166}]},"public_addr":"172.21.15.149:6825/1737275166","cluster_addr":"172.21.15.149:6827/1737275166","heartbeat_back_addr":"172.21.15.149:6831/1737275166","heartbeat_front_addr":"172.21.15.149:6829/1737275166","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:12:53.015078+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:13:18.047053+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:13:41.935891+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:14:15.095388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:14:42.512355+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:15:08.113775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:15:43.828550+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:16:10.901909+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:16:36.422962+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:17:13.924446+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:17:42.814401+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:18:08.672793+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/3090852580":"2024-04-25T09:10:57.377655+0000","172.21.15.90:0/687197394":"2024-04-25T09:10:57.377655+0000","172.21.15.90:6801/1809038230":"2024-04-25T09:10:57.377655+0000","172.21.15.90:6800/1809038230":"2024-04-25T09:10:57.377655+0000","172.21.15.90:0/2964149071":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/3617388113":"2024-04-25T09:10:22.599674+0000","172.21.15.90:6801/3089140842":"2024-04-25T09:10:22.599674+0000","172.21.15.90:6800/3089140842":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/2840416149":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/305432710":"2024-04-25T09:10:57.377655+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T09:24:39.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.815+0000 7ff515c00640 1 -- 172.21.15.90:0/3510602559 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7ff4f40a3bb0 msgr2=0x7ff4f40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:24:39.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.815+0000 7ff515c00640 1 --2- 172.21.15.90:0/3510602559 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7ff4f40a3bb0 0x7ff4f40a6070 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff51019b8a0 tx=0x7ff4f8009260 comp rx=0 tx=0).stop 2024-04-24T09:24:39.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.815+0000 7ff515c00640 1 -- 172.21.15.90:0/3510602559 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff510101ea0 msgr2=0x7ff51019a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:24:39.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.815+0000 7ff515c00640 1 --2- 172.21.15.90:0/3510602559 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff510101ea0 0x7ff51019a3e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff500035040 tx=0x7ff500033d90 comp rx=0 tx=0).stop 2024-04-24T09:24:39.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.815+0000 7ff515c00640 1 -- 172.21.15.90:0/3510602559 shutdown_connections 2024-04-24T09:24:39.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.815+0000 7ff515c00640 1 --2- 172.21.15.90:0/3510602559 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7ff4f40a3bb0 0x7ff4f40a6070 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:24:39.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.815+0000 7ff515c00640 1 --2- 172.21.15.90:0/3510602559 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff5101039b0 0x7ff5101a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:24:39.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.815+0000 7ff515c00640 1 --2- 172.21.15.90:0/3510602559 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff510102ff0 0x7ff51019a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:24:39.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.815+0000 7ff515c00640 1 --2- 172.21.15.90:0/3510602559 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff510101ea0 0x7ff51019a3e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:24:39.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.815+0000 7ff515c00640 1 -- 172.21.15.90:0/3510602559 >> 172.21.15.90:0/3510602559 conn(0x7ff5100fd410 msgr2=0x7ff510074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:24:39.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.815+0000 7ff515c00640 1 -- 172.21.15.90:0/3510602559 shutdown_connections 2024-04-24T09:24:39.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:39.815+0000 7ff515c00640 1 -- 172.21.15.90:0/3510602559 wait complete. 2024-04-24T09:24:39.835 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T09:13:45.548774+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T09:19:07.185580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T09:19:08.374818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T09:24:39.836 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T09:24:40.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.038+0000 7f7a17600640 1 -- 172.21.15.90:0/3757183327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a10105e10 msgr2=0x7f7a1010c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:24:40.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.038+0000 7f7a17600640 1 --2- 172.21.15.90:0/3757183327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a10105e10 0x7f7a1010c6a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7a08009ca0 tx=0x7f7a0802a8c0 comp rx=0 tx=0).stop 2024-04-24T09:24:40.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.039+0000 7f7a17600640 1 -- 172.21.15.90:0/3757183327 shutdown_connections 2024-04-24T09:24:40.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.039+0000 7f7a17600640 1 --2- 172.21.15.90:0/3757183327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a10105e10 0x7f7a1010c6a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:24:40.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.039+0000 7f7a17600640 1 --2- 172.21.15.90:0/3757183327 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a10105450 0x7f7a101058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:24:40.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.039+0000 7f7a17600640 1 --2- 172.21.15.90:0/3757183327 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a10104260 0x7f7a10104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:24:40.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.039+0000 7f7a17600640 1 -- 172.21.15.90:0/3757183327 >> 172.21.15.90:0/3757183327 conn(0x7f7a100ff7b0 msgr2=0x7f7a10101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:24:40.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.039+0000 7f7a17600640 1 -- 172.21.15.90:0/3757183327 shutdown_connections 2024-04-24T09:24:40.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.039+0000 7f7a17600640 1 -- 172.21.15.90:0/3757183327 wait complete. 2024-04-24T09:24:40.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.040+0000 7f7a17600640 1 Processor -- start 2024-04-24T09:24:40.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.040+0000 7f7a17600640 1 -- start start 2024-04-24T09:24:40.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.040+0000 7f7a17600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a10104260 0x7f7a1019a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:24:40.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.040+0000 7f7a17600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a10105450 0x7f7a1019aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:24:40.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.040+0000 7f7a17600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a10105e10 0x7f7a101a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:24:40.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.040+0000 7f7a17600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1010ff40 con 0x7f7a10105450 2024-04-24T09:24:40.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.040+0000 7f7a17600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1010fdc0 con 0x7f7a10105e10 2024-04-24T09:24:40.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.040+0000 7f7a17600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7a101100c0 con 0x7f7a10104260 2024-04-24T09:24:40.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.040+0000 7f7a0fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a10105450 0x7f7a1019aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:24:40.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.040+0000 7f7a14c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a10104260 0x7f7a1019a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:24:40.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.040+0000 7f7a0fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a10105450 0x7f7a1019aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50402/0 (socket says 172.21.15.90:50402) 2024-04-24T09:24:40.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.040+0000 7f7a14c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a10104260 0x7f7a1019a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:49444/0 (socket says 172.21.15.90:49444) 2024-04-24T09:24:40.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.040+0000 7f7a0fe00640 1 -- 172.21.15.90:0/641666625 learned_addr learned my addr 172.21.15.90:0/641666625 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:24:40.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.040+0000 7f7a15600640 1 --2- 172.21.15.90:0/641666625 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a10105e10 0x7f7a101a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:24:40.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.041+0000 7f7a0fe00640 1 -- 172.21.15.90:0/641666625 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a10104260 msgr2=0x7f7a1019a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:24:40.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.041+0000 7f7a0fe00640 1 --2- 172.21.15.90:0/641666625 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a10104260 0x7f7a1019a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:24:40.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.041+0000 7f7a0fe00640 1 -- 172.21.15.90:0/641666625 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a10105e10 msgr2=0x7f7a101a1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:24:40.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.041+0000 7f7a0fe00640 1 --2- 172.21.15.90:0/641666625 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a10105e10 0x7f7a101a1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:24:40.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.041+0000 7f7a0fe00640 1 -- 172.21.15.90:0/641666625 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a101a2150 con 0x7f7a10105450 2024-04-24T09:24:40.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.041+0000 7f7a0fe00640 1 --2- 172.21.15.90:0/641666625 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a10105450 0x7f7a1019aa70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f79fc00e9f0 tx=0x7f79fc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:24:40.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.041+0000 7f7a0d600640 1 -- 172.21.15.90:0/641666625 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79fc00cd80 con 0x7f7a10105450 2024-04-24T09:24:40.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.041+0000 7f7a17600640 1 -- 172.21.15.90:0/641666625 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a101a2440 con 0x7f7a10105450 2024-04-24T09:24:40.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.041+0000 7f7a0d600640 1 -- 172.21.15.90:0/641666625 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79fc005500 con 0x7f7a10105450 2024-04-24T09:24:40.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.041+0000 7f7a0d600640 1 -- 172.21.15.90:0/641666625 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79fc010690 con 0x7f7a10105450 2024-04-24T09:24:40.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.042+0000 7f7a17600640 1 -- 172.21.15.90:0/641666625 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a101a2940 con 0x7f7a10105450 2024-04-24T09:24:40.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.043+0000 7f7a17600640 1 -- 172.21.15.90:0/641666625 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a1010b520 con 0x7f7a10105450 2024-04-24T09:24:40.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.046+0000 7f7a0d600640 1 -- 172.21.15.90:0/641666625 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f79fc0046c0 con 0x7f7a10105450 2024-04-24T09:24:40.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.047+0000 7f7a0d600640 1 --2- 172.21.15.90:0/641666625 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f79f80a3bb0 0x7f79f80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:24:40.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.047+0000 7f7a0d600640 1 -- 172.21.15.90:0/641666625 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f79fc0d0840 con 0x7f7a10105450 2024-04-24T09:24:40.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.048+0000 7f7a14c00640 1 --2- 172.21.15.90:0/641666625 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f79f80a3bb0 0x7f79f80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:24:40.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.048+0000 7f7a14c00640 1 --2- 172.21.15.90:0/641666625 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f79f80a3bb0 0x7f79f80a6070 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7a101052b0 tx=0x7f7a04005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:24:40.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.052+0000 7f7a0d600640 1 -- 172.21.15.90:0/641666625 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f79fc09d1e0 con 0x7f7a10105450 2024-04-24T09:24:40.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.253+0000 7f7a17600640 1 -- 172.21.15.90:0/641666625 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7a10062c00 con 0x7f7a10105450 2024-04-24T09:24:40.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.254+0000 7f7a0d600640 1 -- 172.21.15.90:0/641666625 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f79fc09cae0 con 0x7f7a10105450 2024-04-24T09:24:40.255 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 1 2024-04-24T09:24:40.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.257+0000 7f7a17600640 1 -- 172.21.15.90:0/641666625 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f79f80a3bb0 msgr2=0x7f79f80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:24:40.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.257+0000 7f7a17600640 1 --2- 172.21.15.90:0/641666625 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f79f80a3bb0 0x7f79f80a6070 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7a101052b0 tx=0x7f7a04005f90 comp rx=0 tx=0).stop 2024-04-24T09:24:40.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.257+0000 7f7a17600640 1 -- 172.21.15.90:0/641666625 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a10105450 msgr2=0x7f7a1019aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:24:40.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.257+0000 7f7a17600640 1 --2- 172.21.15.90:0/641666625 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a10105450 0x7f7a1019aa70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f79fc00e9f0 tx=0x7f79fc00edf0 comp rx=0 tx=0).stop 2024-04-24T09:24:40.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.257+0000 7f7a17600640 1 -- 172.21.15.90:0/641666625 shutdown_connections 2024-04-24T09:24:40.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.257+0000 7f7a17600640 1 --2- 172.21.15.90:0/641666625 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f79f80a3bb0 0x7f79f80a6070 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:24:40.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.257+0000 7f7a17600640 1 --2- 172.21.15.90:0/641666625 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a10105e10 0x7f7a101a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:24:40.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.257+0000 7f7a17600640 1 --2- 172.21.15.90:0/641666625 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a10105450 0x7f7a1019aa70 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:24:40.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.257+0000 7f7a17600640 1 --2- 172.21.15.90:0/641666625 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a10104260 0x7f7a1019a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:24:40.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.257+0000 7f7a17600640 1 -- 172.21.15.90:0/641666625 >> 172.21.15.90:0/641666625 conn(0x7f7a100ff7b0 msgr2=0x7f7a101086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:24:40.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.258+0000 7f7a17600640 1 -- 172.21.15.90:0/641666625 shutdown_connections 2024-04-24T09:24:40.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.258+0000 7f7a17600640 1 -- 172.21.15.90:0/641666625 wait complete. 2024-04-24T09:24:40.277 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T09:24:40.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.473+0000 7f68f1400640 1 -- 172.21.15.90:0/2994446598 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68ec0769c0 msgr2=0x7f68ec076de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:24:40.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.473+0000 7f68f1400640 1 --2- 172.21.15.90:0/2994446598 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68ec0769c0 0x7f68ec076de0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f68dc009900 tx=0x7f68dc02a400 comp rx=0 tx=0).stop 2024-04-24T09:24:40.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.473+0000 7f68f1400640 1 -- 172.21.15.90:0/2994446598 shutdown_connections 2024-04-24T09:24:40.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.473+0000 7f68f1400640 1 --2- 172.21.15.90:0/2994446598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68ec0785a0 0x7f68ec078e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:24:40.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.473+0000 7f68f1400640 1 --2- 172.21.15.90:0/2994446598 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f68ec077c00 0x7f68ec078060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:24:40.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.473+0000 7f68f1400640 1 --2- 172.21.15.90:0/2994446598 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68ec0769c0 0x7f68ec076de0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:24:40.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.473+0000 7f68f1400640 1 -- 172.21.15.90:0/2994446598 >> 172.21.15.90:0/2994446598 conn(0x7f68ec0ff7f0 msgr2=0x7f68ec101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:24:40.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.473+0000 7f68f1400640 1 -- 172.21.15.90:0/2994446598 shutdown_connections 2024-04-24T09:24:40.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.473+0000 7f68f1400640 1 -- 172.21.15.90:0/2994446598 wait complete. 2024-04-24T09:24:40.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.474+0000 7f68f1400640 1 Processor -- start 2024-04-24T09:24:40.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.475+0000 7f68f1400640 1 -- start start 2024-04-24T09:24:40.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.475+0000 7f68f1400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f68ec0769c0 0x7f68ec19e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:24:40.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.475+0000 7f68f1400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68ec077c00 0x7f68ec19eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:24:40.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.475+0000 7f68f1400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68ec0785a0 0x7f68ec1a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:24:40.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.475+0000 7f68f1400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f68ec114350 con 0x7f68ec0769c0 2024-04-24T09:24:40.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.475+0000 7f68f1400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f68ec1141d0 con 0x7f68ec077c00 2024-04-24T09:24:40.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.475+0000 7f68f1400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f68ec1144d0 con 0x7f68ec0785a0 2024-04-24T09:24:40.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.475+0000 7f68ea800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f68ec0769c0 0x7f68ec19e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:24:40.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.475+0000 7f68ea800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f68ec0769c0 0x7f68ec19e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50406/0 (socket says 172.21.15.90:50406) 2024-04-24T09:24:40.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.475+0000 7f68ea800640 1 -- 172.21.15.90:0/2298779558 learned_addr learned my addr 172.21.15.90:0/2298779558 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:24:40.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.475+0000 7f68eb200640 1 --2- 172.21.15.90:0/2298779558 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68ec0785a0 0x7f68ec1a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:24:40.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.475+0000 7f68e1e00640 1 --2- 172.21.15.90:0/2298779558 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68ec077c00 0x7f68ec19eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:24:40.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.476+0000 7f68ea800640 1 -- 172.21.15.90:0/2298779558 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68ec0785a0 msgr2=0x7f68ec1a5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:24:40.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.476+0000 7f68ea800640 1 --2- 172.21.15.90:0/2298779558 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68ec0785a0 0x7f68ec1a5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:24:40.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.476+0000 7f68ea800640 1 -- 172.21.15.90:0/2298779558 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68ec077c00 msgr2=0x7f68ec19eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:24:40.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.476+0000 7f68ea800640 1 --2- 172.21.15.90:0/2298779558 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68ec077c00 0x7f68ec19eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:24:40.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.476+0000 7f68ea800640 1 -- 172.21.15.90:0/2298779558 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68ec1a6590 con 0x7f68ec0769c0 2024-04-24T09:24:40.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.476+0000 7f68eb200640 1 --2- 172.21.15.90:0/2298779558 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68ec0785a0 0x7f68ec1a5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:24:40.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.476+0000 7f68ea800640 1 --2- 172.21.15.90:0/2298779558 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f68ec0769c0 0x7f68ec19e970 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f68dc031440 tx=0x7f68dc032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:24:40.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.476+0000 7f68e3e00640 1 -- 172.21.15.90:0/2298779558 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68dc03f030 con 0x7f68ec0769c0 2024-04-24T09:24:40.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.476+0000 7f68e3e00640 1 -- 172.21.15.90:0/2298779558 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68dc003ce0 con 0x7f68ec0769c0 2024-04-24T09:24:40.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.477+0000 7f68f1400640 1 -- 172.21.15.90:0/2298779558 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68ec1a6820 con 0x7f68ec0769c0 2024-04-24T09:24:40.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.477+0000 7f68e3e00640 1 -- 172.21.15.90:0/2298779558 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68dc035070 con 0x7f68ec0769c0 2024-04-24T09:24:40.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.477+0000 7f68f1400640 1 -- 172.21.15.90:0/2298779558 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68ec1a6d10 con 0x7f68ec0769c0 2024-04-24T09:24:40.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.478+0000 7f68f1400640 1 -- 172.21.15.90:0/2298779558 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68ec110bf0 con 0x7f68ec0769c0 2024-04-24T09:24:40.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.480+0000 7f68e3e00640 1 -- 172.21.15.90:0/2298779558 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f68dc042030 con 0x7f68ec0769c0 2024-04-24T09:24:40.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.481+0000 7f68e3e00640 1 --2- 172.21.15.90:0/2298779558 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f68d00a3bb0 0x7f68d00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:24:40.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.481+0000 7f68e3e00640 1 -- 172.21.15.90:0/2298779558 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f68dc0f0cc0 con 0x7f68ec0769c0 2024-04-24T09:24:40.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.481+0000 7f68e1e00640 1 --2- 172.21.15.90:0/2298779558 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f68d00a3bb0 0x7f68d00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:24:40.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.482+0000 7f68e1e00640 1 --2- 172.21.15.90:0/2298779558 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f68d00a3bb0 0x7f68d00a6070 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f68ec19fe80 tx=0x7f68d4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:24:40.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.484+0000 7f68e3e00640 1 -- 172.21.15.90:0/2298779558 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f68dc0bd010 con 0x7f68ec0769c0 2024-04-24T09:24:40.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.684+0000 7f68f1400640 1 -- 172.21.15.90:0/2298779558 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f68ec02d620 con 0x7f68ec0769c0 2024-04-24T09:24:40.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.684+0000 7f68e3e00640 1 -- 172.21.15.90:0/2298779558 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f68dc0c1310 con 0x7f68ec0769c0 2024-04-24T09:24:40.684 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 32 2024-04-24T09:24:40.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.687+0000 7f68f1400640 1 -- 172.21.15.90:0/2298779558 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f68d00a3bb0 msgr2=0x7f68d00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:24:40.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.687+0000 7f68f1400640 1 --2- 172.21.15.90:0/2298779558 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f68d00a3bb0 0x7f68d00a6070 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f68ec19fe80 tx=0x7f68d4005f90 comp rx=0 tx=0).stop 2024-04-24T09:24:40.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.687+0000 7f68f1400640 1 -- 172.21.15.90:0/2298779558 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f68ec0769c0 msgr2=0x7f68ec19e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:24:40.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.687+0000 7f68f1400640 1 --2- 172.21.15.90:0/2298779558 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f68ec0769c0 0x7f68ec19e970 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f68dc031440 tx=0x7f68dc032ce0 comp rx=0 tx=0).stop 2024-04-24T09:24:40.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.687+0000 7f68f1400640 1 -- 172.21.15.90:0/2298779558 shutdown_connections 2024-04-24T09:24:40.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.687+0000 7f68f1400640 1 --2- 172.21.15.90:0/2298779558 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f68d00a3bb0 0x7f68d00a6070 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:24:40.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.687+0000 7f68f1400640 1 --2- 172.21.15.90:0/2298779558 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68ec0785a0 0x7f68ec1a5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:24:40.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.687+0000 7f68f1400640 1 --2- 172.21.15.90:0/2298779558 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f68ec077c00 0x7f68ec19eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:24:40.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.687+0000 7f68f1400640 1 --2- 172.21.15.90:0/2298779558 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f68ec0769c0 0x7f68ec19e970 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:24:40.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.687+0000 7f68f1400640 1 -- 172.21.15.90:0/2298779558 >> 172.21.15.90:0/2298779558 conn(0x7f68ec0ff7f0 msgr2=0x7f68ec102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:24:40.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.688+0000 7f68f1400640 1 -- 172.21.15.90:0/2298779558 shutdown_connections 2024-04-24T09:24:40.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.688+0000 7f68f1400640 1 -- 172.21.15.90:0/2298779558 wait complete. 2024-04-24T09:24:40.706 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T09:24:40.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.915+0000 7fce0bc00640 1 -- 172.21.15.90:0/1701920131 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce04109850 msgr2=0x7fce04109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:24:40.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.915+0000 7fce0bc00640 1 --2- 172.21.15.90:0/1701920131 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce04109850 0x7fce04109cd0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fcdec009860 tx=0x7fcdec02a340 comp rx=0 tx=0).stop 2024-04-24T09:24:40.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.915+0000 7fce0bc00640 1 -- 172.21.15.90:0/1701920131 shutdown_connections 2024-04-24T09:24:40.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.915+0000 7fce0bc00640 1 --2- 172.21.15.90:0/1701920131 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce0410a210 0x7fce04110aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:24:40.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.915+0000 7fce0bc00640 1 --2- 172.21.15.90:0/1701920131 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce04109850 0x7fce04109cd0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:24:40.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.915+0000 7fce0bc00640 1 --2- 172.21.15.90:0/1701920131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce04108660 0x7fce04108a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:24:40.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.915+0000 7fce0bc00640 1 -- 172.21.15.90:0/1701920131 >> 172.21.15.90:0/1701920131 conn(0x7fce04077600 msgr2=0x7fce04079a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:24:40.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.915+0000 7fce0bc00640 1 -- 172.21.15.90:0/1701920131 shutdown_connections 2024-04-24T09:24:40.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.915+0000 7fce0bc00640 1 -- 172.21.15.90:0/1701920131 wait complete. 2024-04-24T09:24:40.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.916+0000 7fce0bc00640 1 Processor -- start 2024-04-24T09:24:40.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.917+0000 7fce0bc00640 1 -- start start 2024-04-24T09:24:40.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.917+0000 7fce0bc00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce04108660 0x7fce0419e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:24:40.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.917+0000 7fce0bc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce04109850 0x7fce0419eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:24:40.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.917+0000 7fce0bc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce0410a210 0x7fce041a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:24:40.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.917+0000 7fce0bc00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fce041142f0 con 0x7fce0410a210 2024-04-24T09:24:40.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.917+0000 7fce0bc00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fce04114170 con 0x7fce04108660 2024-04-24T09:24:40.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.917+0000 7fce0bc00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fce04114470 con 0x7fce04109850 2024-04-24T09:24:40.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.917+0000 7fce09c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce0410a210 0x7fce041a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:24:40.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.917+0000 7fcdfbe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce04109850 0x7fce0419eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:24:40.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.917+0000 7fce09200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce04108660 0x7fce0419e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:24:40.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.917+0000 7fcdfbe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce04109850 0x7fce0419eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:49484/0 (socket says 172.21.15.90:49484) 2024-04-24T09:24:40.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.917+0000 7fce09200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce04108660 0x7fce0419e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.90:43494/0 (socket says 172.21.15.90:43494) 2024-04-24T09:24:40.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.917+0000 7fcdfbe00640 1 -- 172.21.15.90:0/2911974189 learned_addr learned my addr 172.21.15.90:0/2911974189 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:24:40.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.918+0000 7fcdfbe00640 1 -- 172.21.15.90:0/2911974189 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce04108660 msgr2=0x7fce0419e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:24:40.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.918+0000 7fcdfbe00640 1 --2- 172.21.15.90:0/2911974189 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce04108660 0x7fce0419e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:24:40.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.918+0000 7fcdfbe00640 1 -- 172.21.15.90:0/2911974189 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce0410a210 msgr2=0x7fce041a5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:24:40.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.918+0000 7fcdfbe00640 1 --2- 172.21.15.90:0/2911974189 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce0410a210 0x7fce041a5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:24:40.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.918+0000 7fcdfbe00640 1 -- 172.21.15.90:0/2911974189 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce041a6590 con 0x7fce04109850 2024-04-24T09:24:40.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.918+0000 7fce09c00640 1 --2- 172.21.15.90:0/2911974189 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce0410a210 0x7fce041a5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:24:40.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.918+0000 7fcdfbe00640 1 --2- 172.21.15.90:0/2911974189 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce04109850 0x7fce0419eeb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcdec035040 tx=0x7fcdec033d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:24:40.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.919+0000 7fce02000640 1 -- 172.21.15.90:0/2911974189 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdec037040 con 0x7fce04109850 2024-04-24T09:24:40.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.919+0000 7fce0bc00640 1 -- 172.21.15.90:0/2911974189 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce041a6820 con 0x7fce04109850 2024-04-24T09:24:40.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.919+0000 7fce02000640 1 -- 172.21.15.90:0/2911974189 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdec032770 con 0x7fce04109850 2024-04-24T09:24:40.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.919+0000 7fce02000640 1 -- 172.21.15.90:0/2911974189 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdec0046c0 con 0x7fce04109850 2024-04-24T09:24:40.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.919+0000 7fce0bc00640 1 -- 172.21.15.90:0/2911974189 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce041a6d60 con 0x7fce04109850 2024-04-24T09:24:40.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.920+0000 7fce0bc00640 1 -- 172.21.15.90:0/2911974189 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce0410fa00 con 0x7fce04109850 2024-04-24T09:24:40.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.922+0000 7fce02000640 1 -- 172.21.15.90:0/2911974189 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fcdec004050 con 0x7fce04109850 2024-04-24T09:24:40.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.923+0000 7fce02000640 1 --2- 172.21.15.90:0/2911974189 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fcde80a3c30 0x7fcde80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:24:40.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.923+0000 7fce02000640 1 -- 172.21.15.90:0/2911974189 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fcdec0f02f0 con 0x7fce04109850 2024-04-24T09:24:40.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.923+0000 7fce09200640 1 --2- 172.21.15.90:0/2911974189 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fcde80a3c30 0x7fcde80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:24:40.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.924+0000 7fce09200640 1 --2- 172.21.15.90:0/2911974189 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fcde80a3c30 0x7fcde80a60f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fce041096b0 tx=0x7fcdf4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:24:40.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:40.928+0000 7fce02000640 1 -- 172.21.15.90:0/2911974189 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcdec0bcc90 con 0x7fce04109850 2024-04-24T09:24:41.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:24:40 smithi090 ceph-mon[27825]: pgmap v489: 97 pgs: 97 active+clean; 68 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 621 KiB/s wr, 84 op/s 2024-04-24T09:24:41.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:24:40 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/3510602559' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:24:41.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:24:40 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/641666625' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:24:41.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:24:40 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/2298779558' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:24:41.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:24:40 smithi149 ceph-mon[32592]: pgmap v489: 97 pgs: 97 active+clean; 68 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 621 KiB/s wr, 84 op/s 2024-04-24T09:24:41.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:24:40 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/3510602559' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:24:41.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:24:40 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/641666625' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:24:41.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:24:40 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/2298779558' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:24:41.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.131+0000 7fce0bc00640 1 -- 172.21.15.90:0/2911974189 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fce04062c00 con 0x7fce04109850 2024-04-24T09:24:41.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.132+0000 7fce02000640 1 -- 172.21.15.90:0/2911974189 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7fcdec0bc590 con 0x7fce04109850 2024-04-24T09:24:41.132 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 64 2024-04-24T09:24:41.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.134+0000 7fce0bc00640 1 -- 172.21.15.90:0/2911974189 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fcde80a3c30 msgr2=0x7fcde80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:24:41.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.134+0000 7fce0bc00640 1 --2- 172.21.15.90:0/2911974189 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fcde80a3c30 0x7fcde80a60f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fce041096b0 tx=0x7fcdf4009260 comp rx=0 tx=0).stop 2024-04-24T09:24:41.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.134+0000 7fce0bc00640 1 -- 172.21.15.90:0/2911974189 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce04109850 msgr2=0x7fce0419eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:24:41.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.134+0000 7fce0bc00640 1 --2- 172.21.15.90:0/2911974189 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce04109850 0x7fce0419eeb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcdec035040 tx=0x7fcdec033d90 comp rx=0 tx=0).stop 2024-04-24T09:24:41.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.134+0000 7fce0bc00640 1 -- 172.21.15.90:0/2911974189 shutdown_connections 2024-04-24T09:24:41.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.134+0000 7fce0bc00640 1 --2- 172.21.15.90:0/2911974189 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fcde80a3c30 0x7fcde80a60f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:24:41.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.134+0000 7fce0bc00640 1 --2- 172.21.15.90:0/2911974189 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce0410a210 0x7fce041a5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:24:41.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.134+0000 7fce0bc00640 1 --2- 172.21.15.90:0/2911974189 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fce04109850 0x7fce0419eeb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:24:41.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.134+0000 7fce0bc00640 1 --2- 172.21.15.90:0/2911974189 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fce04108660 0x7fce0419e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:24:41.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.134+0000 7fce0bc00640 1 -- 172.21.15.90:0/2911974189 >> 172.21.15.90:0/2911974189 conn(0x7fce04077600 msgr2=0x7fce0410cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:24:41.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.135+0000 7fce0bc00640 1 -- 172.21.15.90:0/2911974189 shutdown_connections 2024-04-24T09:24:41.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.135+0000 7fce0bc00640 1 -- 172.21.15.90:0/2911974189 wait complete. 2024-04-24T09:24:41.153 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-24T09:24:41.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:24:40 smithi114 ceph-mon[32501]: pgmap v489: 97 pgs: 97 active+clean; 68 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 621 KiB/s wr, 84 op/s 2024-04-24T09:24:41.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:24:40 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/3510602559' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:24:41.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:24:40 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/641666625' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:24:41.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:24:40 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/2298779558' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:24:41.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.357+0000 7feab7e00640 1 -- 172.21.15.90:0/1296885792 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feab8107ad0 msgr2=0x7feab8109f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:24:41.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.357+0000 7feab7e00640 1 --2- 172.21.15.90:0/1296885792 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feab8107ad0 0x7feab8109f90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7feab0009be0 tx=0x7feab002a6c0 comp rx=0 tx=0).stop 2024-04-24T09:24:41.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.357+0000 7feab7e00640 1 -- 172.21.15.90:0/1296885792 shutdown_connections 2024-04-24T09:24:41.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.357+0000 7feab7e00640 1 --2- 172.21.15.90:0/1296885792 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feab8107ad0 0x7feab8109f90 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:24:41.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.357+0000 7feab7e00640 1 --2- 172.21.15.90:0/1296885792 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feab8100540 0x7feab8107540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:24:41.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.357+0000 7feab7e00640 1 --2- 172.21.15.90:0/1296885792 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feab80ffbe0 0x7feab8100000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:24:41.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.357+0000 7feab7e00640 1 -- 172.21.15.90:0/1296885792 >> 172.21.15.90:0/1296885792 conn(0x7feab80fb630 msgr2=0x7feab80fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:24:41.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.357+0000 7feab7e00640 1 -- 172.21.15.90:0/1296885792 shutdown_connections 2024-04-24T09:24:41.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.357+0000 7feab7e00640 1 -- 172.21.15.90:0/1296885792 wait complete. 2024-04-24T09:24:41.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.358+0000 7feab7e00640 1 Processor -- start 2024-04-24T09:24:41.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.358+0000 7feab7e00640 1 -- start start 2024-04-24T09:24:41.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.359+0000 7feab7e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feab80ffbe0 0x7feab8102e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:24:41.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.359+0000 7feab7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feab8100540 0x7feab81014d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:24:41.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.359+0000 7feab7e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feab8107ad0 0x7feab8101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:24:41.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.359+0000 7feab7e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7feab810dac0 con 0x7feab8107ad0 2024-04-24T09:24:41.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.359+0000 7feab7e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7feab810d940 con 0x7feab80ffbe0 2024-04-24T09:24:41.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.359+0000 7feab7e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7feab810dc40 con 0x7feab8100540 2024-04-24T09:24:41.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.359+0000 7feab7400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feab8107ad0 0x7feab8101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:24:41.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.359+0000 7feab7400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feab8107ad0 0x7feab8101a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50432/0 (socket says 172.21.15.90:50432) 2024-04-24T09:24:41.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.359+0000 7feab7400640 1 -- 172.21.15.90:0/4109344076 learned_addr learned my addr 172.21.15.90:0/4109344076 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:24:41.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.359+0000 7feab6a00640 1 --2- 172.21.15.90:0/4109344076 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feab80ffbe0 0x7feab8102e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:24:41.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.359+0000 7feab6000640 1 --2- 172.21.15.90:0/4109344076 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feab8100540 0x7feab81014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:24:41.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.359+0000 7feab7400640 1 -- 172.21.15.90:0/4109344076 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feab8100540 msgr2=0x7feab81014d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:24:41.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.359+0000 7feab7400640 1 --2- 172.21.15.90:0/4109344076 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feab8100540 0x7feab81014d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:24:41.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.359+0000 7feab7400640 1 -- 172.21.15.90:0/4109344076 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feab80ffbe0 msgr2=0x7feab8102e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:24:41.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.359+0000 7feab7400640 1 --2- 172.21.15.90:0/4109344076 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feab80ffbe0 0x7feab8102e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:24:41.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.359+0000 7feab7400640 1 -- 172.21.15.90:0/4109344076 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feab81022a0 con 0x7feab8107ad0 2024-04-24T09:24:41.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.359+0000 7feab6a00640 1 --2- 172.21.15.90:0/4109344076 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feab80ffbe0 0x7feab8102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:24:41.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.360+0000 7feab7400640 1 --2- 172.21.15.90:0/4109344076 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feab8107ad0 0x7feab8101a10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7feab0009b30 tx=0x7feab00305d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:24:41.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.360+0000 7feaaf400640 1 -- 172.21.15.90:0/4109344076 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab0041020 con 0x7feab8107ad0 2024-04-24T09:24:41.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.360+0000 7feaaf400640 1 -- 172.21.15.90:0/4109344076 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feab0030740 con 0x7feab8107ad0 2024-04-24T09:24:41.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.360+0000 7feaaf400640 1 -- 172.21.15.90:0/4109344076 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab0036040 con 0x7feab8107ad0 2024-04-24T09:24:41.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.360+0000 7feab7e00640 1 -- 172.21.15.90:0/4109344076 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feab81a21f0 con 0x7feab8107ad0 2024-04-24T09:24:41.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.360+0000 7feab7e00640 1 -- 172.21.15.90:0/4109344076 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feab81a2740 con 0x7feab8107ad0 2024-04-24T09:24:41.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.361+0000 7feab7e00640 1 -- 172.21.15.90:0/4109344076 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feab8106350 con 0x7feab8107ad0 2024-04-24T09:24:41.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.363+0000 7feaaf400640 1 -- 172.21.15.90:0/4109344076 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7feab0044020 con 0x7feab8107ad0 2024-04-24T09:24:41.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.365+0000 7feaaf400640 1 --2- 172.21.15.90:0/4109344076 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fea9c0a3bb0 0x7fea9c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:24:41.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.365+0000 7feaaf400640 1 -- 172.21.15.90:0/4109344076 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7feab00f1080 con 0x7feab8107ad0 2024-04-24T09:24:41.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.365+0000 7feab6a00640 1 --2- 172.21.15.90:0/4109344076 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fea9c0a3bb0 0x7fea9c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:24:41.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.366+0000 7feab6a00640 1 --2- 172.21.15.90:0/4109344076 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fea9c0a3bb0 0x7fea9c0a6070 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7feaa80096c0 tx=0x7feaa8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:24:41.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.369+0000 7feaaf400640 1 -- 172.21.15.90:0/4109344076 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feab004b020 con 0x7feab8107ad0 2024-04-24T09:24:41.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.620+0000 7feab7e00640 1 -- 172.21.15.90:0/4109344076 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7feab8062c00 con 0x7feab8107ad0 2024-04-24T09:24:41.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.652+0000 7feaaf400640 1 -- 172.21.15.90:0/4109344076 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) ==== 81+0+745531 (secure 0 0 0) 0x7feab00bda20 con 0x7feab8107ad0 2024-04-24T09:24:41.660 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:24:41.660 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T09:24:41.660 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 20, 2024-04-24T09:24:41.660 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.660 INFO:teuthology.orchestra.run.smithi090.stdout: "active_gid": 14152, 2024-04-24T09:24:41.661 INFO:teuthology.orchestra.run.smithi090.stdout: "active_name": "x", 2024-04-24T09:24:41.661 INFO:teuthology.orchestra.run.smithi090.stdout: "active_addrs": { 2024-04-24T09:24:41.661 INFO:teuthology.orchestra.run.smithi090.stdout: "addrvec": [ 2024-04-24T09:24:41.661 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:41.661 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v2", 2024-04-24T09:24:41.661 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "172.21.15.90:6800", 2024-04-24T09:24:41.661 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 4261662956 2024-04-24T09:24:41.661 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.661 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:41.661 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v1", 2024-04-24T09:24:41.662 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "172.21.15.90:6801", 2024-04-24T09:24:41.662 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 4261662956 2024-04-24T09:24:41.662 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:41.662 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-04-24T09:24:41.662 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.662 INFO:teuthology.orchestra.run.smithi090.stdout: "active_addr": "172.21.15.90:6801/4261662956", 2024-04-24T09:24:41.662 INFO:teuthology.orchestra.run.smithi090.stdout: "active_change": "2024-04-24T09:10:57.377773+0000", 2024-04-24T09:24:41.662 INFO:teuthology.orchestra.run.smithi090.stdout: "active_mgr_features": 4540701547738038271, 2024-04-24T09:24:41.662 INFO:teuthology.orchestra.run.smithi090.stdout: "available": true, 2024-04-24T09:24:41.663 INFO:teuthology.orchestra.run.smithi090.stdout: "standbys": [ 2024-04-24T09:24:41.663 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:41.663 INFO:teuthology.orchestra.run.smithi090.stdout: "gid": 24109, 2024-04-24T09:24:41.663 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "z", 2024-04-24T09:24:41.663 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr_features": 4540701547738038271, 2024-04-24T09:24:41.663 INFO:teuthology.orchestra.run.smithi090.stdout: "available_modules": [ 2024-04-24T09:24:41.663 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:41.663 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "alerts", 2024-04-24T09:24:41.663 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:41.663 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:41.664 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:41.664 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-04-24T09:24:41.664 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-04-24T09:24:41.664 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:41.664 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.664 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.664 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-04-24T09:24:41.664 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.664 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.664 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.665 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How frequently to reexamine health status", 2024-04-24T09:24:41.665 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.665 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.665 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.665 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.665 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:41.665 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:41.665 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.665 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.666 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.666 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:41.666 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.666 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.666 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:41.666 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:41.666 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:41.666 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:41.666 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:41.667 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:41.667 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:41.667 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:41.667 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.667 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.667 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.667 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.667 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.667 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:41.667 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:41.668 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.668 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.668 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.668 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:41.668 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.668 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.668 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.668 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.669 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.669 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.669 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.669 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.669 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:41.669 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:41.669 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.669 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.669 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.670 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:41.670 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.670 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.670 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:41.670 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:41.670 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:41.670 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:41.670 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:41.670 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:41.671 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:41.671 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:41.671 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.671 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.671 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.671 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.671 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.671 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:41.672 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:41.672 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.672 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.672 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.672 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:41.672 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.672 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.672 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.672 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.672 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.673 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.673 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.673 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.673 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_destination": { 2024-04-24T09:24:41.673 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_destination", 2024-04-24T09:24:41.673 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.673 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.673 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.673 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:41.674 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.674 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.674 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.674 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Email address to send alerts to", 2024-04-24T09:24:41.674 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.674 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.674 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.674 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.675 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_from_name": { 2024-04-24T09:24:41.675 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_from_name", 2024-04-24T09:24:41.675 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.675 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.675 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.675 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "Ceph", 2024-04-24T09:24:41.675 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.675 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.675 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.676 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Email From: name", 2024-04-24T09:24:41.676 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.676 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.676 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.676 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.676 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_host": { 2024-04-24T09:24:41.676 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_host", 2024-04-24T09:24:41.676 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.677 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.677 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.677 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:41.677 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.677 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.677 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.677 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SMTP server", 2024-04-24T09:24:41.677 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.677 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.678 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.678 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.678 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_password": { 2024-04-24T09:24:41.678 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_password", 2024-04-24T09:24:41.678 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.678 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.678 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.678 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:41.679 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.679 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.679 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.679 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Password to authenticate with", 2024-04-24T09:24:41.679 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.679 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.679 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.679 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.679 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_port": { 2024-04-24T09:24:41.679 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_port", 2024-04-24T09:24:41.680 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:41.680 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.680 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.680 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "465", 2024-04-24T09:24:41.680 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.680 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.680 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.680 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SMTP port", 2024-04-24T09:24:41.680 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.681 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.681 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.681 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.681 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_sender": { 2024-04-24T09:24:41.681 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_sender", 2024-04-24T09:24:41.681 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.681 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.681 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.681 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:41.681 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.682 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.682 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.682 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SMTP envelope sender", 2024-04-24T09:24:41.682 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.682 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.682 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.682 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.682 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_ssl": { 2024-04-24T09:24:41.682 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_ssl", 2024-04-24T09:24:41.683 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.683 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.683 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.683 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:41.683 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.683 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.683 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.683 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-24T09:24:41.683 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.683 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.684 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.684 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.684 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_user": { 2024-04-24T09:24:41.684 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_user", 2024-04-24T09:24:41.684 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.684 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.684 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.684 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:41.684 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.685 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.685 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.685 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "User to authenticate as", 2024-04-24T09:24:41.685 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.685 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.685 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.685 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:41.685 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:41.685 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.686 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:41.686 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "balancer", 2024-04-24T09:24:41.686 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:41.686 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:41.686 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:41.686 INFO:teuthology.orchestra.run.smithi090.stdout: "active": { 2024-04-24T09:24:41.686 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "active", 2024-04-24T09:24:41.686 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.687 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.687 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.687 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:41.687 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.687 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.687 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.687 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "automatically balance PGs across cluster", 2024-04-24T09:24:41.687 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.687 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.687 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.688 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.688 INFO:teuthology.orchestra.run.smithi090.stdout: "begin_time": { 2024-04-24T09:24:41.688 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "begin_time", 2024-04-24T09:24:41.688 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.688 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.688 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.688 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0000", 2024-04-24T09:24:41.688 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.688 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.689 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.689 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "beginning time of day to automatically balance", 2024-04-24T09:24:41.689 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T09:24:41.689 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.689 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.689 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.689 INFO:teuthology.orchestra.run.smithi090.stdout: "begin_weekday": { 2024-04-24T09:24:41.689 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "begin_weekday", 2024-04-24T09:24:41.689 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-04-24T09:24:41.689 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.690 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.690 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-04-24T09:24:41.690 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "0", 2024-04-24T09:24:41.690 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "6", 2024-04-24T09:24:41.690 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.690 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-24T09:24:41.690 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T09:24:41.690 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.691 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.691 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.691 INFO:teuthology.orchestra.run.smithi090.stdout: "crush_compat_max_iterations": { 2024-04-24T09:24:41.691 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crush_compat_max_iterations", 2024-04-24T09:24:41.691 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-04-24T09:24:41.691 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.691 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.691 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "25", 2024-04-24T09:24:41.691 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-04-24T09:24:41.691 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "250", 2024-04-24T09:24:41.692 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.692 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-24T09:24:41.692 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.692 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.692 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.692 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.692 INFO:teuthology.orchestra.run.smithi090.stdout: "crush_compat_metrics": { 2024-04-24T09:24:41.692 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crush_compat_metrics", 2024-04-24T09:24:41.692 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.693 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.693 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.693 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "pgs,objects,bytes", 2024-04-24T09:24:41.693 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.693 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.693 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.693 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-24T09:24:41.693 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-24T09:24:41.693 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.694 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.694 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.694 INFO:teuthology.orchestra.run.smithi090.stdout: "crush_compat_step": { 2024-04-24T09:24:41.694 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crush_compat_step", 2024-04-24T09:24:41.694 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-04-24T09:24:41.694 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.694 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.694 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0.5", 2024-04-24T09:24:41.695 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "0.001", 2024-04-24T09:24:41.695 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "0.999", 2024-04-24T09:24:41.695 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.695 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "aggressiveness of optimization", 2024-04-24T09:24:41.695 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-24T09:24:41.695 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.695 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.695 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.695 INFO:teuthology.orchestra.run.smithi090.stdout: "end_time": { 2024-04-24T09:24:41.695 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "end_time", 2024-04-24T09:24:41.696 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.696 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.696 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.696 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "2359", 2024-04-24T09:24:41.696 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.696 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.696 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.696 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "ending time of day to automatically balance", 2024-04-24T09:24:41.696 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T09:24:41.697 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.697 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.697 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.697 INFO:teuthology.orchestra.run.smithi090.stdout: "end_weekday": { 2024-04-24T09:24:41.697 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "end_weekday", 2024-04-24T09:24:41.697 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-04-24T09:24:41.697 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.697 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.697 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-04-24T09:24:41.698 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "0", 2024-04-24T09:24:41.698 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "6", 2024-04-24T09:24:41.698 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.698 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-24T09:24:41.698 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T09:24:41.698 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.698 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.698 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.699 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:41.699 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:41.699 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.699 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.699 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.699 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:41.699 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.699 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.699 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:41.699 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:41.700 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:41.700 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:41.700 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:41.700 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:41.700 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:41.700 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:41.700 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.700 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.700 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.701 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.701 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.701 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:41.701 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:41.701 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.701 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.701 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.701 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:41.702 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.702 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.702 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.702 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.702 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.702 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.702 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.702 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.703 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:41.703 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:41.703 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.703 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.703 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.703 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:41.703 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.703 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.703 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:41.704 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:41.704 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:41.704 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:41.704 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:41.704 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:41.704 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:41.704 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:41.704 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.704 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.705 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.705 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.705 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.705 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:41.705 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:41.705 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.705 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.705 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.706 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:41.706 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.706 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.706 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.706 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.706 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.706 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.706 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.707 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.707 INFO:teuthology.orchestra.run.smithi090.stdout: "min_score": { 2024-04-24T09:24:41.707 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "min_score", 2024-04-24T09:24:41.707 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-04-24T09:24:41.707 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.707 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.707 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-04-24T09:24:41.708 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.708 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.708 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.708 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-24T09:24:41.708 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.708 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.708 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.708 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.709 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": { 2024-04-24T09:24:41.709 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mode", 2024-04-24T09:24:41.709 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.709 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.709 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.709 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "upmap", 2024-04-24T09:24:41.709 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.709 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.709 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:41.710 INFO:teuthology.orchestra.run.smithi090.stdout: "crush-compat", 2024-04-24T09:24:41.710 INFO:teuthology.orchestra.run.smithi090.stdout: "none", 2024-04-24T09:24:41.710 INFO:teuthology.orchestra.run.smithi090.stdout: "read", 2024-04-24T09:24:41.710 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap", 2024-04-24T09:24:41.710 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap-read" 2024-04-24T09:24:41.710 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:41.710 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Balancer mode", 2024-04-24T09:24:41.711 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.711 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.711 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.711 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.711 INFO:teuthology.orchestra.run.smithi090.stdout: "pool_ids": { 2024-04-24T09:24:41.711 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pool_ids", 2024-04-24T09:24:41.711 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.711 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.712 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.712 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:41.712 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.712 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.712 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.712 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-24T09:24:41.712 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.712 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.712 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.713 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.713 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-04-24T09:24:41.713 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-04-24T09:24:41.713 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:41.713 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.713 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.713 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-04-24T09:24:41.713 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.714 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.714 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.714 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-24T09:24:41.714 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.714 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.714 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.714 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.714 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap_max_deviation": { 2024-04-24T09:24:41.715 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "upmap_max_deviation", 2024-04-24T09:24:41.715 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:41.715 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.715 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.715 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-04-24T09:24:41.715 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-04-24T09:24:41.715 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.715 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.716 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-24T09:24:41.716 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-24T09:24:41.716 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.716 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.716 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.716 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap_max_optimizations": { 2024-04-24T09:24:41.716 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "upmap_max_optimizations", 2024-04-24T09:24:41.716 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-04-24T09:24:41.717 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.717 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.717 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-04-24T09:24:41.717 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.717 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.717 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.717 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-24T09:24:41.717 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.717 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.718 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.718 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:41.718 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:41.718 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.718 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:41.718 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cephadm", 2024-04-24T09:24:41.718 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:41.718 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:41.719 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:41.719 INFO:teuthology.orchestra.run.smithi090.stdout: "agent_down_multiplier": { 2024-04-24T09:24:41.719 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "agent_down_multiplier", 2024-04-24T09:24:41.719 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-04-24T09:24:41.719 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.719 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.719 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "3.0", 2024-04-24T09:24:41.719 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.720 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.720 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.720 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-24T09:24:41.720 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.720 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.720 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.720 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.721 INFO:teuthology.orchestra.run.smithi090.stdout: "agent_refresh_rate": { 2024-04-24T09:24:41.721 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "agent_refresh_rate", 2024-04-24T09:24:41.721 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:41.721 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.721 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.721 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "20", 2024-04-24T09:24:41.721 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.721 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.721 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.722 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-24T09:24:41.722 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.722 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.722 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.722 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.722 INFO:teuthology.orchestra.run.smithi090.stdout: "agent_starting_port": { 2024-04-24T09:24:41.722 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "agent_starting_port", 2024-04-24T09:24:41.722 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:41.723 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.723 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.723 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "4721", 2024-04-24T09:24:41.723 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.723 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.723 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.723 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-24T09:24:41.723 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.724 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.724 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.724 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.724 INFO:teuthology.orchestra.run.smithi090.stdout: "allow_ptrace": { 2024-04-24T09:24:41.724 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "allow_ptrace", 2024-04-24T09:24:41.724 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.724 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.724 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.725 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:41.725 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.725 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.725 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.725 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-24T09:24:41.725 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-24T09:24:41.725 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.725 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.725 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.726 INFO:teuthology.orchestra.run.smithi090.stdout: "autotune_interval": { 2024-04-24T09:24:41.726 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "autotune_interval", 2024-04-24T09:24:41.726 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:41.726 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.726 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.726 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-04-24T09:24:41.726 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.726 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.727 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.727 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-24T09:24:41.727 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.727 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.727 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.727 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.727 INFO:teuthology.orchestra.run.smithi090.stdout: "autotune_memory_target_ratio": { 2024-04-24T09:24:41.727 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "autotune_memory_target_ratio", 2024-04-24T09:24:41.727 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-04-24T09:24:41.728 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.728 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.728 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0.7", 2024-04-24T09:24:41.728 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.728 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.728 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.728 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-24T09:24:41.728 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.729 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.729 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.729 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.729 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm_log_destination": { 2024-04-24T09:24:41.729 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cephadm_log_destination", 2024-04-24T09:24:41.729 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.729 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.729 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.729 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:41.730 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.730 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.730 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:41.730 INFO:teuthology.orchestra.run.smithi090.stdout: "file", 2024-04-24T09:24:41.730 INFO:teuthology.orchestra.run.smithi090.stdout: "file,syslog", 2024-04-24T09:24:41.730 INFO:teuthology.orchestra.run.smithi090.stdout: "syslog" 2024-04-24T09:24:41.730 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:41.730 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-24T09:24:41.731 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.731 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.731 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.731 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.731 INFO:teuthology.orchestra.run.smithi090.stdout: "cgroups_split": { 2024-04-24T09:24:41.731 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cgroups_split", 2024-04-24T09:24:41.731 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.731 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.731 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.732 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:41.732 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.732 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.732 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.732 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-24T09:24:41.732 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.732 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.732 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.733 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.733 INFO:teuthology.orchestra.run.smithi090.stdout: "config_checks_enabled": { 2024-04-24T09:24:41.733 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "config_checks_enabled", 2024-04-24T09:24:41.733 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.733 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.733 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.733 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:41.733 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.734 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.734 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.734 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-24T09:24:41.734 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.734 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.734 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.735 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.735 INFO:teuthology.orchestra.run.smithi090.stdout: "config_dashboard": { 2024-04-24T09:24:41.735 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "config_dashboard", 2024-04-24T09:24:41.735 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.735 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.735 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.735 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:41.735 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.736 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.736 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.736 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-24T09:24:41.736 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.736 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.736 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.736 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.736 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_alertmanager": { 2024-04-24T09:24:41.736 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_alertmanager", 2024-04-24T09:24:41.737 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.737 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.737 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.737 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-24T09:24:41.737 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.737 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.737 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.737 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-04-24T09:24:41.737 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.738 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.738 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.738 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.738 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_base": { 2024-04-24T09:24:41.738 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_base", 2024-04-24T09:24:41.738 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.738 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.738 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.738 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-24T09:24:41.739 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.739 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.739 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.739 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Container image name, without the tag", 2024-04-24T09:24:41.739 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.739 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.739 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.739 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.739 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_elasticsearch": { 2024-04-24T09:24:41.739 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_elasticsearch", 2024-04-24T09:24:41.740 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.740 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.740 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.740 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-24T09:24:41.740 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.740 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.740 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.740 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "elasticsearch container image", 2024-04-24T09:24:41.741 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.741 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.741 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.741 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.741 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_grafana": { 2024-04-24T09:24:41.741 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_grafana", 2024-04-24T09:24:41.741 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.741 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.742 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.742 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-24T09:24:41.742 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.742 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.742 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.742 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-04-24T09:24:41.742 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.742 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.743 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.743 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.743 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_haproxy": { 2024-04-24T09:24:41.743 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_haproxy", 2024-04-24T09:24:41.743 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.743 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.743 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.743 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-24T09:24:41.743 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.744 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.744 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.744 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "HAproxy container image", 2024-04-24T09:24:41.744 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.744 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.744 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.744 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.744 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_jaeger_agent": { 2024-04-24T09:24:41.744 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_jaeger_agent", 2024-04-24T09:24:41.744 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.745 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.745 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.745 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-24T09:24:41.745 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.745 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.745 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.745 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Jaeger agent container image", 2024-04-24T09:24:41.745 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.745 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.746 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.746 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.746 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_jaeger_collector": { 2024-04-24T09:24:41.746 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_jaeger_collector", 2024-04-24T09:24:41.746 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.746 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.746 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.746 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-24T09:24:41.746 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.747 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.747 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.747 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Jaeger collector container image", 2024-04-24T09:24:41.747 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.747 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.747 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.747 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.747 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_jaeger_query": { 2024-04-24T09:24:41.747 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_jaeger_query", 2024-04-24T09:24:41.747 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.748 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.748 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.748 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-24T09:24:41.748 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.748 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.748 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.748 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Jaeger query container image", 2024-04-24T09:24:41.748 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.748 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.749 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.749 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.749 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_keepalived": { 2024-04-24T09:24:41.749 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_keepalived", 2024-04-24T09:24:41.749 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.749 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.749 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.749 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-24T09:24:41.749 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.750 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.750 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.750 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Keepalived container image", 2024-04-24T09:24:41.750 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.750 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.750 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.750 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.750 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_loki": { 2024-04-24T09:24:41.751 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_loki", 2024-04-24T09:24:41.751 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.751 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.751 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.751 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-24T09:24:41.751 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.751 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.751 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.752 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Loki container image", 2024-04-24T09:24:41.752 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.752 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.752 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.752 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.752 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_node_exporter": { 2024-04-24T09:24:41.752 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_node_exporter", 2024-04-24T09:24:41.752 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.752 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.753 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.753 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-24T09:24:41.753 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.753 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.753 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.753 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-04-24T09:24:41.753 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.753 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.753 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.754 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.754 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_nvmeof": { 2024-04-24T09:24:41.754 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_nvmeof", 2024-04-24T09:24:41.754 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.754 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.754 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.754 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-24T09:24:41.754 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.754 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.755 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.755 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Nvme-of container image", 2024-04-24T09:24:41.755 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.755 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.755 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.755 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.755 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_prometheus": { 2024-04-24T09:24:41.755 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_prometheus", 2024-04-24T09:24:41.755 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.755 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.756 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.756 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-24T09:24:41.756 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.756 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.756 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.756 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-04-24T09:24:41.756 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.756 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.756 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.757 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.757 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_promtail": { 2024-04-24T09:24:41.757 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_promtail", 2024-04-24T09:24:41.757 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.757 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.757 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.757 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-24T09:24:41.757 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.757 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.758 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.758 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Promtail container image", 2024-04-24T09:24:41.758 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.758 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.758 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.758 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.758 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_samba": { 2024-04-24T09:24:41.758 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_samba", 2024-04-24T09:24:41.758 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.759 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.759 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.759 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-24T09:24:41.759 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.759 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.759 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.759 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Samba/SMB container image", 2024-04-24T09:24:41.759 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.759 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.760 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.760 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.760 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_snmp_gateway": { 2024-04-24T09:24:41.760 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_snmp_gateway", 2024-04-24T09:24:41.760 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.760 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.760 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.760 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-24T09:24:41.760 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.761 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.761 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.761 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SNMP Gateway container image", 2024-04-24T09:24:41.761 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.761 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.761 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.761 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.761 INFO:teuthology.orchestra.run.smithi090.stdout: "container_init": { 2024-04-24T09:24:41.761 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_init", 2024-04-24T09:24:41.761 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.762 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.762 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.762 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:41.762 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.762 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.762 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.762 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Run podman/docker with `--init`", 2024-04-24T09:24:41.762 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.762 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.763 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.763 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.763 INFO:teuthology.orchestra.run.smithi090.stdout: "daemon_cache_timeout": { 2024-04-24T09:24:41.763 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "daemon_cache_timeout", 2024-04-24T09:24:41.763 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:41.763 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.763 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.763 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-04-24T09:24:41.763 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.763 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.764 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.764 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-24T09:24:41.764 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.764 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.764 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.764 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.764 INFO:teuthology.orchestra.run.smithi090.stdout: "default_cephadm_command_timeout": { 2024-04-24T09:24:41.764 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "default_cephadm_command_timeout", 2024-04-24T09:24:41.764 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:41.764 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.765 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.765 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "900", 2024-04-24T09:24:41.765 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.765 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.765 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.765 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-24T09:24:41.765 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.765 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.765 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.766 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.766 INFO:teuthology.orchestra.run.smithi090.stdout: "default_registry": { 2024-04-24T09:24:41.766 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "default_registry", 2024-04-24T09:24:41.766 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.766 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.766 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.766 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io", 2024-04-24T09:24:41.766 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.766 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.766 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.767 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-24T09:24:41.767 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.767 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.767 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.767 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.767 INFO:teuthology.orchestra.run.smithi090.stdout: "device_cache_timeout": { 2024-04-24T09:24:41.767 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "device_cache_timeout", 2024-04-24T09:24:41.767 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:41.767 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.767 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.768 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "1800", 2024-04-24T09:24:41.768 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.768 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.768 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.768 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "seconds to cache device inventory", 2024-04-24T09:24:41.768 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.768 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.769 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.769 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.769 INFO:teuthology.orchestra.run.smithi090.stdout: "device_enhanced_scan": { 2024-04-24T09:24:41.769 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "device_enhanced_scan", 2024-04-24T09:24:41.769 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.769 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.769 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.769 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:41.769 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.770 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.770 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.770 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-24T09:24:41.770 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.770 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.770 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.770 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.770 INFO:teuthology.orchestra.run.smithi090.stdout: "facts_cache_timeout": { 2024-04-24T09:24:41.771 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "facts_cache_timeout", 2024-04-24T09:24:41.771 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:41.771 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.771 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.771 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-04-24T09:24:41.771 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.771 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.771 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.771 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "seconds to cache host facts data", 2024-04-24T09:24:41.772 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.772 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.772 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.772 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.772 INFO:teuthology.orchestra.run.smithi090.stdout: "grafana_dashboards_path": { 2024-04-24T09:24:41.772 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "grafana_dashboards_path", 2024-04-24T09:24:41.772 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.772 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.772 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.773 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-24T09:24:41.773 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.773 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.773 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.773 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-24T09:24:41.773 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.773 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.773 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.773 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.773 INFO:teuthology.orchestra.run.smithi090.stdout: "host_check_interval": { 2024-04-24T09:24:41.774 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "host_check_interval", 2024-04-24T09:24:41.774 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:41.774 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.774 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.774 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-04-24T09:24:41.774 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.774 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.774 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.774 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to perform a host check", 2024-04-24T09:24:41.775 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.775 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.775 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.775 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.775 INFO:teuthology.orchestra.run.smithi090.stdout: "hw_monitoring": { 2024-04-24T09:24:41.775 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "hw_monitoring", 2024-04-24T09:24:41.775 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.775 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.775 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.776 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:41.776 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.776 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.776 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.776 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-24T09:24:41.776 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.776 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.776 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.776 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.777 INFO:teuthology.orchestra.run.smithi090.stdout: "inventory_list_all": { 2024-04-24T09:24:41.777 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "inventory_list_all", 2024-04-24T09:24:41.777 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.777 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.777 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.777 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:41.777 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.777 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.777 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.778 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-24T09:24:41.778 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.778 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.778 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.778 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.778 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:41.778 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:41.778 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.778 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.779 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.779 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:41.779 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.779 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.779 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:41.779 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:41.779 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:41.779 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:41.779 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:41.780 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:41.780 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:41.780 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:41.780 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.780 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.780 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.780 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.780 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.780 INFO:teuthology.orchestra.run.smithi090.stdout: "log_refresh_metadata": { 2024-04-24T09:24:41.781 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_refresh_metadata", 2024-04-24T09:24:41.781 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.781 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.781 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.781 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:41.781 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.781 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.781 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.782 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-24T09:24:41.782 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.782 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.782 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.782 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.782 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:41.782 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:41.782 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.782 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.783 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.783 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:41.783 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.783 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.783 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.783 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-24T09:24:41.783 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.783 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.783 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.783 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.784 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:41.784 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:41.784 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.784 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.784 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.784 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:41.784 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.784 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.785 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:41.785 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:41.785 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:41.785 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:41.785 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:41.785 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:41.785 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:41.785 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:41.785 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.785 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.786 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.786 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.786 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.786 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:41.786 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:41.786 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.786 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.786 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.787 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:41.787 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.787 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.787 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.787 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.787 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.787 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.787 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.787 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.788 INFO:teuthology.orchestra.run.smithi090.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-24T09:24:41.788 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-24T09:24:41.788 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.788 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.788 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.788 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:41.788 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.788 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.788 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.789 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-24T09:24:41.789 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.789 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.789 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.789 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.789 INFO:teuthology.orchestra.run.smithi090.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-24T09:24:41.789 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-24T09:24:41.789 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.790 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.790 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.790 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "*", 2024-04-24T09:24:41.790 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.790 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.790 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.790 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-24T09:24:41.791 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.791 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.791 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.791 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.791 INFO:teuthology.orchestra.run.smithi090.stdout: "max_count_per_host": { 2024-04-24T09:24:41.791 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_count_per_host", 2024-04-24T09:24:41.791 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:41.791 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.792 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.792 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-04-24T09:24:41.792 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.792 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.792 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.792 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "max number of daemons per service per host", 2024-04-24T09:24:41.792 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.792 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.792 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.793 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.793 INFO:teuthology.orchestra.run.smithi090.stdout: "max_osd_draining_count": { 2024-04-24T09:24:41.793 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_osd_draining_count", 2024-04-24T09:24:41.793 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:41.793 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.793 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.793 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-04-24T09:24:41.793 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.793 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.793 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.794 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-24T09:24:41.794 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.794 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.794 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.794 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.794 INFO:teuthology.orchestra.run.smithi090.stdout: "migration_current": { 2024-04-24T09:24:41.794 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "migration_current", 2024-04-24T09:24:41.794 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:41.794 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.795 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.795 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:41.795 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.795 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.795 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.795 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "internal - do not modify", 2024-04-24T09:24:41.795 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.795 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.795 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.795 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.796 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": { 2024-04-24T09:24:41.796 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mode", 2024-04-24T09:24:41.796 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.796 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.796 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.796 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "root", 2024-04-24T09:24:41.796 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.796 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.796 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:41.796 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm-package", 2024-04-24T09:24:41.797 INFO:teuthology.orchestra.run.smithi090.stdout: "root" 2024-04-24T09:24:41.797 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:41.797 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "mode for remote execution of cephadm", 2024-04-24T09:24:41.797 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.797 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.797 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.797 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.797 INFO:teuthology.orchestra.run.smithi090.stdout: "oob_default_addr": { 2024-04-24T09:24:41.797 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "oob_default_addr", 2024-04-24T09:24:41.798 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.798 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.798 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.798 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "169.254.1.1", 2024-04-24T09:24:41.798 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.798 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.798 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.798 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-24T09:24:41.798 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.799 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.799 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.799 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.799 INFO:teuthology.orchestra.run.smithi090.stdout: "prometheus_alerts_path": { 2024-04-24T09:24:41.799 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "prometheus_alerts_path", 2024-04-24T09:24:41.799 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.799 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.799 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.799 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-24T09:24:41.799 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.800 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.800 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.800 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-24T09:24:41.800 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.800 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.800 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.800 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.800 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_insecure": { 2024-04-24T09:24:41.801 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_insecure", 2024-04-24T09:24:41.801 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.801 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.801 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.801 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:41.801 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.801 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.801 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.801 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-24T09:24:41.802 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.802 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.802 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.802 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.802 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_password": { 2024-04-24T09:24:41.802 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_password", 2024-04-24T09:24:41.802 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.802 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.802 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.803 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:41.803 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.803 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.803 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.803 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-24T09:24:41.803 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.803 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.803 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.803 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.804 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_url": { 2024-04-24T09:24:41.804 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_url", 2024-04-24T09:24:41.804 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.804 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.804 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.804 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:41.804 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.804 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.804 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.804 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-24T09:24:41.805 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.805 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.805 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.805 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.805 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_username": { 2024-04-24T09:24:41.805 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_username", 2024-04-24T09:24:41.805 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.805 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.805 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.806 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:41.806 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.806 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.806 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.806 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-24T09:24:41.806 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.806 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.806 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.806 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.807 INFO:teuthology.orchestra.run.smithi090.stdout: "secure_monitoring_stack": { 2024-04-24T09:24:41.807 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "secure_monitoring_stack", 2024-04-24T09:24:41.807 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.807 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.807 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.807 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:41.807 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.807 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.807 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.807 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-24T09:24:41.808 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.808 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.808 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.808 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.808 INFO:teuthology.orchestra.run.smithi090.stdout: "service_discovery_port": { 2024-04-24T09:24:41.808 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "service_discovery_port", 2024-04-24T09:24:41.808 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:41.808 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.809 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.809 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8765", 2024-04-24T09:24:41.809 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.809 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.809 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.809 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "cephadm service discovery port", 2024-04-24T09:24:41.809 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.809 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.810 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.810 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.810 INFO:teuthology.orchestra.run.smithi090.stdout: "ssh_config_file": { 2024-04-24T09:24:41.810 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssh_config_file", 2024-04-24T09:24:41.810 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.810 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.810 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.810 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:41.810 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.811 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.811 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.811 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-24T09:24:41.811 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.811 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.811 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.811 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.811 INFO:teuthology.orchestra.run.smithi090.stdout: "use_agent": { 2024-04-24T09:24:41.811 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "use_agent", 2024-04-24T09:24:41.812 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.812 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.812 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.812 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:41.812 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.812 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.812 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.812 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-24T09:24:41.812 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.813 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.813 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.813 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.813 INFO:teuthology.orchestra.run.smithi090.stdout: "use_repo_digest": { 2024-04-24T09:24:41.813 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "use_repo_digest", 2024-04-24T09:24:41.813 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.813 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.813 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.813 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:41.814 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.814 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.814 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.814 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-24T09:24:41.814 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.814 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.814 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.815 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.815 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_on_failed_host_check": { 2024-04-24T09:24:41.815 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_on_failed_host_check", 2024-04-24T09:24:41.815 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.815 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.815 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.815 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:41.815 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.816 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.816 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.816 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise a health warning if the host check fails", 2024-04-24T09:24:41.816 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.816 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.816 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.816 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.816 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_on_stray_daemons": { 2024-04-24T09:24:41.816 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_on_stray_daemons", 2024-04-24T09:24:41.817 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.817 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.817 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.817 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:41.817 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.817 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.817 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.817 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-24T09:24:41.817 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.817 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.818 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.818 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.818 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_on_stray_hosts": { 2024-04-24T09:24:41.818 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_on_stray_hosts", 2024-04-24T09:24:41.818 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.818 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.818 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.818 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:41.819 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.819 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.819 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.819 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-24T09:24:41.819 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.819 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.819 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.819 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:41.819 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:41.820 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.820 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:41.820 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crash", 2024-04-24T09:24:41.820 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:41.820 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:41.820 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:41.820 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:41.821 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:41.821 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.821 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.821 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.821 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:41.821 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.821 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.821 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:41.822 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:41.822 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:41.822 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:41.822 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:41.822 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:41.822 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:41.823 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:41.823 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.823 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.823 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.823 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.823 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.823 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:41.824 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:41.824 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.824 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.824 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.824 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:41.824 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.824 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.824 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.825 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.825 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.825 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.825 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.825 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.825 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:41.825 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:41.825 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.825 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.826 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.826 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:41.826 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.826 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.826 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:41.826 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:41.826 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:41.826 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:41.827 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:41.827 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:41.827 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:41.827 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:41.827 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.827 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.827 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.827 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.828 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.828 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:41.828 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:41.828 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.828 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.828 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.828 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:41.829 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.829 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.829 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.829 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.829 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.829 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.829 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.830 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.830 INFO:teuthology.orchestra.run.smithi090.stdout: "retain_interval": { 2024-04-24T09:24:41.830 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "retain_interval", 2024-04-24T09:24:41.830 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:41.830 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.830 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.830 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "31536000", 2024-04-24T09:24:41.830 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.831 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.831 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.831 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-24T09:24:41.831 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.831 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.831 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.831 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.832 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_recent_interval": { 2024-04-24T09:24:41.832 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_recent_interval", 2024-04-24T09:24:41.832 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:41.832 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.832 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.832 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "1209600", 2024-04-24T09:24:41.832 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.833 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.833 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.833 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-24T09:24:41.833 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.833 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.833 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.833 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:41.833 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:41.834 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.834 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:41.834 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "dashboard", 2024-04-24T09:24:41.834 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:41.834 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:41.834 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:41.835 INFO:teuthology.orchestra.run.smithi090.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-24T09:24:41.835 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-24T09:24:41.835 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:41.835 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.835 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.835 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-04-24T09:24:41.835 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.835 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.836 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.836 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.836 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.836 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.836 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.836 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.836 INFO:teuthology.orchestra.run.smithi090.stdout: "ALERTMANAGER_API_HOST": { 2024-04-24T09:24:41.836 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-24T09:24:41.836 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.837 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.837 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.837 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:41.837 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.837 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.837 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.837 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.837 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.837 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.838 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.838 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.838 INFO:teuthology.orchestra.run.smithi090.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-24T09:24:41.838 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-24T09:24:41.838 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.838 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.838 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.839 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:41.839 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.839 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.839 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.839 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.839 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.839 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.839 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.840 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.840 INFO:teuthology.orchestra.run.smithi090.stdout: "AUDIT_API_ENABLED": { 2024-04-24T09:24:41.840 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "AUDIT_API_ENABLED", 2024-04-24T09:24:41.840 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.840 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.840 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.840 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:41.840 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.841 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.841 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.841 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.841 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.841 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.841 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.841 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.841 INFO:teuthology.orchestra.run.smithi090.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-24T09:24:41.841 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-24T09:24:41.842 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.842 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.842 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.842 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:41.842 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.842 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.842 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.842 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.843 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.843 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.843 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.843 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.843 INFO:teuthology.orchestra.run.smithi090.stdout: "ENABLE_BROWSABLE_API": { 2024-04-24T09:24:41.843 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-24T09:24:41.843 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.843 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.843 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.844 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:41.844 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.844 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.844 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.844 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.844 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.844 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.844 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.844 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.844 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-24T09:24:41.845 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-24T09:24:41.845 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.845 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.845 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.845 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:41.845 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.845 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.845 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.845 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.846 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.846 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.846 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.846 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.846 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-24T09:24:41.846 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-24T09:24:41.846 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.846 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.846 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.847 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:41.847 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.847 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.847 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.847 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.847 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.847 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.847 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.847 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.848 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-24T09:24:41.848 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-24T09:24:41.848 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.848 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.848 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.848 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:41.848 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.848 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.848 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.849 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.849 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.849 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.849 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.849 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.849 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-24T09:24:41.849 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-24T09:24:41.849 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.849 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.850 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.850 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:41.850 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.850 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.850 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.850 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.850 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.850 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.850 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.851 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.851 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-24T09:24:41.851 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-24T09:24:41.851 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.851 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.851 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.851 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:41.851 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.852 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.852 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.852 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.852 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.852 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.852 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.852 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.852 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-24T09:24:41.852 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-24T09:24:41.853 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.853 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.853 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.853 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:41.853 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.853 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.853 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.853 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.853 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.853 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.854 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.854 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.854 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-24T09:24:41.854 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-24T09:24:41.854 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.854 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.854 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.854 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:41.854 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.855 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.855 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.855 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.855 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.855 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.855 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.855 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.855 INFO:teuthology.orchestra.run.smithi090.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-24T09:24:41.855 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-24T09:24:41.856 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.856 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.856 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.856 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:41.856 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.856 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.856 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.856 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.856 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.856 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.857 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.857 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.857 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_PASSWORD": { 2024-04-24T09:24:41.857 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-24T09:24:41.857 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.857 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.857 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.857 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "admin", 2024-04-24T09:24:41.857 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.857 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.858 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.858 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.858 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.858 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.858 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.858 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.858 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-24T09:24:41.858 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-24T09:24:41.859 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.859 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.859 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.859 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:41.859 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.859 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.859 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.859 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.859 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.860 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.860 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.860 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.860 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_URL": { 2024-04-24T09:24:41.860 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_URL", 2024-04-24T09:24:41.860 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.860 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.860 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.861 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:41.861 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.861 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.861 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.861 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.861 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.861 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.861 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.861 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.862 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_USERNAME": { 2024-04-24T09:24:41.862 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-24T09:24:41.862 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.862 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.862 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.862 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "admin", 2024-04-24T09:24:41.862 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.862 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.862 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.863 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.863 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.863 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.863 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.863 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.863 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-24T09:24:41.863 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-24T09:24:41.863 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.864 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.864 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.864 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:41.864 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.864 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.864 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.864 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.864 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.865 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.865 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.865 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.865 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-24T09:24:41.865 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-24T09:24:41.865 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.865 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.866 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.866 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:41.866 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.866 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.866 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.866 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.866 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.867 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.867 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.867 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.867 INFO:teuthology.orchestra.run.smithi090.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-24T09:24:41.867 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-24T09:24:41.867 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.867 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.867 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.867 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:41.868 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.868 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.868 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.868 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.868 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.868 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.868 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.868 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.869 INFO:teuthology.orchestra.run.smithi090.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-24T09:24:41.869 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-24T09:24:41.869 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.869 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.869 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.869 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:41.869 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.869 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.870 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.870 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.870 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.870 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.870 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.870 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.870 INFO:teuthology.orchestra.run.smithi090.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-24T09:24:41.871 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-24T09:24:41.871 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.871 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.871 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.871 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "[]", 2024-04-24T09:24:41.871 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.871 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.871 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.872 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.872 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.872 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.872 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.872 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.872 INFO:teuthology.orchestra.run.smithi090.stdout: "MULTICLUSTER_CONFIG": { 2024-04-24T09:24:41.872 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-24T09:24:41.872 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.873 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.873 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.873 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "{}", 2024-04-24T09:24:41.873 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.873 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.873 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.873 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.874 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.874 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.874 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.874 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.874 INFO:teuthology.orchestra.run.smithi090.stdout: "PROMETHEUS_API_HOST": { 2024-04-24T09:24:41.874 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-24T09:24:41.874 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.874 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.875 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.875 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:41.875 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.875 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.875 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.875 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.875 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.875 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.876 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.876 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.876 INFO:teuthology.orchestra.run.smithi090.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-24T09:24:41.876 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-24T09:24:41.876 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.876 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.876 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.876 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:41.876 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.877 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.877 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.877 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.877 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.877 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.877 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.877 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.877 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-24T09:24:41.878 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-24T09:24:41.878 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.878 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.878 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.878 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:41.878 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.878 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.879 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.879 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.879 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.879 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.879 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.879 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.879 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-24T09:24:41.879 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-24T09:24:41.880 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.880 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.880 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.880 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:41.880 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.880 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.880 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.880 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.881 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.881 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.881 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.881 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.881 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-24T09:24:41.881 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-24T09:24:41.881 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.881 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.881 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.882 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:41.882 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.882 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.882 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.882 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.882 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.882 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.883 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.883 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.883 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-24T09:24:41.883 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-24T09:24:41.883 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.883 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.883 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.883 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:41.884 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.884 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.884 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.884 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.884 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.884 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.884 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.884 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.884 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-24T09:24:41.884 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-24T09:24:41.885 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.885 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.885 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.885 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:41.885 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.885 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.885 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.885 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.886 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.886 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.886 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.886 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.886 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-24T09:24:41.886 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-24T09:24:41.886 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.886 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.887 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.887 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:41.887 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.887 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.887 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.887 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.887 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.887 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.887 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.888 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.888 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-24T09:24:41.888 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-24T09:24:41.888 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.888 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.888 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.888 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:41.888 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.888 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.889 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.889 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.889 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.889 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.889 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.889 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.889 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_ENABLED": { 2024-04-24T09:24:41.889 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-24T09:24:41.889 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.890 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.890 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.890 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:41.890 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.890 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.890 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.890 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.890 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.891 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.891 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.891 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.891 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-24T09:24:41.891 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-24T09:24:41.891 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.891 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.891 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.891 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-24T09:24:41.892 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.892 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.892 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.892 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.892 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.892 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.892 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.892 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.893 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-24T09:24:41.893 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-24T09:24:41.893 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:41.893 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.893 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.893 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-04-24T09:24:41.893 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.893 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.893 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.893 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.894 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.894 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.894 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.894 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.894 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-24T09:24:41.894 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-24T09:24:41.894 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:41.894 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.894 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.895 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8", 2024-04-24T09:24:41.895 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.895 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.895 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.895 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.895 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.895 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.895 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.895 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.895 INFO:teuthology.orchestra.run.smithi090.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-24T09:24:41.896 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-24T09:24:41.896 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:41.896 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.896 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.896 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "45", 2024-04-24T09:24:41.896 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.896 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.896 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.896 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.896 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.897 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.897 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.897 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.897 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_ACCESS_KEY": { 2024-04-24T09:24:41.897 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-24T09:24:41.897 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.897 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.897 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.897 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:41.898 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.898 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.898 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.898 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.898 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.898 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.898 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.898 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.899 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-24T09:24:41.899 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-24T09:24:41.899 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.899 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.899 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.899 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "admin", 2024-04-24T09:24:41.899 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.899 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.900 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.900 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.900 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.900 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.900 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.900 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.900 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_SECRET_KEY": { 2024-04-24T09:24:41.900 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-24T09:24:41.900 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.901 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.901 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.901 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:41.901 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.901 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.901 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.901 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.901 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.901 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.902 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.902 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.902 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_SSL_VERIFY": { 2024-04-24T09:24:41.902 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-24T09:24:41.902 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.902 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.902 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.902 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:41.902 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.903 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.903 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.903 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.903 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.903 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.903 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.903 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.903 INFO:teuthology.orchestra.run.smithi090.stdout: "UNSAFE_TLS_v1_2": { 2024-04-24T09:24:41.903 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-24T09:24:41.904 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.904 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.904 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.904 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:41.904 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.904 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.904 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.904 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.904 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.905 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.905 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.905 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.905 INFO:teuthology.orchestra.run.smithi090.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-24T09:24:41.905 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-24T09:24:41.905 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:41.905 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.905 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.905 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-04-24T09:24:41.905 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.906 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.906 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.906 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.906 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.906 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.906 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.906 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.907 INFO:teuthology.orchestra.run.smithi090.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-24T09:24:41.907 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-24T09:24:41.907 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:41.907 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.907 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.907 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-04-24T09:24:41.907 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.907 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.907 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.908 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.908 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.908 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.908 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.908 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.908 INFO:teuthology.orchestra.run.smithi090.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-24T09:24:41.908 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-24T09:24:41.909 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:41.909 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.909 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.909 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-04-24T09:24:41.909 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.909 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.909 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.909 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.910 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.910 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.910 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.910 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.910 INFO:teuthology.orchestra.run.smithi090.stdout: "cross_origin_url": { 2024-04-24T09:24:41.910 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cross_origin_url", 2024-04-24T09:24:41.910 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.911 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.911 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.911 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:41.911 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.911 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.911 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.911 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.911 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.911 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.912 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.912 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.912 INFO:teuthology.orchestra.run.smithi090.stdout: "crt_file": { 2024-04-24T09:24:41.912 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crt_file", 2024-04-24T09:24:41.912 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.912 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.912 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.912 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:41.913 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.913 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.913 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.913 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.913 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.913 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.913 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.913 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.914 INFO:teuthology.orchestra.run.smithi090.stdout: "debug": { 2024-04-24T09:24:41.914 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "debug", 2024-04-24T09:24:41.914 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.914 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.914 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.914 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:41.914 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.914 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.915 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.915 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Enable/disable debug options", 2024-04-24T09:24:41.915 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.915 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.915 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.915 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.915 INFO:teuthology.orchestra.run.smithi090.stdout: "jwt_token_ttl": { 2024-04-24T09:24:41.915 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "jwt_token_ttl", 2024-04-24T09:24:41.916 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:41.916 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.916 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.916 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "28800", 2024-04-24T09:24:41.916 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.916 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.916 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.917 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.917 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.917 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.917 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.917 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.917 INFO:teuthology.orchestra.run.smithi090.stdout: "key_file": { 2024-04-24T09:24:41.917 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "key_file", 2024-04-24T09:24:41.917 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.917 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.918 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.918 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:41.918 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.918 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.918 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.918 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.918 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.918 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.919 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.919 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.919 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:41.919 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:41.919 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.919 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.919 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.919 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:41.919 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.920 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.920 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:41.920 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:41.920 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:41.920 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:41.920 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:41.920 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:41.921 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:41.921 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:41.921 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.921 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.921 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.921 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.921 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.921 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:41.921 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:41.922 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.922 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.922 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.922 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:41.922 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.922 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.922 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.922 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.923 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.923 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.923 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.923 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.923 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:41.923 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:41.923 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.923 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.923 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.923 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:41.924 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.924 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.924 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:41.924 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:41.924 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:41.924 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:41.924 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:41.924 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:41.924 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:41.925 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:41.925 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.925 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.925 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.925 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.925 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.925 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:41.925 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:41.925 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.925 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.926 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.926 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:41.926 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.926 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.926 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.926 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.926 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.926 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.927 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.927 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.927 INFO:teuthology.orchestra.run.smithi090.stdout: "motd": { 2024-04-24T09:24:41.927 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "motd", 2024-04-24T09:24:41.927 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.927 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.927 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.927 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:41.928 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.928 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.928 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.928 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "The message of the day", 2024-04-24T09:24:41.928 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.928 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.928 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.928 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.928 INFO:teuthology.orchestra.run.smithi090.stdout: "redirect_resolve_ip_addr": { 2024-04-24T09:24:41.928 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "redirect_resolve_ip_addr", 2024-04-24T09:24:41.929 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.929 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.929 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.929 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:41.929 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.929 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.929 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.929 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.929 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.930 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.930 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.930 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.930 INFO:teuthology.orchestra.run.smithi090.stdout: "server_addr": { 2024-04-24T09:24:41.930 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_addr", 2024-04-24T09:24:41.930 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.930 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.930 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.930 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "::", 2024-04-24T09:24:41.931 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.931 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.931 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.931 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.931 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.931 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.931 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.931 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.931 INFO:teuthology.orchestra.run.smithi090.stdout: "server_port": { 2024-04-24T09:24:41.932 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_port", 2024-04-24T09:24:41.932 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:41.932 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.932 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.932 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8080", 2024-04-24T09:24:41.932 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.932 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.932 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.932 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.933 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.933 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.933 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.933 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.933 INFO:teuthology.orchestra.run.smithi090.stdout: "ssl": { 2024-04-24T09:24:41.933 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssl", 2024-04-24T09:24:41.933 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.933 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.933 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.934 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:41.934 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.934 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.934 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.934 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.934 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.934 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.934 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.935 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.935 INFO:teuthology.orchestra.run.smithi090.stdout: "ssl_server_port": { 2024-04-24T09:24:41.935 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssl_server_port", 2024-04-24T09:24:41.935 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:41.935 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.935 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.935 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8443", 2024-04-24T09:24:41.935 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.935 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.936 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.936 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.936 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.936 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.936 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.936 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.936 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_behaviour": { 2024-04-24T09:24:41.936 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_behaviour", 2024-04-24T09:24:41.936 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.937 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.937 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.937 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "redirect", 2024-04-24T09:24:41.937 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.937 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.937 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:41.937 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:41.937 INFO:teuthology.orchestra.run.smithi090.stdout: "redirect" 2024-04-24T09:24:41.937 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:41.938 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.938 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.938 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.938 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.938 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.938 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_error_status_code": { 2024-04-24T09:24:41.938 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_error_status_code", 2024-04-24T09:24:41.938 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:41.938 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.939 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.939 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "500", 2024-04-24T09:24:41.939 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "400", 2024-04-24T09:24:41.939 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "599", 2024-04-24T09:24:41.939 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.939 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.939 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.939 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.939 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.940 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.940 INFO:teuthology.orchestra.run.smithi090.stdout: "url_prefix": { 2024-04-24T09:24:41.940 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "url_prefix", 2024-04-24T09:24:41.940 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.940 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.940 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.940 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:41.940 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.940 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.941 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.941 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.941 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.941 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.941 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.941 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:41.941 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:41.941 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.942 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:41.942 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "devicehealth", 2024-04-24T09:24:41.942 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:41.942 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:41.942 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:41.942 INFO:teuthology.orchestra.run.smithi090.stdout: "enable_monitoring": { 2024-04-24T09:24:41.942 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enable_monitoring", 2024-04-24T09:24:41.942 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.942 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.943 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.943 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:41.943 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.943 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.943 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.943 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "monitor device health metrics", 2024-04-24T09:24:41.943 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.943 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.944 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.944 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.944 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:41.944 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:41.944 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.944 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.944 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.944 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:41.945 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.945 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.945 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:41.945 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:41.945 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:41.945 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:41.945 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:41.945 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:41.945 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:41.946 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:41.946 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.946 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.946 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.946 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.946 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.946 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:41.946 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:41.947 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.947 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.947 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.947 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:41.947 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.947 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.947 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.947 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.947 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.948 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.948 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.948 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.948 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:41.948 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:41.948 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.948 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.948 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.948 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:41.949 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.949 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.949 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:41.949 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:41.949 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:41.949 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:41.949 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:41.949 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:41.950 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:41.950 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:41.950 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.950 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.950 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.950 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.951 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.951 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:41.951 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:41.951 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.951 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.951 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.951 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:41.951 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.951 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.952 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.952 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.952 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.952 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.952 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.952 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.952 INFO:teuthology.orchestra.run.smithi090.stdout: "mark_out_threshold": { 2024-04-24T09:24:41.952 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mark_out_threshold", 2024-04-24T09:24:41.953 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:41.953 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.953 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.953 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "2419200", 2024-04-24T09:24:41.953 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.953 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.953 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.953 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-24T09:24:41.953 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.954 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.954 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.954 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.954 INFO:teuthology.orchestra.run.smithi090.stdout: "pool_name": { 2024-04-24T09:24:41.954 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pool_name", 2024-04-24T09:24:41.954 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.954 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.954 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.955 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "device_health_metrics", 2024-04-24T09:24:41.955 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.955 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.955 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.955 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-24T09:24:41.955 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.955 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.955 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.955 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.956 INFO:teuthology.orchestra.run.smithi090.stdout: "retention_period": { 2024-04-24T09:24:41.956 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "retention_period", 2024-04-24T09:24:41.956 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:41.956 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.956 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.956 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "15552000", 2024-04-24T09:24:41.956 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.956 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.956 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.957 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how long to retain device health metrics", 2024-04-24T09:24:41.957 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.957 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.957 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.957 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.957 INFO:teuthology.orchestra.run.smithi090.stdout: "scrape_frequency": { 2024-04-24T09:24:41.957 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "scrape_frequency", 2024-04-24T09:24:41.957 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:41.957 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.958 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.958 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "86400", 2024-04-24T09:24:41.958 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.958 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.958 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.958 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-24T09:24:41.958 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.958 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.959 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.959 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.959 INFO:teuthology.orchestra.run.smithi090.stdout: "self_heal": { 2024-04-24T09:24:41.959 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "self_heal", 2024-04-24T09:24:41.959 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.959 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.959 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.959 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:41.959 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.960 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.960 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.960 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-24T09:24:41.960 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.960 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.960 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.960 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.960 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-04-24T09:24:41.961 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-04-24T09:24:41.961 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:41.961 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.961 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.961 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-04-24T09:24:41.961 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.961 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.961 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.961 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to wake up and check device health", 2024-04-24T09:24:41.961 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.962 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.962 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.962 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.962 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_threshold": { 2024-04-24T09:24:41.962 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_threshold", 2024-04-24T09:24:41.962 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:41.962 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.963 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.963 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "7257600", 2024-04-24T09:24:41.963 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.963 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.963 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.963 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-24T09:24:41.963 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.963 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.963 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.963 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:41.964 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:41.964 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.964 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:41.964 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "diskprediction_local", 2024-04-24T09:24:41.964 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:41.964 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:41.964 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:41.964 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:41.964 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:41.965 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.965 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.965 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.965 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:41.965 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.965 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.965 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:41.965 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:41.965 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:41.966 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:41.966 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:41.966 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:41.966 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:41.966 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:41.966 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.966 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.966 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.967 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.967 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.967 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:41.967 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:41.967 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.967 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.967 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.967 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:41.967 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.967 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.968 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.968 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.968 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.968 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.968 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.968 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.968 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:41.968 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:41.968 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.969 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.969 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.969 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:41.969 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.969 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.969 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:41.969 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:41.969 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:41.970 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:41.970 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:41.970 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:41.970 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:41.970 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:41.970 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.970 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.970 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.971 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.971 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.971 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:41.971 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:41.971 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.971 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.971 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.971 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:41.971 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.972 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.972 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.972 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.972 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.972 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.972 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.972 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.972 INFO:teuthology.orchestra.run.smithi090.stdout: "predict_interval": { 2024-04-24T09:24:41.972 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "predict_interval", 2024-04-24T09:24:41.973 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.973 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.973 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.973 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "86400", 2024-04-24T09:24:41.973 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.973 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.973 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.973 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.973 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.974 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.974 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.974 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.974 INFO:teuthology.orchestra.run.smithi090.stdout: "predictor_model": { 2024-04-24T09:24:41.974 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "predictor_model", 2024-04-24T09:24:41.974 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.974 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.974 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.974 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "prophetstor", 2024-04-24T09:24:41.975 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.975 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.975 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.975 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.975 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.975 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.975 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.975 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.976 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-04-24T09:24:41.976 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-04-24T09:24:41.976 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.976 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.976 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.976 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-04-24T09:24:41.976 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.976 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.976 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.977 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.977 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.977 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.977 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.977 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:41.977 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:41.977 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.977 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:41.978 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "influx", 2024-04-24T09:24:41.978 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": false, 2024-04-24T09:24:41.978 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "influxdb python module not found", 2024-04-24T09:24:41.978 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:41.978 INFO:teuthology.orchestra.run.smithi090.stdout: "batch_size": { 2024-04-24T09:24:41.978 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "batch_size", 2024-04-24T09:24:41.978 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:41.978 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.979 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.979 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5000", 2024-04-24T09:24:41.979 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.979 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.979 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.979 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-24T09:24:41.979 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.979 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.980 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.980 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.980 INFO:teuthology.orchestra.run.smithi090.stdout: "database": { 2024-04-24T09:24:41.980 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "database", 2024-04-24T09:24:41.980 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.980 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.980 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.980 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "ceph", 2024-04-24T09:24:41.981 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.981 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.981 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.981 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-24T09:24:41.981 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.981 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.981 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.981 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.981 INFO:teuthology.orchestra.run.smithi090.stdout: "hostname": { 2024-04-24T09:24:41.982 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "hostname", 2024-04-24T09:24:41.982 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.982 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.982 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.982 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:41.982 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.982 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.982 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.983 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "InfluxDB server hostname", 2024-04-24T09:24:41.983 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.983 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.983 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.983 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.983 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-04-24T09:24:41.983 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-04-24T09:24:41.983 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:41.984 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.984 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.984 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "30", 2024-04-24T09:24:41.984 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "5", 2024-04-24T09:24:41.984 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.984 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.984 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-24T09:24:41.984 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.984 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.985 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.985 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.985 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:41.985 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:41.985 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.985 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.985 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.985 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:41.985 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.986 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.986 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:41.986 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:41.986 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:41.986 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:41.986 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:41.986 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:41.986 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:41.987 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:41.987 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.987 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.987 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.987 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.987 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.987 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:41.987 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:41.987 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.988 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.988 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.988 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:41.988 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.988 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.988 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.988 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.989 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.989 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.989 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.989 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.989 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:41.989 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:41.989 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.989 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.989 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.990 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:41.990 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.990 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.990 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:41.990 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:41.990 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:41.990 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:41.991 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:41.991 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:41.991 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:41.991 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:41.991 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.991 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.991 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.991 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.991 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.991 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:41.992 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:41.992 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:41.992 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.992 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:41.992 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:41.992 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.992 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.992 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.993 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:41.993 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.993 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.993 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.993 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.993 INFO:teuthology.orchestra.run.smithi090.stdout: "password": { 2024-04-24T09:24:41.993 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "password", 2024-04-24T09:24:41.993 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.993 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.993 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.994 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:41.994 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.994 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.994 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.994 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "password of InfluxDB server user", 2024-04-24T09:24:41.994 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.994 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.994 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.994 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.995 INFO:teuthology.orchestra.run.smithi090.stdout: "port": { 2024-04-24T09:24:41.995 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "port", 2024-04-24T09:24:41.995 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:41.995 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.995 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.995 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8086", 2024-04-24T09:24:41.995 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.995 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.996 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.996 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "InfluxDB server port", 2024-04-24T09:24:41.996 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.996 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.996 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.996 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.996 INFO:teuthology.orchestra.run.smithi090.stdout: "ssl": { 2024-04-24T09:24:41.996 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssl", 2024-04-24T09:24:41.996 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:41.997 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.997 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.997 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "false", 2024-04-24T09:24:41.997 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:41.997 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:41.997 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.997 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T09:24:41.997 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.998 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.998 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:41.998 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:41.998 INFO:teuthology.orchestra.run.smithi090.stdout: "threads": { 2024-04-24T09:24:41.998 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "threads", 2024-04-24T09:24:41.998 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:41.998 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:41.998 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:41.999 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-04-24T09:24:41.999 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-04-24T09:24:41.999 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "32", 2024-04-24T09:24:41.999 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:41.999 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-24T09:24:41.999 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:41.999 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:41.999 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.000 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.000 INFO:teuthology.orchestra.run.smithi090.stdout: "username": { 2024-04-24T09:24:42.000 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "username", 2024-04-24T09:24:42.000 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.000 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.000 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.000 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.000 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.001 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.001 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.001 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "username of InfluxDB server user", 2024-04-24T09:24:42.001 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.001 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.001 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.001 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.001 INFO:teuthology.orchestra.run.smithi090.stdout: "verify_ssl": { 2024-04-24T09:24:42.001 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "verify_ssl", 2024-04-24T09:24:42.002 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.002 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.002 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.002 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "true", 2024-04-24T09:24:42.002 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.002 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.002 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.002 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T09:24:42.002 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.003 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.003 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.003 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.003 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.003 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.003 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.003 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "insights", 2024-04-24T09:24:42.003 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.004 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.004 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.004 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.004 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.004 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.004 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.004 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.004 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.005 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.005 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.005 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.005 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.005 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.005 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.005 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.005 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.005 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.006 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.006 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.006 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.006 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.006 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.006 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.006 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.006 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.007 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.007 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.007 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.007 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.007 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.007 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.007 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.007 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.007 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.007 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.008 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.008 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.008 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.008 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.008 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.008 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.008 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.008 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.008 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.009 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.009 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.009 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.009 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.009 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.009 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.009 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.009 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.009 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.010 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.010 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.010 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.010 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.010 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.010 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.010 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.010 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.011 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.011 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.011 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.011 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.011 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.011 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.011 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.011 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.011 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.012 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.012 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.012 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.012 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.012 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.012 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "iostat", 2024-04-24T09:24:42.012 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.012 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.012 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.012 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.013 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.013 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.013 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.013 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.013 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.013 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.013 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.013 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.013 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.013 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.014 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.014 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.014 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.014 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.014 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.014 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.014 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.014 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.014 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.015 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.015 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.015 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.015 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.015 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.015 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.015 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.015 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.016 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.016 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.016 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.016 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.016 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.016 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.016 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.017 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.017 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.017 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.017 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.017 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.017 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.017 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.017 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.017 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.018 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.018 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.018 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.018 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.018 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.018 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.018 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.018 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.019 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.019 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.019 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.019 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.019 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.019 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.019 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.020 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.020 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.020 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.020 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.020 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.020 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.020 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.020 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.021 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.021 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.021 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.021 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.021 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.021 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.021 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "k8sevents", 2024-04-24T09:24:42.021 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.022 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.022 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.022 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_event_retention_days": { 2024-04-24T09:24:42.022 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ceph_event_retention_days", 2024-04-24T09:24:42.022 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.022 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.022 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.022 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "7", 2024-04-24T09:24:42.022 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.023 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.023 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.023 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-24T09:24:42.023 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.023 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.023 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.023 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.023 INFO:teuthology.orchestra.run.smithi090.stdout: "config_check_secs": { 2024-04-24T09:24:42.024 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "config_check_secs", 2024-04-24T09:24:42.024 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.024 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.024 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.024 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-04-24T09:24:42.024 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "10", 2024-04-24T09:24:42.024 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.024 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.025 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-24T09:24:42.025 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.025 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.025 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.025 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.025 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.025 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.025 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.025 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.026 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.026 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.026 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.026 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.026 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.026 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.026 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.026 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.026 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.027 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.027 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.027 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.027 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.027 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.027 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.027 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.027 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.028 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.028 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.028 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.028 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.028 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.028 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.028 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.029 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.029 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.029 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.029 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.029 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.029 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.029 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.029 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.030 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.030 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.030 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.030 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.030 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.030 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.030 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.030 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.030 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.031 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.031 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.031 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.031 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.031 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.031 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.031 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.031 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.032 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.032 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.032 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.032 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.032 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.032 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.032 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.032 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.032 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.033 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.033 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.033 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.033 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.033 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.033 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.033 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.033 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.033 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.034 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.034 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.034 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "localpool", 2024-04-24T09:24:42.034 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.034 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.034 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.034 INFO:teuthology.orchestra.run.smithi090.stdout: "failure_domain": { 2024-04-24T09:24:42.034 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "failure_domain", 2024-04-24T09:24:42.035 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.035 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.035 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.035 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "host", 2024-04-24T09:24:42.035 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.035 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.035 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.035 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "failure domain for any created local pool", 2024-04-24T09:24:42.036 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-24T09:24:42.036 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.036 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.036 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.036 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.036 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.036 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.036 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.037 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.037 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.037 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.037 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.037 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.037 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.037 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.037 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.037 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.038 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.038 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.038 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.038 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.038 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.038 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.038 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.038 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.039 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.039 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.039 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.039 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.039 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.039 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.039 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.039 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.039 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.040 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.040 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.040 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.040 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.040 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.040 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.040 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.040 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.040 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.041 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.041 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.041 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.041 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.041 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.041 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.041 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.041 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.042 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.042 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.042 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.042 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.042 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.042 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.042 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.042 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.043 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.043 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.043 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.043 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.043 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.043 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.043 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.043 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.043 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.044 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.044 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.044 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.044 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.044 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.044 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.044 INFO:teuthology.orchestra.run.smithi090.stdout: "min_size": { 2024-04-24T09:24:42.044 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "min_size", 2024-04-24T09:24:42.045 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.045 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.045 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.045 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.045 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.045 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.045 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.045 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "default min_size for any created local pool", 2024-04-24T09:24:42.045 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-24T09:24:42.046 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.046 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.046 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.046 INFO:teuthology.orchestra.run.smithi090.stdout: "num_rep": { 2024-04-24T09:24:42.046 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "num_rep", 2024-04-24T09:24:42.046 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.046 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.046 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.047 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "3", 2024-04-24T09:24:42.047 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.047 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.047 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.047 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "default replica count for any created local pool", 2024-04-24T09:24:42.047 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.047 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.047 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.048 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.048 INFO:teuthology.orchestra.run.smithi090.stdout: "pg_num": { 2024-04-24T09:24:42.048 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pg_num", 2024-04-24T09:24:42.048 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.048 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.048 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.048 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "128", 2024-04-24T09:24:42.048 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.049 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.049 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.049 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "default pg_num for any created local pool", 2024-04-24T09:24:42.049 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.049 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.049 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.049 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.050 INFO:teuthology.orchestra.run.smithi090.stdout: "prefix": { 2024-04-24T09:24:42.050 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "prefix", 2024-04-24T09:24:42.050 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.050 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.050 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.050 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.050 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.050 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.051 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.051 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "name prefix for any created local pool", 2024-04-24T09:24:42.051 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.051 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.051 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.051 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.051 INFO:teuthology.orchestra.run.smithi090.stdout: "subtree": { 2024-04-24T09:24:42.051 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "subtree", 2024-04-24T09:24:42.052 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.052 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.052 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.052 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "rack", 2024-04-24T09:24:42.052 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.052 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.052 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.052 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-24T09:24:42.053 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-24T09:24:42.053 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.053 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.053 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.053 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.053 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.053 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.053 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mds_autoscaler", 2024-04-24T09:24:42.054 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.054 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.054 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.054 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.054 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.054 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.054 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.054 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.054 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.055 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.055 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.055 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.055 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.055 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.055 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.055 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.055 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.056 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.056 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.056 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.056 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.056 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.056 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.056 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.056 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.056 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.056 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.057 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.057 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.057 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.057 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.057 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.057 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.057 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.057 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.057 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.058 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.058 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.058 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.058 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.058 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.058 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.058 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.058 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.058 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.059 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.059 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.059 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.059 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.059 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.059 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.059 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.059 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.060 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.060 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.060 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.060 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.060 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.060 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.060 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.060 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.061 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.061 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.061 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.061 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.061 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.061 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.061 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.061 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.061 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.062 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.062 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.062 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.062 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.062 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.062 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.062 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mirroring", 2024-04-24T09:24:42.062 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.062 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.062 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.063 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.063 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.063 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.063 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.063 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.063 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.063 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.063 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.064 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.064 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.064 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.064 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.064 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.064 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.064 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.064 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.064 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.065 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.065 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.065 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.065 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.065 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.065 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.065 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.065 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.065 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.066 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.066 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.066 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.066 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.066 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.066 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.066 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.066 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.067 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.067 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.067 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.067 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.067 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.067 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.067 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.068 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.068 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.068 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.068 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.068 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.068 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.068 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.068 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.068 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.068 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.069 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.069 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.069 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.069 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.069 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.069 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.069 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.069 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.069 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.070 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.070 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.070 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.070 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.070 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.070 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.070 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.070 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.071 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.071 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.071 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.071 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.071 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.071 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "nfs", 2024-04-24T09:24:42.071 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.071 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.071 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.072 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.072 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.072 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.072 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.072 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.072 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.072 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.072 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.072 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.073 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.073 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.073 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.073 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.073 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.073 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.073 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.073 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.073 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.073 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.074 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.074 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.074 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.074 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.074 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.074 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.074 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.074 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.074 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.075 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.075 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.075 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.075 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.075 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.075 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.075 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.075 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.076 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.076 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.076 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.076 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.076 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.076 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.076 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.076 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.077 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.077 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.077 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.077 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.077 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.077 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.077 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.077 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.078 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.078 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.078 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.078 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.078 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.078 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.078 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.078 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.079 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.079 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.079 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.079 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.079 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.079 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.079 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.079 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.079 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.080 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.080 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.080 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.080 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.080 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "orchestrator", 2024-04-24T09:24:42.080 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.080 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.081 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.081 INFO:teuthology.orchestra.run.smithi090.stdout: "fail_fs": { 2024-04-24T09:24:42.081 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "fail_fs", 2024-04-24T09:24:42.081 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.081 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.081 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.081 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.081 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.082 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.082 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.082 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-24T09:24:42.082 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.082 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.082 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.082 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.082 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.083 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.083 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.083 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.083 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.083 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.083 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.083 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.083 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.083 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.083 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.084 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.084 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.084 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.084 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.084 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.084 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.084 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.085 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.085 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.085 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.085 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.085 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.085 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.085 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.085 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.085 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.086 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.086 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.086 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.086 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.086 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.086 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.086 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.086 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.087 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.087 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.087 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.087 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.087 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.087 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.087 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.087 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.087 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.088 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.088 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.088 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.088 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.088 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.088 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.088 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.088 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.088 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.089 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.089 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.089 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.089 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.089 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.089 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.089 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.089 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.089 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.089 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.090 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.090 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.090 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.090 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.090 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.090 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.090 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.090 INFO:teuthology.orchestra.run.smithi090.stdout: "orchestrator": { 2024-04-24T09:24:42.091 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "orchestrator", 2024-04-24T09:24:42.091 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.091 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.091 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.091 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.091 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.091 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.091 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.091 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm", 2024-04-24T09:24:42.092 INFO:teuthology.orchestra.run.smithi090.stdout: "rook", 2024-04-24T09:24:42.092 INFO:teuthology.orchestra.run.smithi090.stdout: "test_orchestrator" 2024-04-24T09:24:42.092 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.092 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Orchestrator backend", 2024-04-24T09:24:42.092 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.092 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.092 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.092 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.093 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.093 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.093 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.093 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "osd_perf_query", 2024-04-24T09:24:42.093 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.093 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.093 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.093 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.094 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.094 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.094 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.094 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.094 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.094 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.094 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.094 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.094 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.095 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.095 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.095 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.095 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.095 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.095 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.095 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.095 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.096 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.096 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.096 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.096 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.096 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.096 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.096 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.096 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.096 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.096 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.097 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.097 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.097 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.097 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.097 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.097 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.097 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.097 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.098 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.098 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.098 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.098 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.098 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.098 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.098 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.098 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.098 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.099 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.099 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.099 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.099 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.099 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.099 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.099 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.099 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.100 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.100 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.100 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.100 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.100 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.100 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.100 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.100 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.100 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.101 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.101 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.101 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.101 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.101 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.101 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.101 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.101 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.102 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.102 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.102 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.102 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "osd_support", 2024-04-24T09:24:42.102 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.102 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.102 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.102 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.103 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.103 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.103 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.103 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.103 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.103 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.103 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.103 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.103 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.104 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.104 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.104 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.104 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.104 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.104 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.104 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.104 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.104 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.104 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.105 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.105 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.105 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.105 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.105 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.105 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.105 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.105 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.106 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.106 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.106 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.106 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.106 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.106 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.106 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.106 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.106 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.107 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.107 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.107 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.107 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.107 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.107 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.107 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.107 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.107 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.108 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.108 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.108 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.108 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.108 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.108 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.108 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.108 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.108 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.108 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.109 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.109 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.109 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.109 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.109 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.109 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.109 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.109 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.109 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.110 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.110 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.110 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.110 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.110 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.110 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.110 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.110 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.111 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pg_autoscaler", 2024-04-24T09:24:42.111 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.111 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.111 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.111 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.111 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.111 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.111 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.112 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.112 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.112 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.112 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.112 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.112 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.112 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.112 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.113 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.113 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.113 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.113 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.113 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.113 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.113 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.113 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.114 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.114 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.114 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.114 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.114 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.114 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.114 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.114 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.114 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.115 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.115 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.115 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.115 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.115 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.115 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.115 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.115 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.115 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.115 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.116 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.116 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.116 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.116 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.116 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.116 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.116 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.116 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.117 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.117 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.117 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.117 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.117 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.117 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.117 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.117 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.117 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.118 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.118 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.118 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.118 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.118 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.118 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.118 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.118 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.118 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.119 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.119 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.119 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.119 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.119 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.119 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-04-24T09:24:42.119 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-04-24T09:24:42.119 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:42.119 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.120 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.120 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-04-24T09:24:42.120 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.120 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.120 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.120 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.120 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.120 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.120 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.121 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.121 INFO:teuthology.orchestra.run.smithi090.stdout: "threshold": { 2024-04-24T09:24:42.121 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "threshold", 2024-04-24T09:24:42.121 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-04-24T09:24:42.121 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.121 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.121 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "3.0", 2024-04-24T09:24:42.121 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1.0", 2024-04-24T09:24:42.121 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.122 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.122 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "scaling threshold", 2024-04-24T09:24:42.122 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-24T09:24:42.122 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.122 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.122 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.122 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.122 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.123 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.123 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "progress", 2024-04-24T09:24:42.123 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.123 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.123 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.123 INFO:teuthology.orchestra.run.smithi090.stdout: "allow_pg_recovery_event": { 2024-04-24T09:24:42.123 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "allow_pg_recovery_event", 2024-04-24T09:24:42.123 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.123 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.123 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.124 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.124 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.124 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.124 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.124 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-24T09:24:42.124 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.124 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.124 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.124 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.125 INFO:teuthology.orchestra.run.smithi090.stdout: "enabled": { 2024-04-24T09:24:42.125 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enabled", 2024-04-24T09:24:42.125 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.125 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.125 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.125 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:42.125 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.125 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.125 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.126 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.126 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.126 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.126 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.126 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.126 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.126 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.126 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.126 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.127 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.127 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.127 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.127 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.127 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.127 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.127 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.127 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.128 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.128 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.128 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.128 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.128 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.128 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.128 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.128 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.129 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.129 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.129 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.129 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.129 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.129 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.129 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.129 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.130 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.130 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.130 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.130 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.130 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.130 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.130 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.130 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.131 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.131 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.131 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.131 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.131 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.131 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.131 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.131 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.132 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.132 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.132 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.132 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.132 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.132 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.132 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.132 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.133 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.133 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.133 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.133 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.133 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.133 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.133 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.133 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.134 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.134 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.134 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.134 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.134 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.134 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.134 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.135 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.135 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.135 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.135 INFO:teuthology.orchestra.run.smithi090.stdout: "max_completed_events": { 2024-04-24T09:24:42.135 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_completed_events", 2024-04-24T09:24:42.135 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.135 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.135 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.136 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "50", 2024-04-24T09:24:42.136 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.136 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.136 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.136 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "number of past completed events to remember", 2024-04-24T09:24:42.136 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.136 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.136 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.136 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.137 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-04-24T09:24:42.137 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-04-24T09:24:42.137 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:42.137 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.137 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.137 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-04-24T09:24:42.137 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.137 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.137 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.138 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how long the module is going to sleep", 2024-04-24T09:24:42.138 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.138 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.138 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.138 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.138 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.138 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.138 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.138 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "prometheus", 2024-04-24T09:24:42.139 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.139 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.139 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.139 INFO:teuthology.orchestra.run.smithi090.stdout: "cache": { 2024-04-24T09:24:42.139 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cache", 2024-04-24T09:24:42.139 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.139 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.139 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.140 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:42.140 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.140 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.140 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.140 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.140 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.140 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.140 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.140 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.141 INFO:teuthology.orchestra.run.smithi090.stdout: "exclude_perf_counters": { 2024-04-24T09:24:42.141 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "exclude_perf_counters", 2024-04-24T09:24:42.141 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.141 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.141 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.141 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:42.141 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.141 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.141 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.142 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-24T09:24:42.142 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-24T09:24:42.142 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.142 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.142 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.142 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.142 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.142 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.143 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.143 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.143 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.143 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.143 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.143 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.143 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.143 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.143 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.143 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.144 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.144 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.144 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.144 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.144 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.144 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.144 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.144 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.144 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.145 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.145 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.145 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.145 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.145 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.145 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.145 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.145 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.145 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.146 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.146 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.146 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.146 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.146 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.146 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.146 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.146 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.147 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.147 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.147 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.147 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.147 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.147 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.147 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.147 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.147 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.147 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.148 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.148 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.148 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.148 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.148 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.148 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.148 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.148 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.148 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.149 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.149 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.149 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.149 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.149 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.149 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.149 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.149 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.149 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.150 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.150 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.150 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.150 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_stats_pools": { 2024-04-24T09:24:42.150 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_stats_pools", 2024-04-24T09:24:42.150 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.150 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.150 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.150 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.151 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.151 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.151 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.151 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.151 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.151 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.151 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.151 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.152 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-24T09:24:42.152 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-24T09:24:42.152 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.152 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.152 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.152 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "300", 2024-04-24T09:24:42.152 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.152 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.153 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.153 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.153 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.153 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.153 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.153 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.153 INFO:teuthology.orchestra.run.smithi090.stdout: "scrape_interval": { 2024-04-24T09:24:42.153 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "scrape_interval", 2024-04-24T09:24:42.154 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-04-24T09:24:42.154 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.154 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.154 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "15.0", 2024-04-24T09:24:42.154 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.154 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.154 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.154 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.155 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.155 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.155 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.155 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.155 INFO:teuthology.orchestra.run.smithi090.stdout: "server_addr": { 2024-04-24T09:24:42.155 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_addr", 2024-04-24T09:24:42.155 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.155 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.156 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.156 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "::", 2024-04-24T09:24:42.156 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.156 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.156 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.156 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-24T09:24:42.156 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.156 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.156 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.157 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.157 INFO:teuthology.orchestra.run.smithi090.stdout: "server_port": { 2024-04-24T09:24:42.157 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_port", 2024-04-24T09:24:42.157 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.157 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.157 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.157 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "9283", 2024-04-24T09:24:42.157 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.158 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.158 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.158 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-24T09:24:42.158 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.158 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.158 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.158 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.158 INFO:teuthology.orchestra.run.smithi090.stdout: "stale_cache_strategy": { 2024-04-24T09:24:42.158 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "stale_cache_strategy", 2024-04-24T09:24:42.159 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.159 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.159 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.159 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "log", 2024-04-24T09:24:42.159 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.159 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.159 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.159 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.159 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.160 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.160 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.160 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.160 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_behaviour": { 2024-04-24T09:24:42.160 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_behaviour", 2024-04-24T09:24:42.160 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.160 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.160 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.161 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "default", 2024-04-24T09:24:42.161 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.161 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.161 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.161 INFO:teuthology.orchestra.run.smithi090.stdout: "default", 2024-04-24T09:24:42.161 INFO:teuthology.orchestra.run.smithi090.stdout: "error" 2024-04-24T09:24:42.161 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.161 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.161 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.162 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.162 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.162 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.162 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_error_status_code": { 2024-04-24T09:24:42.162 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_error_status_code", 2024-04-24T09:24:42.162 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.162 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.162 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.163 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "500", 2024-04-24T09:24:42.163 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "400", 2024-04-24T09:24:42.163 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "599", 2024-04-24T09:24:42.163 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.163 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.163 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.163 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.163 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.163 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.164 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.164 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.164 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.164 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_support", 2024-04-24T09:24:42.164 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.164 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.164 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.164 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.164 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.165 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.165 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.165 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.165 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.165 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.165 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.165 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.165 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.165 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.166 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.166 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.166 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.166 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.166 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.166 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.166 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.166 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.167 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.167 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.167 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.167 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.167 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.167 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.167 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.167 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.167 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.168 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.168 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.168 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.168 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.168 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.168 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.168 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.168 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.168 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.169 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.169 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.169 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.169 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.169 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.169 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.169 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.169 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.169 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.170 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.170 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.170 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.170 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.170 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.170 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.170 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.170 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.171 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.171 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.171 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.171 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.171 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.171 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.171 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.171 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.171 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.172 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.172 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.172 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.172 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.172 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.172 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.172 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.172 INFO:teuthology.orchestra.run.smithi090.stdout: "max_concurrent_snap_create": { 2024-04-24T09:24:42.172 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_concurrent_snap_create", 2024-04-24T09:24:42.173 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.173 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.173 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.173 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-04-24T09:24:42.173 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.173 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.173 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.173 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.173 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.174 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.174 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.174 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.174 INFO:teuthology.orchestra.run.smithi090.stdout: "mirror_snapshot_schedule": { 2024-04-24T09:24:42.174 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mirror_snapshot_schedule", 2024-04-24T09:24:42.174 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.174 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.174 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.174 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.175 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.175 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.175 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.175 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.175 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.175 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.175 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.175 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.176 INFO:teuthology.orchestra.run.smithi090.stdout: "trash_purge_schedule": { 2024-04-24T09:24:42.176 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "trash_purge_schedule", 2024-04-24T09:24:42.176 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.176 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.176 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.176 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.177 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.177 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.177 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.177 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.177 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.177 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.177 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.177 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.177 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.177 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.178 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.178 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "restful", 2024-04-24T09:24:42.178 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.178 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.178 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.178 INFO:teuthology.orchestra.run.smithi090.stdout: "enable_auth": { 2024-04-24T09:24:42.178 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enable_auth", 2024-04-24T09:24:42.178 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.179 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.179 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.179 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:42.179 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.179 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.179 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.179 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.179 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.179 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.180 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.180 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.180 INFO:teuthology.orchestra.run.smithi090.stdout: "key_file": { 2024-04-24T09:24:42.180 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "key_file", 2024-04-24T09:24:42.180 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.180 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.180 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.180 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.181 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.181 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.181 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.181 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.181 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.181 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.181 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.181 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.181 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.182 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.182 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.182 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.182 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.182 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.182 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.182 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.182 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.183 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.183 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.183 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.183 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.183 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.183 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.183 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.183 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.183 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.184 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.184 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.184 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.184 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.184 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.184 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.184 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.184 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.184 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.185 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.185 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.185 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.185 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.185 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.185 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.185 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.185 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.185 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.186 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.186 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.186 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.186 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.186 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.186 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.186 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.186 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.186 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.187 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.187 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.187 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.187 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.187 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.187 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.187 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.187 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.187 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.187 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.188 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.188 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.188 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.188 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.188 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.188 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.188 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.188 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.188 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.189 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.189 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.189 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.189 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.189 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.189 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.189 INFO:teuthology.orchestra.run.smithi090.stdout: "server_addr": { 2024-04-24T09:24:42.189 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_addr", 2024-04-24T09:24:42.189 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.189 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.190 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.190 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.190 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.190 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.190 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.190 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.190 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.190 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.191 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.191 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.191 INFO:teuthology.orchestra.run.smithi090.stdout: "server_port": { 2024-04-24T09:24:42.191 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_port", 2024-04-24T09:24:42.191 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.191 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.191 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.191 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.191 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.192 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.192 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.192 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.192 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.192 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.192 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.192 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.192 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.192 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.193 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.193 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rgw", 2024-04-24T09:24:42.193 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.193 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.193 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.193 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.193 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.193 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.194 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.194 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.194 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.194 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.194 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.194 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.194 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.194 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.195 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.195 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.195 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.195 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.195 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.195 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.195 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.195 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.195 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.196 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.196 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.196 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.196 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.196 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.196 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.196 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.196 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.197 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.197 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.197 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.197 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.197 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.197 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.197 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.197 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.197 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.198 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.198 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.198 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.198 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.198 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.198 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.198 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.199 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.199 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.199 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.199 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.199 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.199 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.199 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.199 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.199 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.200 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.200 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.200 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.200 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.200 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.200 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.200 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.200 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.201 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.201 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.201 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.201 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.201 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.201 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.201 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.201 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.201 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.202 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.202 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.202 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.202 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rook", 2024-04-24T09:24:42.202 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.202 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.202 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.202 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.202 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.203 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.203 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.203 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.203 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.203 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.203 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.203 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.203 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.203 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.204 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.204 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.204 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.204 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.204 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.204 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.204 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.204 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.205 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.205 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.205 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.205 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.205 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.205 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.205 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.205 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.205 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.206 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.206 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.206 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.206 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.206 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.206 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.206 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.206 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.207 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.207 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.207 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.207 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.207 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.207 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.207 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.207 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.207 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.208 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.208 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.208 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.208 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.208 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.208 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.208 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.208 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.208 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.208 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.209 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.209 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.209 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.209 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.209 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.209 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.209 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.209 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.209 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.210 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.210 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.210 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.210 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.210 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.210 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.210 INFO:teuthology.orchestra.run.smithi090.stdout: "storage_class": { 2024-04-24T09:24:42.210 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "storage_class", 2024-04-24T09:24:42.211 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.211 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.211 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.211 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "local", 2024-04-24T09:24:42.211 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.211 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.211 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.211 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-24T09:24:42.211 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.212 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.212 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.212 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.212 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.212 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.212 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.212 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "selftest", 2024-04-24T09:24:42.212 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.212 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.212 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.213 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.213 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.213 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.213 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.213 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.213 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.213 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.213 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.213 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.213 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.214 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.214 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.214 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.214 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.214 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.214 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.214 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.214 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.215 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.215 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.215 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.215 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.215 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.215 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.215 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.215 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.215 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.215 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.216 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.216 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.216 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.216 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.216 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.216 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.216 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.216 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.216 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.216 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.217 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.217 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.217 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.217 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.217 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.217 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.217 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.217 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.218 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.218 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.218 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.218 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.218 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.218 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.218 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.218 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.218 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.219 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.219 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.219 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.219 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.219 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.219 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.219 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.219 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.219 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.220 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.221 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.221 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.222 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.222 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.222 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.222 INFO:teuthology.orchestra.run.smithi090.stdout: "roption1": { 2024-04-24T09:24:42.222 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "roption1", 2024-04-24T09:24:42.222 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.222 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.222 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.223 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.223 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.223 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.223 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.223 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.223 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.223 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.223 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.223 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.223 INFO:teuthology.orchestra.run.smithi090.stdout: "roption2": { 2024-04-24T09:24:42.224 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "roption2", 2024-04-24T09:24:42.224 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.224 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.224 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.224 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "xyz", 2024-04-24T09:24:42.224 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.224 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.225 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.225 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.225 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.225 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.225 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.225 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.225 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption1": { 2024-04-24T09:24:42.225 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption1", 2024-04-24T09:24:42.226 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.226 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.226 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.226 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.226 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.226 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.226 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.226 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.226 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.227 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.227 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.227 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.227 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption2": { 2024-04-24T09:24:42.227 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption2", 2024-04-24T09:24:42.227 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.227 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.227 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.227 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.228 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.228 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.228 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.228 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.228 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.228 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.228 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.228 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.228 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption3": { 2024-04-24T09:24:42.229 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption3", 2024-04-24T09:24:42.229 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-04-24T09:24:42.229 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.229 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.229 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.229 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.229 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.229 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.229 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.230 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.230 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.230 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.230 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.230 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption4": { 2024-04-24T09:24:42.230 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption4", 2024-04-24T09:24:42.230 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.230 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.231 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.231 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.231 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.231 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.231 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.231 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.231 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.231 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.231 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.232 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.232 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption5": { 2024-04-24T09:24:42.232 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption5", 2024-04-24T09:24:42.232 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.232 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.232 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.232 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.232 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.232 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.233 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.233 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.233 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.233 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.233 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.233 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.233 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption6": { 2024-04-24T09:24:42.233 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption6", 2024-04-24T09:24:42.233 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.234 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.234 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.234 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:42.234 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.234 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.234 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.234 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.234 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.235 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.235 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.235 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.235 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption7": { 2024-04-24T09:24:42.235 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption7", 2024-04-24T09:24:42.235 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.235 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.235 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.236 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.236 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-04-24T09:24:42.236 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "42", 2024-04-24T09:24:42.236 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.236 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.236 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.236 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.236 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.236 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.237 INFO:teuthology.orchestra.run.smithi090.stdout: "testkey": { 2024-04-24T09:24:42.237 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "testkey", 2024-04-24T09:24:42.237 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.237 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.237 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.237 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.237 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.237 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.237 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.238 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.238 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.238 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.238 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.238 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.238 INFO:teuthology.orchestra.run.smithi090.stdout: "testlkey": { 2024-04-24T09:24:42.238 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "testlkey", 2024-04-24T09:24:42.238 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.239 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.239 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.239 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.239 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.239 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.239 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.239 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.239 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.239 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.240 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.240 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.240 INFO:teuthology.orchestra.run.smithi090.stdout: "testnewline": { 2024-04-24T09:24:42.240 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "testnewline", 2024-04-24T09:24:42.240 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.240 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.240 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.240 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.241 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.241 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.241 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.241 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.241 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.241 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.241 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.241 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.241 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.242 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.242 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.242 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snap_schedule", 2024-04-24T09:24:42.242 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.242 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.242 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.242 INFO:teuthology.orchestra.run.smithi090.stdout: "allow_m_granularity": { 2024-04-24T09:24:42.242 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "allow_m_granularity", 2024-04-24T09:24:42.243 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.243 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.243 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.243 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.243 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.243 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.243 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.243 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "allow minute scheduled snapshots", 2024-04-24T09:24:42.243 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.243 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.244 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.244 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.244 INFO:teuthology.orchestra.run.smithi090.stdout: "dump_on_update": { 2024-04-24T09:24:42.244 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "dump_on_update", 2024-04-24T09:24:42.244 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.244 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.244 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.244 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.244 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.245 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.245 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.245 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "dump database to debug log on update", 2024-04-24T09:24:42.245 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.245 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.245 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.245 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.245 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.245 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.246 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.246 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.246 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.246 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.246 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.246 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.246 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.246 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.247 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.247 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.247 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.247 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.247 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.247 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.247 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.247 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.247 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.248 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.248 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.248 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.248 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.248 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.248 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.248 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.248 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.248 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.249 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.249 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.249 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.249 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.249 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.249 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.249 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.249 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.249 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.250 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.250 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.250 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.250 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.250 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.250 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.250 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.250 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.251 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.251 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.251 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.251 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.251 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.251 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.251 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.251 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.251 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.251 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.252 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.252 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.252 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.252 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.252 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.252 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.252 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.252 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.252 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.253 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.253 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.253 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.253 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.253 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.253 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.253 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.253 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.254 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.254 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "stats", 2024-04-24T09:24:42.254 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.254 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.254 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.254 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.254 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.254 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.254 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.255 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.255 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.255 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.255 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.255 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.255 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.255 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.255 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.255 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.256 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.256 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.256 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.256 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.256 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.256 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.256 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.256 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.256 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.256 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.257 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.257 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.257 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.257 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.257 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.257 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.257 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.257 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.257 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.258 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.258 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.258 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.258 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.258 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.258 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.258 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.259 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.259 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.259 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.259 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.259 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.259 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.259 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.259 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.260 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.260 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.260 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.260 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.260 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.260 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.260 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.260 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.261 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.261 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.261 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.261 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.261 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.261 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.261 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.261 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.261 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.261 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.262 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.262 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.262 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.262 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.262 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.262 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.262 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.262 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.262 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "status", 2024-04-24T09:24:42.263 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.263 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.263 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.263 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.263 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.263 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.263 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.263 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.263 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.263 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.264 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.264 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.264 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.264 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.264 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.264 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.264 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.264 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.264 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.265 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.265 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.265 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.265 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.265 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.265 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.265 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.265 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.265 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.266 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.266 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.266 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.266 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.266 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.266 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.266 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.266 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.267 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.267 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.267 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.267 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.267 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.267 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.267 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.267 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.267 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.268 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.268 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.268 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.268 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.268 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.268 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.268 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.268 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.268 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.269 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.269 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.269 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.269 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.269 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.269 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.269 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.269 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.269 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.270 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.270 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.270 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.270 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.270 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.270 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.270 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.270 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.271 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.271 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.271 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.271 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.271 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.271 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "telegraf", 2024-04-24T09:24:42.271 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.271 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.271 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.272 INFO:teuthology.orchestra.run.smithi090.stdout: "address": { 2024-04-24T09:24:42.272 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "address", 2024-04-24T09:24:42.272 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.272 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.272 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.272 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-24T09:24:42.272 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.272 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.272 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.273 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.273 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.273 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.273 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.273 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.273 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-04-24T09:24:42.273 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-04-24T09:24:42.273 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:42.273 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.274 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.274 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "15", 2024-04-24T09:24:42.274 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.274 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.274 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.274 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.274 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.274 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.274 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.275 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.275 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.275 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.275 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.275 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.275 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.275 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.275 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.275 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.276 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.276 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.276 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.276 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.276 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.276 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.276 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.276 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.277 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.277 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.277 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.277 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.277 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.277 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.277 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.277 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.277 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.277 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.278 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.278 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.278 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.278 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.278 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.278 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.278 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.278 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.279 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.279 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.279 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.279 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.279 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.279 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.279 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.279 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.280 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.280 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.280 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.280 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.280 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.280 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.280 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.280 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.281 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.281 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.281 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.281 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.281 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.281 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.281 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.282 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.282 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.282 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.282 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.282 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.282 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.282 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.282 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.283 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.283 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.283 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.283 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.283 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.283 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.283 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.283 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.284 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "telemetry", 2024-04-24T09:24:42.284 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.284 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.284 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.284 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_basic": { 2024-04-24T09:24:42.284 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_basic", 2024-04-24T09:24:42.284 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.284 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.284 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.285 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:42.285 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.285 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.285 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.285 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-24T09:24:42.285 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.285 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.285 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.285 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.286 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_crash": { 2024-04-24T09:24:42.286 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_crash", 2024-04-24T09:24:42.286 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.286 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.286 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.286 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:42.286 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.286 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.286 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.287 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-24T09:24:42.287 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.287 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.287 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.287 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.287 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_device": { 2024-04-24T09:24:42.287 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_device", 2024-04-24T09:24:42.287 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.287 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.288 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.288 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:42.288 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.288 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.288 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.288 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-24T09:24:42.288 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.288 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.288 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.289 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.289 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_ident": { 2024-04-24T09:24:42.289 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_ident", 2024-04-24T09:24:42.289 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.289 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.289 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.289 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.289 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.289 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.290 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.290 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-24T09:24:42.290 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.290 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.290 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.290 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.290 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_perf": { 2024-04-24T09:24:42.290 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_perf", 2024-04-24T09:24:42.290 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.291 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.291 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.291 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.291 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.291 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.291 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.291 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-24T09:24:42.291 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.291 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.292 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.292 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.292 INFO:teuthology.orchestra.run.smithi090.stdout: "contact": { 2024-04-24T09:24:42.292 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "contact", 2024-04-24T09:24:42.292 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.292 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.292 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.292 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.292 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.292 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.293 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.293 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.293 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.293 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.293 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.293 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.293 INFO:teuthology.orchestra.run.smithi090.stdout: "description": { 2024-04-24T09:24:42.293 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "description", 2024-04-24T09:24:42.294 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.294 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.294 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.294 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.294 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.294 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.294 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.294 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.294 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.295 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.295 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.295 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.295 INFO:teuthology.orchestra.run.smithi090.stdout: "device_url": { 2024-04-24T09:24:42.295 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "device_url", 2024-04-24T09:24:42.295 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.295 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.295 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.295 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-24T09:24:42.295 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.296 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.296 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.296 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.296 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.296 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.296 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.296 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.296 INFO:teuthology.orchestra.run.smithi090.stdout: "enabled": { 2024-04-24T09:24:42.296 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enabled", 2024-04-24T09:24:42.297 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.297 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.297 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.297 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.297 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.297 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.297 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.297 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.297 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.298 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.298 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.298 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.298 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-04-24T09:24:42.298 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-04-24T09:24:42.298 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.298 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.298 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.298 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "24", 2024-04-24T09:24:42.299 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "8", 2024-04-24T09:24:42.299 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.299 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.299 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.299 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.299 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.299 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.299 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.299 INFO:teuthology.orchestra.run.smithi090.stdout: "last_opt_revision": { 2024-04-24T09:24:42.300 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "last_opt_revision", 2024-04-24T09:24:42.300 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.300 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.300 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.300 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "1", 2024-04-24T09:24:42.300 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.300 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.300 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.301 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.301 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.301 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.301 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.301 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.301 INFO:teuthology.orchestra.run.smithi090.stdout: "leaderboard": { 2024-04-24T09:24:42.301 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "leaderboard", 2024-04-24T09:24:42.301 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.301 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.302 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.302 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.302 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.302 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.302 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.302 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.302 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.302 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.303 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.303 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.303 INFO:teuthology.orchestra.run.smithi090.stdout: "leaderboard_description": { 2024-04-24T09:24:42.303 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "leaderboard_description", 2024-04-24T09:24:42.303 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.303 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.303 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.303 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.303 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.304 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.304 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.304 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.304 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.304 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.304 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.304 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.304 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.304 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.304 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.305 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.305 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.305 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.305 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.305 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.305 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.305 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.305 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.306 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.306 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.306 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.306 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.306 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.306 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.306 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.306 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.306 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.307 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.307 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.307 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.307 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.307 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.307 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.307 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.307 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.307 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.308 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.308 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.308 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.308 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.308 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.308 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.308 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.308 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.308 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.309 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.309 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.309 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.309 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.309 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.309 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.309 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.309 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.309 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.310 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.310 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.310 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.310 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.310 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.310 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.310 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.310 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.311 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.311 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.311 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.311 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.311 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.311 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.311 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.311 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.311 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.312 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.312 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.312 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.312 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.312 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.312 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.312 INFO:teuthology.orchestra.run.smithi090.stdout: "organization": { 2024-04-24T09:24:42.312 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "organization", 2024-04-24T09:24:42.313 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.313 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.313 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.313 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.313 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.313 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.313 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.313 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.313 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.313 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.314 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.314 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.314 INFO:teuthology.orchestra.run.smithi090.stdout: "proxy": { 2024-04-24T09:24:42.314 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "proxy", 2024-04-24T09:24:42.314 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.314 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.314 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.314 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.315 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.315 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.315 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.315 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.315 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.315 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.315 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.315 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.315 INFO:teuthology.orchestra.run.smithi090.stdout: "url": { 2024-04-24T09:24:42.316 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "url", 2024-04-24T09:24:42.316 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.316 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.316 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.316 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-24T09:24:42.316 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.316 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.316 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.316 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.317 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.317 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.317 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.317 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.317 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.317 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.317 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.317 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "test_orchestrator", 2024-04-24T09:24:42.318 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.318 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.318 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.318 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.318 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.318 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.318 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.318 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.318 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.319 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.319 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.319 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.319 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.319 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.319 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.319 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.319 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.319 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.320 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.320 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.320 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.320 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.320 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.320 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.320 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.320 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.321 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.321 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.321 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.321 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.321 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.321 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.321 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.321 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.321 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.321 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.322 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.322 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.322 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.322 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.322 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.322 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.322 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.322 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.322 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.323 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.323 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.323 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.323 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.323 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.323 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.323 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.323 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.323 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.324 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.324 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.324 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.324 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.324 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.324 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.324 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.324 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.324 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.324 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.325 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.325 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.325 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.325 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.325 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.325 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.325 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.325 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.325 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.325 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.326 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.326 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.326 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "volumes", 2024-04-24T09:24:42.326 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.326 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.326 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.326 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.326 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.327 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.327 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.327 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.327 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.327 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.327 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.327 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.327 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.327 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.327 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.328 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.328 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.328 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.328 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.328 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.328 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.328 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.328 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.328 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.329 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.329 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.329 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.329 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.329 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.329 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.329 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.329 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.330 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.330 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.330 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.330 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.330 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.330 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.330 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.330 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.330 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.331 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.331 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.331 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.331 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.331 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.331 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.331 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.331 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.331 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.332 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.332 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.332 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.332 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.332 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.332 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.332 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.332 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.332 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.333 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.333 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.333 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.333 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.333 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.333 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.333 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.333 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.333 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.333 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.334 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.334 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.334 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.334 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.334 INFO:teuthology.orchestra.run.smithi090.stdout: "max_concurrent_clones": { 2024-04-24T09:24:42.334 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_concurrent_clones", 2024-04-24T09:24:42.334 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.334 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.335 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.335 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "4", 2024-04-24T09:24:42.335 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.335 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.335 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.335 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-24T09:24:42.335 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.335 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.336 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.336 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.336 INFO:teuthology.orchestra.run.smithi090.stdout: "periodic_async_work": { 2024-04-24T09:24:42.336 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "periodic_async_work", 2024-04-24T09:24:42.336 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.336 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.336 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.336 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.336 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.337 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.337 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.337 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Periodically check for async work", 2024-04-24T09:24:42.337 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.337 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.337 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.337 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.337 INFO:teuthology.orchestra.run.smithi090.stdout: "snapshot_clone_delay": { 2024-04-24T09:24:42.338 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snapshot_clone_delay", 2024-04-24T09:24:42.338 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.338 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.338 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.338 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-04-24T09:24:42.338 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.338 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.338 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.339 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-24T09:24:42.339 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.339 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.339 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.339 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.339 INFO:teuthology.orchestra.run.smithi090.stdout: "snapshot_clone_no_wait": { 2024-04-24T09:24:42.339 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snapshot_clone_no_wait", 2024-04-24T09:24:42.339 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.339 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.340 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.340 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:42.340 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.340 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.340 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.340 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-24T09:24:42.340 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.340 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.340 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.341 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.341 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.341 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.341 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.341 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix", 2024-04-24T09:24:42.341 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.341 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.341 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.341 INFO:teuthology.orchestra.run.smithi090.stdout: "discovery_interval": { 2024-04-24T09:24:42.342 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "discovery_interval", 2024-04-24T09:24:42.342 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-04-24T09:24:42.342 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.342 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.342 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "100", 2024-04-24T09:24:42.342 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.342 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.342 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.343 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.343 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.343 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.343 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.343 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.343 INFO:teuthology.orchestra.run.smithi090.stdout: "identifier": { 2024-04-24T09:24:42.343 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "identifier", 2024-04-24T09:24:42.343 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.344 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.344 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.344 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.344 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.344 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.344 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.344 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.344 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.344 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.345 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.345 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.345 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-04-24T09:24:42.345 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-04-24T09:24:42.345 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:42.345 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.345 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.345 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-04-24T09:24:42.346 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.346 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.346 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.346 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.346 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.346 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.346 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.346 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.347 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.347 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.347 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.347 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.347 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.347 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.347 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.347 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.347 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.348 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.348 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.348 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.348 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.348 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.348 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.348 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.348 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.348 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.349 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.349 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.349 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.349 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.349 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.349 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.349 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.349 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.349 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.349 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.350 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.350 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.350 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.350 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.350 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.350 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.350 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.350 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.350 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.351 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.351 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.351 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.351 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.351 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.351 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.351 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.351 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.351 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.352 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.352 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.352 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.352 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.352 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.352 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.352 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.352 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.352 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.352 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.353 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.353 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.353 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.353 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.353 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.353 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.353 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.353 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.353 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.353 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.354 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.354 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.354 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.354 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.354 INFO:teuthology.orchestra.run.smithi090.stdout: "zabbix_host": { 2024-04-24T09:24:42.354 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix_host", 2024-04-24T09:24:42.354 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.354 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.355 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.355 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.355 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.355 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.355 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.355 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.355 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.355 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.356 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.356 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.356 INFO:teuthology.orchestra.run.smithi090.stdout: "zabbix_port": { 2024-04-24T09:24:42.356 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix_port", 2024-04-24T09:24:42.356 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.356 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.356 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.356 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10051", 2024-04-24T09:24:42.356 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.356 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.357 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.357 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.357 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.357 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.357 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.357 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.357 INFO:teuthology.orchestra.run.smithi090.stdout: "zabbix_sender": { 2024-04-24T09:24:42.357 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix_sender", 2024-04-24T09:24:42.357 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.358 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.358 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.358 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-24T09:24:42.358 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.358 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.358 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.358 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.358 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.358 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.358 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.359 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.359 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.359 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.359 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-04-24T09:24:42.359 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.359 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.359 INFO:teuthology.orchestra.run.smithi090.stdout: "gid": 24110, 2024-04-24T09:24:42.359 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "y", 2024-04-24T09:24:42.359 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr_features": 4540701547738038271, 2024-04-24T09:24:42.360 INFO:teuthology.orchestra.run.smithi090.stdout: "available_modules": [ 2024-04-24T09:24:42.360 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.360 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "alerts", 2024-04-24T09:24:42.360 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.360 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.360 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.360 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-04-24T09:24:42.360 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-04-24T09:24:42.360 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:42.360 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.361 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.361 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-04-24T09:24:42.361 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.361 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.361 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.361 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How frequently to reexamine health status", 2024-04-24T09:24:42.361 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.361 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.361 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.361 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.362 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.362 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.362 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.362 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.362 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.362 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.362 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.362 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.362 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.363 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.363 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.363 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.363 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.363 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.363 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.363 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.363 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.363 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.363 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.364 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.364 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.364 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.364 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.364 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.364 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.364 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.364 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.364 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.364 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.365 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.365 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.365 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.365 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.365 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.365 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.365 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.365 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.365 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.366 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.366 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.366 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.366 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.366 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.366 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.366 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.366 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.367 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.367 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.367 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.367 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.367 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.367 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.367 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.367 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.367 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.368 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.368 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.368 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.368 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.368 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.368 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.368 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.368 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.369 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.369 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.369 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.369 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.369 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.369 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.369 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.369 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_destination": { 2024-04-24T09:24:42.369 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_destination", 2024-04-24T09:24:42.370 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.370 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.370 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.370 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.370 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.370 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.370 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.370 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Email address to send alerts to", 2024-04-24T09:24:42.371 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.371 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.371 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.371 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.371 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_from_name": { 2024-04-24T09:24:42.371 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_from_name", 2024-04-24T09:24:42.371 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.371 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.371 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.372 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "Ceph", 2024-04-24T09:24:42.372 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.372 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.372 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.372 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Email From: name", 2024-04-24T09:24:42.372 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.372 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.372 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.373 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.373 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_host": { 2024-04-24T09:24:42.373 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_host", 2024-04-24T09:24:42.373 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.373 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.373 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.373 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.373 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.374 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.374 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.374 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SMTP server", 2024-04-24T09:24:42.374 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.374 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.374 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.374 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.374 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_password": { 2024-04-24T09:24:42.374 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_password", 2024-04-24T09:24:42.375 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.375 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.375 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.375 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.375 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.375 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.375 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.375 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Password to authenticate with", 2024-04-24T09:24:42.375 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.376 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.376 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.376 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.376 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_port": { 2024-04-24T09:24:42.376 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_port", 2024-04-24T09:24:42.376 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.376 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.376 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.376 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "465", 2024-04-24T09:24:42.377 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.377 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.377 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.377 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SMTP port", 2024-04-24T09:24:42.377 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.377 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.377 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.377 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.377 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_sender": { 2024-04-24T09:24:42.378 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_sender", 2024-04-24T09:24:42.378 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.378 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.378 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.378 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.378 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.378 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.378 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.379 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SMTP envelope sender", 2024-04-24T09:24:42.379 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.379 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.379 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.379 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.379 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_ssl": { 2024-04-24T09:24:42.379 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_ssl", 2024-04-24T09:24:42.379 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.379 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.380 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.380 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:42.380 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.380 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.380 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.380 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-24T09:24:42.380 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.380 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.380 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.381 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.381 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_user": { 2024-04-24T09:24:42.381 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_user", 2024-04-24T09:24:42.381 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.381 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.381 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.381 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.381 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.381 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.382 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.382 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "User to authenticate as", 2024-04-24T09:24:42.382 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.382 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.382 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.382 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.382 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.382 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.383 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.383 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "balancer", 2024-04-24T09:24:42.383 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.383 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.383 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.383 INFO:teuthology.orchestra.run.smithi090.stdout: "active": { 2024-04-24T09:24:42.383 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "active", 2024-04-24T09:24:42.383 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.383 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.383 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.384 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:42.384 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.384 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.384 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.384 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "automatically balance PGs across cluster", 2024-04-24T09:24:42.384 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.384 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.384 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.384 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.385 INFO:teuthology.orchestra.run.smithi090.stdout: "begin_time": { 2024-04-24T09:24:42.385 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "begin_time", 2024-04-24T09:24:42.385 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.385 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.385 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.385 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0000", 2024-04-24T09:24:42.385 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.385 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.386 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.386 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "beginning time of day to automatically balance", 2024-04-24T09:24:42.386 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T09:24:42.386 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.386 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.386 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.386 INFO:teuthology.orchestra.run.smithi090.stdout: "begin_weekday": { 2024-04-24T09:24:42.386 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "begin_weekday", 2024-04-24T09:24:42.386 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-04-24T09:24:42.387 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.387 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.387 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-04-24T09:24:42.387 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "0", 2024-04-24T09:24:42.387 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "6", 2024-04-24T09:24:42.387 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.387 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-24T09:24:42.387 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T09:24:42.387 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.388 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.388 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.388 INFO:teuthology.orchestra.run.smithi090.stdout: "crush_compat_max_iterations": { 2024-04-24T09:24:42.388 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crush_compat_max_iterations", 2024-04-24T09:24:42.388 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-04-24T09:24:42.388 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.388 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.388 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "25", 2024-04-24T09:24:42.389 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-04-24T09:24:42.389 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "250", 2024-04-24T09:24:42.389 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.389 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-24T09:24:42.389 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.389 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.389 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.389 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.389 INFO:teuthology.orchestra.run.smithi090.stdout: "crush_compat_metrics": { 2024-04-24T09:24:42.390 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crush_compat_metrics", 2024-04-24T09:24:42.390 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.390 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.390 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.390 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "pgs,objects,bytes", 2024-04-24T09:24:42.390 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.390 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.390 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.391 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-24T09:24:42.391 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-24T09:24:42.391 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.391 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.391 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.391 INFO:teuthology.orchestra.run.smithi090.stdout: "crush_compat_step": { 2024-04-24T09:24:42.391 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crush_compat_step", 2024-04-24T09:24:42.391 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-04-24T09:24:42.391 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.391 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.392 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0.5", 2024-04-24T09:24:42.392 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "0.001", 2024-04-24T09:24:42.392 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "0.999", 2024-04-24T09:24:42.392 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.392 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "aggressiveness of optimization", 2024-04-24T09:24:42.392 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-24T09:24:42.392 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.392 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.392 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.393 INFO:teuthology.orchestra.run.smithi090.stdout: "end_time": { 2024-04-24T09:24:42.393 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "end_time", 2024-04-24T09:24:42.393 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.393 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.393 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.393 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "2359", 2024-04-24T09:24:42.393 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.393 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.393 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.394 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "ending time of day to automatically balance", 2024-04-24T09:24:42.394 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T09:24:42.394 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.394 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.394 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.394 INFO:teuthology.orchestra.run.smithi090.stdout: "end_weekday": { 2024-04-24T09:24:42.394 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "end_weekday", 2024-04-24T09:24:42.394 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-04-24T09:24:42.394 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.395 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.395 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-04-24T09:24:42.395 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "0", 2024-04-24T09:24:42.395 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "6", 2024-04-24T09:24:42.395 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.395 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-24T09:24:42.395 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T09:24:42.395 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.395 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.395 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.396 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.396 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.396 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.396 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.396 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.396 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.396 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.396 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.397 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.397 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.397 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.397 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.397 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.397 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.397 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.397 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.397 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.398 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.398 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.398 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.398 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.398 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.398 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.398 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.398 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.398 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.399 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.399 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.399 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.399 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.399 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.399 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.399 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.399 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.399 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.400 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.400 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.400 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.400 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.400 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.400 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.400 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.400 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.401 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.401 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.401 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.401 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.401 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.401 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.401 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.401 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.401 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.402 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.402 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.402 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.402 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.402 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.402 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.402 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.402 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.402 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.403 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.403 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.403 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.403 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.403 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.403 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.403 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.403 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.403 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.404 INFO:teuthology.orchestra.run.smithi090.stdout: "min_score": { 2024-04-24T09:24:42.404 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "min_score", 2024-04-24T09:24:42.404 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-04-24T09:24:42.404 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.404 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.404 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-04-24T09:24:42.404 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.404 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.404 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.404 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-24T09:24:42.405 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.405 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.405 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.405 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.405 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": { 2024-04-24T09:24:42.405 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mode", 2024-04-24T09:24:42.405 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.406 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.406 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.406 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "upmap", 2024-04-24T09:24:42.406 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.406 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.406 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.406 INFO:teuthology.orchestra.run.smithi090.stdout: "crush-compat", 2024-04-24T09:24:42.407 INFO:teuthology.orchestra.run.smithi090.stdout: "none", 2024-04-24T09:24:42.407 INFO:teuthology.orchestra.run.smithi090.stdout: "read", 2024-04-24T09:24:42.407 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap", 2024-04-24T09:24:42.407 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap-read" 2024-04-24T09:24:42.407 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.407 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Balancer mode", 2024-04-24T09:24:42.407 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.407 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.408 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.408 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.408 INFO:teuthology.orchestra.run.smithi090.stdout: "pool_ids": { 2024-04-24T09:24:42.408 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pool_ids", 2024-04-24T09:24:42.408 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.408 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.408 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.408 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.408 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.409 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.409 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.409 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-24T09:24:42.409 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.409 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.409 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.409 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.409 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-04-24T09:24:42.410 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-04-24T09:24:42.410 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:42.410 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.410 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.410 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-04-24T09:24:42.410 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.410 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.410 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.411 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-24T09:24:42.411 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.411 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.411 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.411 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.411 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap_max_deviation": { 2024-04-24T09:24:42.411 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "upmap_max_deviation", 2024-04-24T09:24:42.411 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.411 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.412 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.412 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-04-24T09:24:42.412 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-04-24T09:24:42.412 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.412 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.412 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-24T09:24:42.412 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-24T09:24:42.412 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.413 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.413 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.413 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap_max_optimizations": { 2024-04-24T09:24:42.413 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "upmap_max_optimizations", 2024-04-24T09:24:42.413 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-04-24T09:24:42.413 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.413 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.413 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-04-24T09:24:42.414 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.414 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.414 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.414 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-24T09:24:42.414 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.414 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.414 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.414 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.414 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.415 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.415 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.415 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cephadm", 2024-04-24T09:24:42.415 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.415 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.415 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.415 INFO:teuthology.orchestra.run.smithi090.stdout: "agent_down_multiplier": { 2024-04-24T09:24:42.415 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "agent_down_multiplier", 2024-04-24T09:24:42.415 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-04-24T09:24:42.416 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.416 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.416 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "3.0", 2024-04-24T09:24:42.416 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.416 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.416 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.416 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-24T09:24:42.416 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.416 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.417 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.417 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.417 INFO:teuthology.orchestra.run.smithi090.stdout: "agent_refresh_rate": { 2024-04-24T09:24:42.417 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "agent_refresh_rate", 2024-04-24T09:24:42.417 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:42.417 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.417 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.417 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "20", 2024-04-24T09:24:42.417 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.418 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.418 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.418 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-24T09:24:42.418 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.418 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.418 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.418 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.418 INFO:teuthology.orchestra.run.smithi090.stdout: "agent_starting_port": { 2024-04-24T09:24:42.419 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "agent_starting_port", 2024-04-24T09:24:42.419 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.419 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.419 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.419 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "4721", 2024-04-24T09:24:42.419 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.419 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.419 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.419 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-24T09:24:42.420 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.420 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.420 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.420 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.420 INFO:teuthology.orchestra.run.smithi090.stdout: "allow_ptrace": { 2024-04-24T09:24:42.420 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "allow_ptrace", 2024-04-24T09:24:42.420 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.420 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.420 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.421 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.421 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.421 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.421 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.421 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-24T09:24:42.421 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-24T09:24:42.421 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.421 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.421 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.421 INFO:teuthology.orchestra.run.smithi090.stdout: "autotune_interval": { 2024-04-24T09:24:42.422 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "autotune_interval", 2024-04-24T09:24:42.422 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:42.422 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.422 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.422 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-04-24T09:24:42.422 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.422 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.422 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.422 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-24T09:24:42.423 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.423 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.423 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.423 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.423 INFO:teuthology.orchestra.run.smithi090.stdout: "autotune_memory_target_ratio": { 2024-04-24T09:24:42.423 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "autotune_memory_target_ratio", 2024-04-24T09:24:42.423 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-04-24T09:24:42.423 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.423 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.424 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0.7", 2024-04-24T09:24:42.424 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.424 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.424 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.424 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-24T09:24:42.424 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.424 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.424 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.425 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.425 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm_log_destination": { 2024-04-24T09:24:42.425 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cephadm_log_destination", 2024-04-24T09:24:42.426 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.426 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.426 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.426 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.426 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.426 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.426 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.426 INFO:teuthology.orchestra.run.smithi090.stdout: "file", 2024-04-24T09:24:42.426 INFO:teuthology.orchestra.run.smithi090.stdout: "file,syslog", 2024-04-24T09:24:42.427 INFO:teuthology.orchestra.run.smithi090.stdout: "syslog" 2024-04-24T09:24:42.427 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.427 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-24T09:24:42.427 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.427 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.427 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.427 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.427 INFO:teuthology.orchestra.run.smithi090.stdout: "cgroups_split": { 2024-04-24T09:24:42.428 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cgroups_split", 2024-04-24T09:24:42.428 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.428 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.428 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.428 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:42.428 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.428 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.428 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.428 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-24T09:24:42.429 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.429 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.429 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.429 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.429 INFO:teuthology.orchestra.run.smithi090.stdout: "config_checks_enabled": { 2024-04-24T09:24:42.429 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "config_checks_enabled", 2024-04-24T09:24:42.429 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.429 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.430 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.430 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.430 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.430 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.430 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.430 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-24T09:24:42.430 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.430 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.431 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.431 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.431 INFO:teuthology.orchestra.run.smithi090.stdout: "config_dashboard": { 2024-04-24T09:24:42.431 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "config_dashboard", 2024-04-24T09:24:42.431 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.431 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.431 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.431 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:42.432 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.432 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.432 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.432 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-24T09:24:42.432 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.432 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.432 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.432 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.432 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_alertmanager": { 2024-04-24T09:24:42.433 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_alertmanager", 2024-04-24T09:24:42.433 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.433 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.433 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.433 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-24T09:24:42.433 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.433 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.433 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.433 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-04-24T09:24:42.434 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.434 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.434 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.434 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.434 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_base": { 2024-04-24T09:24:42.434 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_base", 2024-04-24T09:24:42.434 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.434 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.435 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.435 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-24T09:24:42.435 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.435 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.435 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.435 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Container image name, without the tag", 2024-04-24T09:24:42.435 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.435 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.436 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.436 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.436 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_elasticsearch": { 2024-04-24T09:24:42.436 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_elasticsearch", 2024-04-24T09:24:42.436 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.436 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.436 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.436 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-24T09:24:42.436 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.437 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.437 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.437 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "elasticsearch container image", 2024-04-24T09:24:42.437 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.437 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.437 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.437 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.437 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_grafana": { 2024-04-24T09:24:42.438 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_grafana", 2024-04-24T09:24:42.438 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.438 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.438 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.438 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-24T09:24:42.438 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.438 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.438 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.438 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-04-24T09:24:42.439 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.439 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.439 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.439 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.439 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_haproxy": { 2024-04-24T09:24:42.439 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_haproxy", 2024-04-24T09:24:42.439 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.439 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.440 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.440 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-24T09:24:42.440 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.440 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.440 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.440 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "HAproxy container image", 2024-04-24T09:24:42.440 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.440 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.440 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.441 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.441 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_jaeger_agent": { 2024-04-24T09:24:42.441 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_jaeger_agent", 2024-04-24T09:24:42.441 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.441 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.441 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.441 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-24T09:24:42.441 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.441 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.442 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.442 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Jaeger agent container image", 2024-04-24T09:24:42.442 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.442 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.442 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.442 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.442 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_jaeger_collector": { 2024-04-24T09:24:42.442 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_jaeger_collector", 2024-04-24T09:24:42.442 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.443 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.443 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.443 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-24T09:24:42.443 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.443 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.443 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.443 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Jaeger collector container image", 2024-04-24T09:24:42.443 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.443 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.444 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.444 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.444 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_jaeger_query": { 2024-04-24T09:24:42.444 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_jaeger_query", 2024-04-24T09:24:42.444 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.444 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.444 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.444 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-24T09:24:42.444 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.445 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.445 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.445 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Jaeger query container image", 2024-04-24T09:24:42.445 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.445 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.445 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.445 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.445 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_keepalived": { 2024-04-24T09:24:42.446 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_keepalived", 2024-04-24T09:24:42.446 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.446 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.446 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.446 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-24T09:24:42.446 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.446 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.446 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.446 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Keepalived container image", 2024-04-24T09:24:42.447 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.447 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.447 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.447 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.447 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_loki": { 2024-04-24T09:24:42.447 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_loki", 2024-04-24T09:24:42.447 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.447 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.447 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.448 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-24T09:24:42.448 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.448 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.448 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.448 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Loki container image", 2024-04-24T09:24:42.448 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.448 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.448 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.448 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.449 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_node_exporter": { 2024-04-24T09:24:42.449 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_node_exporter", 2024-04-24T09:24:42.449 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.449 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.449 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.449 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-24T09:24:42.449 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.449 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.449 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.450 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-04-24T09:24:42.450 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.450 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.450 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.450 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.450 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_nvmeof": { 2024-04-24T09:24:42.450 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_nvmeof", 2024-04-24T09:24:42.450 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.451 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.451 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.451 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-24T09:24:42.451 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.451 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.451 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.451 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Nvme-of container image", 2024-04-24T09:24:42.451 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.452 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.452 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.452 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.452 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_prometheus": { 2024-04-24T09:24:42.452 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_prometheus", 2024-04-24T09:24:42.452 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.452 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.452 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.452 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-24T09:24:42.453 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.453 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.453 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.453 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-04-24T09:24:42.453 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.453 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.453 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.453 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.453 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_promtail": { 2024-04-24T09:24:42.453 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_promtail", 2024-04-24T09:24:42.454 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.454 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.454 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.454 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-24T09:24:42.454 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.454 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.454 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.454 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Promtail container image", 2024-04-24T09:24:42.455 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.455 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.455 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.455 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.455 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_samba": { 2024-04-24T09:24:42.455 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_samba", 2024-04-24T09:24:42.455 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.455 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.455 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.456 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-24T09:24:42.456 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.456 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.456 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.456 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Samba/SMB container image", 2024-04-24T09:24:42.456 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.456 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.456 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.456 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.457 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_snmp_gateway": { 2024-04-24T09:24:42.457 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_snmp_gateway", 2024-04-24T09:24:42.457 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.457 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.457 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.457 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-24T09:24:42.457 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.457 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.457 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.457 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SNMP Gateway container image", 2024-04-24T09:24:42.458 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.458 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.458 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.458 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.458 INFO:teuthology.orchestra.run.smithi090.stdout: "container_init": { 2024-04-24T09:24:42.458 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_init", 2024-04-24T09:24:42.458 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.458 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.458 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.458 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:42.459 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.459 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.459 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.459 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Run podman/docker with `--init`", 2024-04-24T09:24:42.459 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.459 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.459 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.459 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.459 INFO:teuthology.orchestra.run.smithi090.stdout: "daemon_cache_timeout": { 2024-04-24T09:24:42.460 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "daemon_cache_timeout", 2024-04-24T09:24:42.460 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:42.460 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.460 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.460 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-04-24T09:24:42.460 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.460 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.460 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.460 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-24T09:24:42.461 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.461 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.461 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.461 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.461 INFO:teuthology.orchestra.run.smithi090.stdout: "default_cephadm_command_timeout": { 2024-04-24T09:24:42.461 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "default_cephadm_command_timeout", 2024-04-24T09:24:42.461 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.461 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.461 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.461 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "900", 2024-04-24T09:24:42.462 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.462 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.462 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.462 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-24T09:24:42.462 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.462 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.462 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.462 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.463 INFO:teuthology.orchestra.run.smithi090.stdout: "default_registry": { 2024-04-24T09:24:42.463 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "default_registry", 2024-04-24T09:24:42.463 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.463 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.463 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.463 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io", 2024-04-24T09:24:42.463 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.463 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.463 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.464 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-24T09:24:42.464 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.464 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.464 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.464 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.464 INFO:teuthology.orchestra.run.smithi090.stdout: "device_cache_timeout": { 2024-04-24T09:24:42.464 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "device_cache_timeout", 2024-04-24T09:24:42.464 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:42.464 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.465 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.465 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "1800", 2024-04-24T09:24:42.465 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.465 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.465 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.465 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "seconds to cache device inventory", 2024-04-24T09:24:42.465 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.465 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.465 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.466 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.466 INFO:teuthology.orchestra.run.smithi090.stdout: "device_enhanced_scan": { 2024-04-24T09:24:42.466 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "device_enhanced_scan", 2024-04-24T09:24:42.466 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.466 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.466 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.466 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.466 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.466 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.467 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.467 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-24T09:24:42.467 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.467 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.467 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.467 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.467 INFO:teuthology.orchestra.run.smithi090.stdout: "facts_cache_timeout": { 2024-04-24T09:24:42.467 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "facts_cache_timeout", 2024-04-24T09:24:42.467 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:42.468 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.468 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.468 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-04-24T09:24:42.468 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.468 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.468 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.468 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "seconds to cache host facts data", 2024-04-24T09:24:42.468 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.468 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.468 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.469 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.469 INFO:teuthology.orchestra.run.smithi090.stdout: "grafana_dashboards_path": { 2024-04-24T09:24:42.469 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "grafana_dashboards_path", 2024-04-24T09:24:42.469 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.469 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.469 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.469 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-24T09:24:42.469 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.469 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.470 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.470 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-24T09:24:42.470 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.470 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.470 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.470 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.470 INFO:teuthology.orchestra.run.smithi090.stdout: "host_check_interval": { 2024-04-24T09:24:42.470 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "host_check_interval", 2024-04-24T09:24:42.471 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:42.471 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.471 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.471 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-04-24T09:24:42.471 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.471 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.471 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.471 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to perform a host check", 2024-04-24T09:24:42.471 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.472 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.472 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.472 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.472 INFO:teuthology.orchestra.run.smithi090.stdout: "hw_monitoring": { 2024-04-24T09:24:42.472 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "hw_monitoring", 2024-04-24T09:24:42.472 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.472 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.472 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.472 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.472 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.473 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.473 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.473 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-24T09:24:42.473 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.473 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.473 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.473 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.473 INFO:teuthology.orchestra.run.smithi090.stdout: "inventory_list_all": { 2024-04-24T09:24:42.473 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "inventory_list_all", 2024-04-24T09:24:42.473 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.474 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.474 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.474 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.474 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.474 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.474 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.474 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-24T09:24:42.474 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.475 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.475 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.475 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.475 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.475 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.475 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.475 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.475 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.475 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.475 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.476 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.476 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.476 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.476 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.476 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.476 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.476 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.476 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.476 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.476 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.477 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.477 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.477 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.477 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.477 INFO:teuthology.orchestra.run.smithi090.stdout: "log_refresh_metadata": { 2024-04-24T09:24:42.477 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_refresh_metadata", 2024-04-24T09:24:42.477 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.477 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.477 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.478 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.478 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.478 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.478 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.478 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-24T09:24:42.478 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.478 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.479 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.479 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.479 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.479 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.479 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.479 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.479 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.479 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:42.480 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.480 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.480 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.480 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-24T09:24:42.480 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.480 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.480 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.480 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.480 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.481 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.481 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.481 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.481 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.481 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.481 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.481 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.481 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.481 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.482 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.482 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.482 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.482 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.482 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.482 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.482 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.482 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.483 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.483 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.483 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.483 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.483 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.483 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.483 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.483 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.483 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.484 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.484 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.484 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.484 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.484 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.484 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.484 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.484 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.484 INFO:teuthology.orchestra.run.smithi090.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-24T09:24:42.485 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-24T09:24:42.485 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.485 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.485 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.485 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.485 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.485 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.485 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.485 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-24T09:24:42.486 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.486 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.486 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.486 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.486 INFO:teuthology.orchestra.run.smithi090.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-24T09:24:42.486 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-24T09:24:42.486 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.486 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.487 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.487 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "*", 2024-04-24T09:24:42.487 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.487 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.487 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.487 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-24T09:24:42.487 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.487 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.487 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.488 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.488 INFO:teuthology.orchestra.run.smithi090.stdout: "max_count_per_host": { 2024-04-24T09:24:42.488 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_count_per_host", 2024-04-24T09:24:42.488 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.488 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.488 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.488 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-04-24T09:24:42.488 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.488 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.489 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.489 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "max number of daemons per service per host", 2024-04-24T09:24:42.489 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.489 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.489 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.489 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.489 INFO:teuthology.orchestra.run.smithi090.stdout: "max_osd_draining_count": { 2024-04-24T09:24:42.489 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_osd_draining_count", 2024-04-24T09:24:42.489 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.490 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.490 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.490 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-04-24T09:24:42.490 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.490 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.490 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.490 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-24T09:24:42.490 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.491 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.491 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.491 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.491 INFO:teuthology.orchestra.run.smithi090.stdout: "migration_current": { 2024-04-24T09:24:42.491 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "migration_current", 2024-04-24T09:24:42.491 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.491 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.491 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.491 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.492 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.492 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.492 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.492 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "internal - do not modify", 2024-04-24T09:24:42.492 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.492 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.492 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.492 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.492 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": { 2024-04-24T09:24:42.493 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mode", 2024-04-24T09:24:42.493 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.493 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.493 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.493 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "root", 2024-04-24T09:24:42.493 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.493 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.493 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.494 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm-package", 2024-04-24T09:24:42.494 INFO:teuthology.orchestra.run.smithi090.stdout: "root" 2024-04-24T09:24:42.494 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.494 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "mode for remote execution of cephadm", 2024-04-24T09:24:42.494 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.494 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.494 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.494 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.495 INFO:teuthology.orchestra.run.smithi090.stdout: "oob_default_addr": { 2024-04-24T09:24:42.495 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "oob_default_addr", 2024-04-24T09:24:42.495 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.495 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.495 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.495 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "169.254.1.1", 2024-04-24T09:24:42.495 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.495 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.496 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.496 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-24T09:24:42.496 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.496 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.496 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.496 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.496 INFO:teuthology.orchestra.run.smithi090.stdout: "prometheus_alerts_path": { 2024-04-24T09:24:42.496 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "prometheus_alerts_path", 2024-04-24T09:24:42.497 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.497 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.497 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.497 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-24T09:24:42.497 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.497 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.497 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.497 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-24T09:24:42.497 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.498 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.498 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.498 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.498 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_insecure": { 2024-04-24T09:24:42.498 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_insecure", 2024-04-24T09:24:42.498 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.498 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.498 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.499 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.499 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.499 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.499 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.499 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-24T09:24:42.499 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.499 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.500 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.500 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.500 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_password": { 2024-04-24T09:24:42.500 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_password", 2024-04-24T09:24:42.500 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.500 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.500 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.500 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.500 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.501 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.501 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.501 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-24T09:24:42.501 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.501 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.501 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.501 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.501 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_url": { 2024-04-24T09:24:42.502 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_url", 2024-04-24T09:24:42.502 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.502 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.502 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.502 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.502 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.502 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.502 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.503 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-24T09:24:42.503 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.503 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.503 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.503 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.503 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_username": { 2024-04-24T09:24:42.503 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_username", 2024-04-24T09:24:42.503 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.503 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.504 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.504 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.504 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.504 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.504 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.504 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-24T09:24:42.504 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.504 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.504 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.505 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.505 INFO:teuthology.orchestra.run.smithi090.stdout: "secure_monitoring_stack": { 2024-04-24T09:24:42.505 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "secure_monitoring_stack", 2024-04-24T09:24:42.505 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.505 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.505 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.505 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.505 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.505 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.506 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.506 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-24T09:24:42.506 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.506 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.506 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.506 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.506 INFO:teuthology.orchestra.run.smithi090.stdout: "service_discovery_port": { 2024-04-24T09:24:42.507 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "service_discovery_port", 2024-04-24T09:24:42.507 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.507 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.507 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.507 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8765", 2024-04-24T09:24:42.507 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.507 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.507 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.507 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "cephadm service discovery port", 2024-04-24T09:24:42.508 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.508 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.508 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.508 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.508 INFO:teuthology.orchestra.run.smithi090.stdout: "ssh_config_file": { 2024-04-24T09:24:42.508 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssh_config_file", 2024-04-24T09:24:42.508 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.508 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.508 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.509 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.509 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.509 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.509 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.509 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-24T09:24:42.509 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.509 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.509 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.509 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.510 INFO:teuthology.orchestra.run.smithi090.stdout: "use_agent": { 2024-04-24T09:24:42.510 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "use_agent", 2024-04-24T09:24:42.510 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.510 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.510 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.510 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.510 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.510 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.511 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.511 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-24T09:24:42.511 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.511 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.511 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.511 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.511 INFO:teuthology.orchestra.run.smithi090.stdout: "use_repo_digest": { 2024-04-24T09:24:42.511 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "use_repo_digest", 2024-04-24T09:24:42.512 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.512 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.512 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.512 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:42.512 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.512 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.512 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.512 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-24T09:24:42.512 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.513 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.513 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.513 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.513 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_on_failed_host_check": { 2024-04-24T09:24:42.513 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_on_failed_host_check", 2024-04-24T09:24:42.513 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.513 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.513 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.513 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:42.514 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.514 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.514 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.514 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise a health warning if the host check fails", 2024-04-24T09:24:42.514 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.514 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.514 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.514 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.515 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_on_stray_daemons": { 2024-04-24T09:24:42.515 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_on_stray_daemons", 2024-04-24T09:24:42.515 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.515 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.515 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.516 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:24:41 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/2911974189' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:24:42.517 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:24:41 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/4109344076' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-24T09:24:42.517 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:24:41 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/2911974189' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:24:42.517 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:24:41 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/4109344076' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-24T09:24:42.520 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:42.520 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.520 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.520 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.520 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-24T09:24:42.520 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.520 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.521 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.521 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.521 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_on_stray_hosts": { 2024-04-24T09:24:42.521 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_on_stray_hosts", 2024-04-24T09:24:42.521 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.521 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.521 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.521 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:42.521 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.521 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.522 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.522 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-24T09:24:42.522 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.522 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.522 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.522 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.522 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.522 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.523 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.523 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crash", 2024-04-24T09:24:42.523 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.523 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.523 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.523 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.523 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.523 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.523 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.524 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.524 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.524 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.524 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.524 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.524 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.524 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.524 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.524 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.524 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.525 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.525 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.525 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.525 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.525 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.525 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.525 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.525 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.525 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.526 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.526 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.526 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.526 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.526 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.526 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.526 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.526 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.527 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.527 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.527 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.527 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.527 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.527 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.527 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.527 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.527 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.527 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.528 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.528 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.528 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.528 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.528 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.528 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.528 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.528 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.528 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.528 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.529 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.529 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.529 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.529 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.529 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.529 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.529 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.529 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.529 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.530 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.530 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.530 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.530 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.530 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.530 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.530 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.530 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.531 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.531 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.531 INFO:teuthology.orchestra.run.smithi090.stdout: "retain_interval": { 2024-04-24T09:24:42.531 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "retain_interval", 2024-04-24T09:24:42.531 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:42.531 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.531 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.531 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "31536000", 2024-04-24T09:24:42.531 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.532 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.532 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.532 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-24T09:24:42.532 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.532 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.532 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.532 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.532 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_recent_interval": { 2024-04-24T09:24:42.532 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_recent_interval", 2024-04-24T09:24:42.533 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:42.533 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.533 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.533 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "1209600", 2024-04-24T09:24:42.533 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.533 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.533 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.533 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-24T09:24:42.534 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.534 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.534 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.534 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.534 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.534 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.534 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.534 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "dashboard", 2024-04-24T09:24:42.534 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.535 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.535 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.535 INFO:teuthology.orchestra.run.smithi090.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-24T09:24:42.535 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-24T09:24:42.535 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.535 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.535 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.535 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-04-24T09:24:42.536 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.536 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.536 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.536 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.536 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.536 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.536 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.536 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.536 INFO:teuthology.orchestra.run.smithi090.stdout: "ALERTMANAGER_API_HOST": { 2024-04-24T09:24:42.536 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-24T09:24:42.537 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.537 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.537 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.537 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.537 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.537 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.537 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.537 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.537 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.538 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.538 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.538 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.538 INFO:teuthology.orchestra.run.smithi090.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-24T09:24:42.538 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-24T09:24:42.538 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.538 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.538 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.538 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:42.538 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.539 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.539 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.539 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.539 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.539 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.539 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.539 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.539 INFO:teuthology.orchestra.run.smithi090.stdout: "AUDIT_API_ENABLED": { 2024-04-24T09:24:42.539 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "AUDIT_API_ENABLED", 2024-04-24T09:24:42.540 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.540 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.540 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.540 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.540 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.540 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.540 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.540 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.541 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.541 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.541 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.541 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.541 INFO:teuthology.orchestra.run.smithi090.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-24T09:24:42.541 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-24T09:24:42.541 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.541 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.541 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.542 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:42.542 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.542 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.542 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.542 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.542 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.542 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.542 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.542 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.543 INFO:teuthology.orchestra.run.smithi090.stdout: "ENABLE_BROWSABLE_API": { 2024-04-24T09:24:42.543 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-24T09:24:42.543 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.543 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.543 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.543 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:42.543 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.543 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.543 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.544 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.544 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.544 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.544 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.544 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.544 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-24T09:24:42.544 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-24T09:24:42.544 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.544 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.544 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.545 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:42.545 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.545 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.545 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.545 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.545 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.545 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.545 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.546 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.546 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-24T09:24:42.546 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-24T09:24:42.546 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.546 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.546 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.546 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:42.546 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.547 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.547 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.547 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.547 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.547 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.547 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.547 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.547 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-24T09:24:42.547 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-24T09:24:42.548 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.548 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.548 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.548 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:42.548 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.548 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.548 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.548 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.549 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.549 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.549 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.549 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.549 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-24T09:24:42.549 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-24T09:24:42.549 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.549 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.549 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.549 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:42.550 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.550 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.550 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.550 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.550 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.550 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.550 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.551 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.551 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-24T09:24:42.551 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-24T09:24:42.551 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.551 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.551 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.551 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:42.551 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.552 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.552 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.552 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.552 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.552 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.552 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.552 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.552 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-24T09:24:42.552 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-24T09:24:42.553 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.553 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.553 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.553 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:42.553 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.553 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.553 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.553 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.554 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.554 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.554 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.554 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.554 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-24T09:24:42.554 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-24T09:24:42.554 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.554 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.554 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.555 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:42.555 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.555 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.555 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.555 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.555 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.555 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.555 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.555 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.556 INFO:teuthology.orchestra.run.smithi090.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-24T09:24:42.556 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-24T09:24:42.556 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.556 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.556 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.556 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.556 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.556 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.556 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.557 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.557 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.557 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.557 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.557 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.557 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_PASSWORD": { 2024-04-24T09:24:42.557 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-24T09:24:42.557 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.557 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.557 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.558 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "admin", 2024-04-24T09:24:42.558 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.558 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.558 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.558 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.558 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.558 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.558 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.559 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.559 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-24T09:24:42.559 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-24T09:24:42.559 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.559 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.559 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.559 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:42.559 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.559 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.560 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.560 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.560 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.560 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.560 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.560 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.560 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_URL": { 2024-04-24T09:24:42.560 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_URL", 2024-04-24T09:24:42.560 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.561 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.561 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.561 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.561 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.561 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.561 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.561 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.561 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.561 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.561 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.562 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.562 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_USERNAME": { 2024-04-24T09:24:42.562 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-24T09:24:42.562 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.562 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.562 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.562 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "admin", 2024-04-24T09:24:42.562 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.563 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.563 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.563 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.563 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.563 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.563 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.563 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.563 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-24T09:24:42.563 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-24T09:24:42.564 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.564 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.564 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.564 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.564 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.564 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.564 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.564 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.564 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.564 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.565 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.565 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.565 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-24T09:24:42.565 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-24T09:24:42.565 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.565 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.565 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.565 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.565 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.566 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.566 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.566 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.566 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.566 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.566 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.566 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.566 INFO:teuthology.orchestra.run.smithi090.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-24T09:24:42.566 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-24T09:24:42.567 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.567 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.567 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.567 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:42.567 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.567 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.567 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.567 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.567 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.568 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.568 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.568 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.568 INFO:teuthology.orchestra.run.smithi090.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-24T09:24:42.568 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-24T09:24:42.568 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.568 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.568 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.568 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.568 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.569 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.569 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.569 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.569 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.569 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.569 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.569 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.569 INFO:teuthology.orchestra.run.smithi090.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-24T09:24:42.569 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-24T09:24:42.570 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.570 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.570 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.570 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "[]", 2024-04-24T09:24:42.570 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.570 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.570 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.570 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.570 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.571 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.571 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.571 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.571 INFO:teuthology.orchestra.run.smithi090.stdout: "MULTICLUSTER_CONFIG": { 2024-04-24T09:24:42.571 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-24T09:24:42.571 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.571 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.571 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.571 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "{}", 2024-04-24T09:24:42.571 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.572 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.572 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.572 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.572 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.572 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.572 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.572 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.572 INFO:teuthology.orchestra.run.smithi090.stdout: "PROMETHEUS_API_HOST": { 2024-04-24T09:24:42.573 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-24T09:24:42.573 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.573 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.573 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.573 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.573 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.573 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.573 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.573 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.574 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.574 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.574 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.574 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.574 INFO:teuthology.orchestra.run.smithi090.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-24T09:24:42.574 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-24T09:24:42.574 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.574 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.574 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.575 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:42.575 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.575 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.575 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.575 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.575 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.575 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.575 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.575 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.576 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-24T09:24:42.576 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-24T09:24:42.576 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.576 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.576 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.576 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.576 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.576 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.576 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.577 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.577 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.577 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.577 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.577 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.577 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-24T09:24:42.577 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-24T09:24:42.577 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.577 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.577 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.578 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.578 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.578 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.578 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.578 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.578 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.578 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.578 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.579 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.579 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-24T09:24:42.579 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-24T09:24:42.579 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.579 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.579 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.579 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:42.579 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.579 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.580 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.580 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.580 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.580 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.580 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.580 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.580 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-24T09:24:42.580 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-24T09:24:42.580 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.581 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.581 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.581 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:42.581 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.581 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.581 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.581 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.581 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.581 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.582 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.582 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.582 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-24T09:24:42.582 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-24T09:24:42.582 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.582 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.582 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.582 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.583 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.583 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.583 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.583 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.583 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.583 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.583 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.583 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.583 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-24T09:24:42.584 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-24T09:24:42.584 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.584 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.584 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.584 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.584 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.584 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.584 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.584 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.585 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.585 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.585 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.585 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.585 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-24T09:24:42.585 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-24T09:24:42.585 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.585 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.585 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.586 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.586 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.586 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.586 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.586 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.586 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.586 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.586 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.587 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.587 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_ENABLED": { 2024-04-24T09:24:42.587 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-24T09:24:42.587 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.587 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.587 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.587 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:42.587 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.587 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.588 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.588 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.588 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.588 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.588 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.588 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.588 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-24T09:24:42.588 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-24T09:24:42.588 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.589 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.589 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.589 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-24T09:24:42.589 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.589 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.589 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.589 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.589 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.589 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.590 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.590 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.590 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-24T09:24:42.590 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-24T09:24:42.590 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.590 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.590 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.590 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-04-24T09:24:42.591 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.591 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.591 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.591 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.591 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.591 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.591 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.591 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.591 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-24T09:24:42.592 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-24T09:24:42.592 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.592 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.592 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.592 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8", 2024-04-24T09:24:42.592 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.592 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.592 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.592 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.593 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.593 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.593 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.593 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.593 INFO:teuthology.orchestra.run.smithi090.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-24T09:24:42.593 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-24T09:24:42.593 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.593 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.593 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.593 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "45", 2024-04-24T09:24:42.594 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.594 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.594 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.594 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.594 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.594 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.594 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.594 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.594 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_ACCESS_KEY": { 2024-04-24T09:24:42.595 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-24T09:24:42.595 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.595 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.595 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.595 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.595 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.595 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.595 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.595 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.595 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.596 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.596 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.596 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.596 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-24T09:24:42.596 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-24T09:24:42.596 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.596 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.596 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.596 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "admin", 2024-04-24T09:24:42.597 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.597 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.597 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.597 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.597 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.597 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.597 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.597 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.597 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_SECRET_KEY": { 2024-04-24T09:24:42.598 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-24T09:24:42.598 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.598 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.598 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.598 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.598 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.598 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.598 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.598 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.599 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.599 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.599 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.599 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.599 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_SSL_VERIFY": { 2024-04-24T09:24:42.599 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-24T09:24:42.599 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.599 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.599 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.600 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:42.600 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.600 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.600 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.600 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.600 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.600 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.600 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.600 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.601 INFO:teuthology.orchestra.run.smithi090.stdout: "UNSAFE_TLS_v1_2": { 2024-04-24T09:24:42.601 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-24T09:24:42.601 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.601 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.601 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.601 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.601 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.601 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.601 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.602 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.602 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.602 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.602 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.602 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.602 INFO:teuthology.orchestra.run.smithi090.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-24T09:24:42.602 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-24T09:24:42.602 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.602 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.603 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.603 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-04-24T09:24:42.603 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.603 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.603 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.603 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.603 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.603 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.603 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.603 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.604 INFO:teuthology.orchestra.run.smithi090.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-24T09:24:42.604 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-24T09:24:42.604 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.604 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.604 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.604 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-04-24T09:24:42.604 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.604 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.604 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.605 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.605 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.605 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.605 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.605 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.605 INFO:teuthology.orchestra.run.smithi090.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-24T09:24:42.605 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-24T09:24:42.605 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.605 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.605 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.606 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-04-24T09:24:42.606 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.606 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.606 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.606 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.606 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.606 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.606 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.606 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.606 INFO:teuthology.orchestra.run.smithi090.stdout: "cross_origin_url": { 2024-04-24T09:24:42.607 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cross_origin_url", 2024-04-24T09:24:42.607 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.607 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.607 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.607 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.607 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.607 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.607 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.607 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.608 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.608 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.608 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.608 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.608 INFO:teuthology.orchestra.run.smithi090.stdout: "crt_file": { 2024-04-24T09:24:42.608 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crt_file", 2024-04-24T09:24:42.608 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.608 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.608 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.608 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.609 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.609 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.609 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.609 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.609 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.609 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.609 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.609 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.610 INFO:teuthology.orchestra.run.smithi090.stdout: "debug": { 2024-04-24T09:24:42.610 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "debug", 2024-04-24T09:24:42.610 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.610 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.610 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.610 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.610 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.610 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.611 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.611 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Enable/disable debug options", 2024-04-24T09:24:42.611 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.611 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.611 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.611 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.611 INFO:teuthology.orchestra.run.smithi090.stdout: "jwt_token_ttl": { 2024-04-24T09:24:42.611 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "jwt_token_ttl", 2024-04-24T09:24:42.611 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.612 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.612 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.612 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "28800", 2024-04-24T09:24:42.612 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.612 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.612 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.612 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.612 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.612 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.613 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.613 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.613 INFO:teuthology.orchestra.run.smithi090.stdout: "key_file": { 2024-04-24T09:24:42.613 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "key_file", 2024-04-24T09:24:42.613 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.613 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.613 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.613 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.613 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.614 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.614 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.614 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.614 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.614 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.614 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.614 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.614 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.614 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.615 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.615 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.615 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.615 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.615 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.615 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.615 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.615 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.615 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.616 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.616 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.616 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.616 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.616 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.616 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.616 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.616 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.616 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.617 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.617 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.617 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.617 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.617 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.617 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.617 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.617 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.617 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.617 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.618 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.618 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.618 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.618 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.618 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.618 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.618 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.618 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.618 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.619 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.619 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.619 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.619 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.619 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.619 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.619 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.619 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.620 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.620 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.620 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.620 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.620 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.620 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.620 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.620 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.621 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.621 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.621 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.621 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.621 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.621 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.621 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.621 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.621 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.621 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.622 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.622 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.622 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.622 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.622 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.622 INFO:teuthology.orchestra.run.smithi090.stdout: "motd": { 2024-04-24T09:24:42.622 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "motd", 2024-04-24T09:24:42.622 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.622 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.623 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.623 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.623 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.623 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.623 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.623 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "The message of the day", 2024-04-24T09:24:42.623 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.623 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.623 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.624 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.624 INFO:teuthology.orchestra.run.smithi090.stdout: "redirect_resolve_ip_addr": { 2024-04-24T09:24:42.624 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "redirect_resolve_ip_addr", 2024-04-24T09:24:42.624 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.624 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.624 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.624 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.624 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.624 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.625 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.625 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.625 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.625 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.625 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.625 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.625 INFO:teuthology.orchestra.run.smithi090.stdout: "server_addr": { 2024-04-24T09:24:42.625 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_addr", 2024-04-24T09:24:42.625 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.626 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.626 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.626 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "::", 2024-04-24T09:24:42.626 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.626 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.626 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.626 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.626 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.626 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.627 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.627 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.627 INFO:teuthology.orchestra.run.smithi090.stdout: "server_port": { 2024-04-24T09:24:42.627 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_port", 2024-04-24T09:24:42.627 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.627 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.627 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.627 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8080", 2024-04-24T09:24:42.627 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.628 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.628 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.628 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.628 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.628 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.628 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.628 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.628 INFO:teuthology.orchestra.run.smithi090.stdout: "ssl": { 2024-04-24T09:24:42.628 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssl", 2024-04-24T09:24:42.629 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.629 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.629 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.629 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:42.629 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.629 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.629 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.629 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.629 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.630 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.630 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.630 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.630 INFO:teuthology.orchestra.run.smithi090.stdout: "ssl_server_port": { 2024-04-24T09:24:42.630 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssl_server_port", 2024-04-24T09:24:42.630 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.630 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.630 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.631 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8443", 2024-04-24T09:24:42.631 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.631 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.631 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.631 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.631 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.631 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.631 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.632 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.632 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_behaviour": { 2024-04-24T09:24:42.632 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_behaviour", 2024-04-24T09:24:42.632 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.632 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.632 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.632 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "redirect", 2024-04-24T09:24:42.632 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.632 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.633 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.633 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.633 INFO:teuthology.orchestra.run.smithi090.stdout: "redirect" 2024-04-24T09:24:42.633 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.633 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.633 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.633 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.633 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.633 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.634 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_error_status_code": { 2024-04-24T09:24:42.634 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_error_status_code", 2024-04-24T09:24:42.634 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.634 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.634 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.634 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "500", 2024-04-24T09:24:42.634 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "400", 2024-04-24T09:24:42.634 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "599", 2024-04-24T09:24:42.634 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.635 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.635 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.635 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.635 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.635 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.635 INFO:teuthology.orchestra.run.smithi090.stdout: "url_prefix": { 2024-04-24T09:24:42.635 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "url_prefix", 2024-04-24T09:24:42.635 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.636 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.636 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.636 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.636 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.636 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.636 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.636 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.636 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.636 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.636 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.637 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.637 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.637 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.637 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.637 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "devicehealth", 2024-04-24T09:24:42.637 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.637 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.637 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.638 INFO:teuthology.orchestra.run.smithi090.stdout: "enable_monitoring": { 2024-04-24T09:24:42.638 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enable_monitoring", 2024-04-24T09:24:42.638 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.638 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.638 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.638 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:42.638 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.638 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.638 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.639 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "monitor device health metrics", 2024-04-24T09:24:42.639 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.639 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.639 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.639 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.639 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.639 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.639 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.639 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.640 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.640 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.640 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.640 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.640 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.640 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.640 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.640 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.641 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.641 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.641 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.641 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.641 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.641 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.641 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.641 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.641 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.642 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.642 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.642 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.642 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.642 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.642 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.642 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.642 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.642 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.643 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.643 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.643 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.643 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.643 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.643 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.643 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.643 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.643 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.644 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.644 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.644 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.644 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.644 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.644 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.644 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.644 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.644 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.645 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.645 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.645 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.645 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.645 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.645 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.645 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.645 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.646 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.646 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.646 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.646 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.646 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.646 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.646 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.646 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.647 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.647 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.647 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.647 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.647 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.647 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.647 INFO:teuthology.orchestra.run.smithi090.stdout: "mark_out_threshold": { 2024-04-24T09:24:42.647 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mark_out_threshold", 2024-04-24T09:24:42.647 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:42.648 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.648 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.648 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "2419200", 2024-04-24T09:24:42.648 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.648 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.648 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.648 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-24T09:24:42.648 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.649 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.649 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.649 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.649 INFO:teuthology.orchestra.run.smithi090.stdout: "pool_name": { 2024-04-24T09:24:42.649 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pool_name", 2024-04-24T09:24:42.649 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.649 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.649 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.650 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "device_health_metrics", 2024-04-24T09:24:42.650 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.650 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.650 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.650 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-24T09:24:42.650 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.650 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.651 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.651 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.651 INFO:teuthology.orchestra.run.smithi090.stdout: "retention_period": { 2024-04-24T09:24:42.651 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "retention_period", 2024-04-24T09:24:42.651 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:42.651 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.651 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.651 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "15552000", 2024-04-24T09:24:42.651 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.652 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.652 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.652 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how long to retain device health metrics", 2024-04-24T09:24:42.652 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.652 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.652 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.652 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.652 INFO:teuthology.orchestra.run.smithi090.stdout: "scrape_frequency": { 2024-04-24T09:24:42.652 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "scrape_frequency", 2024-04-24T09:24:42.653 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:42.653 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.653 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.653 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "86400", 2024-04-24T09:24:42.653 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.653 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.653 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.653 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-24T09:24:42.654 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.654 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.654 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.654 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.654 INFO:teuthology.orchestra.run.smithi090.stdout: "self_heal": { 2024-04-24T09:24:42.654 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "self_heal", 2024-04-24T09:24:42.654 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.654 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.655 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.655 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:42.655 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.655 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.655 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.655 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-24T09:24:42.655 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.655 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.655 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.656 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.656 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-04-24T09:24:42.656 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-04-24T09:24:42.656 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:42.656 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.656 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.656 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-04-24T09:24:42.656 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.656 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.657 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.657 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to wake up and check device health", 2024-04-24T09:24:42.657 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.657 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.657 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.657 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.657 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_threshold": { 2024-04-24T09:24:42.657 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_threshold", 2024-04-24T09:24:42.657 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:42.658 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.658 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.658 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "7257600", 2024-04-24T09:24:42.658 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.658 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.658 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.658 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-24T09:24:42.658 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.658 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.659 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.659 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.659 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.659 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.659 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.659 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "diskprediction_local", 2024-04-24T09:24:42.659 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.659 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.659 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.660 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.660 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.660 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.660 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.660 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.660 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.660 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.660 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.661 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.661 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.661 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.661 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.661 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.661 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.661 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.661 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.661 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.662 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.662 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.662 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.662 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.662 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.662 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.662 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.662 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.663 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.663 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.663 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.663 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.663 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.663 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.663 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.663 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.663 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.663 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.664 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.664 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.664 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.664 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.664 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.664 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.664 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.664 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.664 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.665 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.665 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.665 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.665 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.665 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.665 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.665 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.665 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.665 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.666 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.666 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.666 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.666 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.666 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.666 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.666 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.666 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.666 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.667 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.667 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.667 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.667 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.667 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.667 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.667 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.667 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.667 INFO:teuthology.orchestra.run.smithi090.stdout: "predict_interval": { 2024-04-24T09:24:42.668 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "predict_interval", 2024-04-24T09:24:42.668 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.668 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.668 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.668 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "86400", 2024-04-24T09:24:42.668 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.668 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.668 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.668 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.669 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.669 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.669 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.669 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.669 INFO:teuthology.orchestra.run.smithi090.stdout: "predictor_model": { 2024-04-24T09:24:42.669 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "predictor_model", 2024-04-24T09:24:42.669 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.669 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.669 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.669 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "prophetstor", 2024-04-24T09:24:42.670 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.670 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.670 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.670 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.670 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.670 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.670 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.671 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.671 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-04-24T09:24:42.671 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-04-24T09:24:42.671 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.671 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.671 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.671 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-04-24T09:24:42.671 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.671 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.672 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.672 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.672 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.672 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.672 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.672 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.672 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.672 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.672 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.673 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "influx", 2024-04-24T09:24:42.673 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": false, 2024-04-24T09:24:42.673 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "influxdb python module not found", 2024-04-24T09:24:42.673 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.673 INFO:teuthology.orchestra.run.smithi090.stdout: "batch_size": { 2024-04-24T09:24:42.673 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "batch_size", 2024-04-24T09:24:42.673 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.673 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.673 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.674 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5000", 2024-04-24T09:24:42.674 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.674 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.674 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.674 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-24T09:24:42.674 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.674 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.674 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.674 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.675 INFO:teuthology.orchestra.run.smithi090.stdout: "database": { 2024-04-24T09:24:42.675 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "database", 2024-04-24T09:24:42.675 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.675 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.675 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.675 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "ceph", 2024-04-24T09:24:42.675 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.675 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.675 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.676 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-24T09:24:42.676 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.676 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.676 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.676 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.676 INFO:teuthology.orchestra.run.smithi090.stdout: "hostname": { 2024-04-24T09:24:42.676 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "hostname", 2024-04-24T09:24:42.676 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.676 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.677 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.677 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.677 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.677 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.677 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.677 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "InfluxDB server hostname", 2024-04-24T09:24:42.677 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.677 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.677 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.677 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.678 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-04-24T09:24:42.678 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-04-24T09:24:42.678 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:42.678 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.678 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.678 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "30", 2024-04-24T09:24:42.678 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "5", 2024-04-24T09:24:42.678 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.678 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.679 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-24T09:24:42.679 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.679 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.679 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.679 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.679 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.679 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.679 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.680 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.680 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.680 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.680 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.680 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.680 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.680 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.680 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.681 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.681 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.681 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.681 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.681 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.681 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.681 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.681 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.681 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.682 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.682 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.682 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.682 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.682 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.682 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.682 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.682 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.682 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.683 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.683 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.683 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.683 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.683 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.683 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.683 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.683 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.683 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.684 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.684 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.684 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.684 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.684 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.684 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.684 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.684 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.684 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.685 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.685 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.685 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.685 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.685 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.685 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.685 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.685 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.685 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.686 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.686 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.686 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.686 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.686 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.686 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.686 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.686 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.687 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.687 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.687 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.687 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.687 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.687 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.687 INFO:teuthology.orchestra.run.smithi090.stdout: "password": { 2024-04-24T09:24:42.687 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "password", 2024-04-24T09:24:42.687 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.688 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.688 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.688 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.688 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.688 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.688 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.688 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "password of InfluxDB server user", 2024-04-24T09:24:42.688 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.688 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.689 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.689 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.689 INFO:teuthology.orchestra.run.smithi090.stdout: "port": { 2024-04-24T09:24:42.689 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "port", 2024-04-24T09:24:42.689 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.689 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.689 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.689 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8086", 2024-04-24T09:24:42.689 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.690 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.690 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.690 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "InfluxDB server port", 2024-04-24T09:24:42.690 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.690 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.690 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.690 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.690 INFO:teuthology.orchestra.run.smithi090.stdout: "ssl": { 2024-04-24T09:24:42.691 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssl", 2024-04-24T09:24:42.691 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.691 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.691 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.691 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "false", 2024-04-24T09:24:42.691 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.691 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.691 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.691 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T09:24:42.692 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.692 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.692 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.692 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.692 INFO:teuthology.orchestra.run.smithi090.stdout: "threads": { 2024-04-24T09:24:42.692 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "threads", 2024-04-24T09:24:42.692 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.692 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.692 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.693 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-04-24T09:24:42.693 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-04-24T09:24:42.693 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "32", 2024-04-24T09:24:42.693 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.693 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-24T09:24:42.693 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.693 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.693 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.693 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.694 INFO:teuthology.orchestra.run.smithi090.stdout: "username": { 2024-04-24T09:24:42.694 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "username", 2024-04-24T09:24:42.694 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.694 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.694 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.694 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.694 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.694 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.694 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.695 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "username of InfluxDB server user", 2024-04-24T09:24:42.695 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.695 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.695 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.695 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.695 INFO:teuthology.orchestra.run.smithi090.stdout: "verify_ssl": { 2024-04-24T09:24:42.695 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "verify_ssl", 2024-04-24T09:24:42.695 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.695 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.696 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.696 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "true", 2024-04-24T09:24:42.696 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.696 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.696 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.696 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T09:24:42.696 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.696 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.696 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.697 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.697 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.697 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.697 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.697 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "insights", 2024-04-24T09:24:42.697 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.697 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.697 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.697 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.698 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.698 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.698 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.698 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.698 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.698 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.698 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.698 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.698 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.699 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.699 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.699 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.699 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.699 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.699 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.699 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.699 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.699 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.700 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.700 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.700 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.700 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.700 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.700 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.700 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.700 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.700 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.701 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.701 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.701 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.701 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.701 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.701 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.701 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.701 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.702 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.702 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.702 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.702 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.702 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.702 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.702 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.702 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.702 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.703 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.703 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.703 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.703 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.703 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.703 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.703 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.703 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.703 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.704 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.704 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.704 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.704 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.704 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.704 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.704 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.704 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.704 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.704 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.705 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.705 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.705 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.705 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.705 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.705 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.705 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.705 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.705 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.705 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "iostat", 2024-04-24T09:24:42.706 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.706 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.706 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.706 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.706 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.706 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.706 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.706 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.706 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.707 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.707 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.707 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.707 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.707 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.707 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.707 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.707 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.707 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.707 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.708 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.708 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.708 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.708 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.708 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.708 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.708 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.708 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.708 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.709 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.709 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.709 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.709 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.709 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.709 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.709 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.709 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.709 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.709 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.710 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.710 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.710 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.710 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.710 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.710 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.710 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.710 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.710 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.711 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.711 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.711 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.711 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.711 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.711 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.711 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.711 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.712 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.712 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.712 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.712 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.712 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.712 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.712 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.712 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.712 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.712 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.713 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.713 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.713 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.713 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.713 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.713 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.713 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.713 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.713 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.714 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.714 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.714 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "k8sevents", 2024-04-24T09:24:42.714 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.714 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.714 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.714 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_event_retention_days": { 2024-04-24T09:24:42.714 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ceph_event_retention_days", 2024-04-24T09:24:42.715 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.715 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.715 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.715 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "7", 2024-04-24T09:24:42.715 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.715 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.715 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.715 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-24T09:24:42.716 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.716 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.716 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.716 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.716 INFO:teuthology.orchestra.run.smithi090.stdout: "config_check_secs": { 2024-04-24T09:24:42.716 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "config_check_secs", 2024-04-24T09:24:42.716 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.716 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.716 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.717 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-04-24T09:24:42.717 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "10", 2024-04-24T09:24:42.717 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.717 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.717 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-24T09:24:42.717 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.717 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.717 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.717 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.718 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.718 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.718 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.718 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.718 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.718 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.718 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.718 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.718 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.719 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.719 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.719 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.719 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.719 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.719 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.719 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.719 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.720 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.720 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.720 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.720 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.720 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.720 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.720 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.720 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.720 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.721 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.721 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.721 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.721 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.721 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.721 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.721 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.721 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.721 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.722 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.722 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.722 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.722 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.722 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.722 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.722 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.722 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.722 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.723 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.723 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.723 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.723 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.723 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.723 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.723 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.723 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.723 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.724 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.724 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.724 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.724 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.724 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.724 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.724 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.724 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.724 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.725 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.725 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.725 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.725 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.725 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.725 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.725 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.725 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.725 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.726 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.726 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.726 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "localpool", 2024-04-24T09:24:42.726 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.726 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.726 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.726 INFO:teuthology.orchestra.run.smithi090.stdout: "failure_domain": { 2024-04-24T09:24:42.726 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "failure_domain", 2024-04-24T09:24:42.727 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.727 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.727 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.727 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "host", 2024-04-24T09:24:42.727 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.727 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.727 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.727 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "failure domain for any created local pool", 2024-04-24T09:24:42.727 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-24T09:24:42.728 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.728 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.728 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.728 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.728 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.728 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.728 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.728 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.728 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.729 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.729 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.729 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.729 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.729 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.729 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.729 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.729 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.729 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.729 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.730 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.730 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.730 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.730 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.730 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.730 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.730 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.731 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.731 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.731 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.731 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.731 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.731 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.731 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.731 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.731 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.732 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.732 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.732 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.732 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.732 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.732 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.732 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.732 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.732 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.733 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.733 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.733 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.733 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.733 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.733 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.733 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.733 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.733 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.733 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.734 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.734 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.734 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.734 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.734 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.734 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.734 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.734 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.734 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.735 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.735 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.735 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.735 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.735 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.735 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.735 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.735 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.735 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.736 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.736 INFO:teuthology.orchestra.run.smithi090.stdout: "min_size": { 2024-04-24T09:24:42.736 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "min_size", 2024-04-24T09:24:42.736 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.736 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.736 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.736 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.736 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.736 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.736 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.737 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "default min_size for any created local pool", 2024-04-24T09:24:42.737 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-24T09:24:42.737 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.737 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.737 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.737 INFO:teuthology.orchestra.run.smithi090.stdout: "num_rep": { 2024-04-24T09:24:42.737 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "num_rep", 2024-04-24T09:24:42.737 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.738 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.738 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.738 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "3", 2024-04-24T09:24:42.738 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.738 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.738 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.738 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "default replica count for any created local pool", 2024-04-24T09:24:42.738 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.738 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.739 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.739 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.739 INFO:teuthology.orchestra.run.smithi090.stdout: "pg_num": { 2024-04-24T09:24:42.739 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pg_num", 2024-04-24T09:24:42.739 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.739 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.739 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.739 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "128", 2024-04-24T09:24:42.739 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.740 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.740 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.740 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "default pg_num for any created local pool", 2024-04-24T09:24:42.740 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.740 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.740 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.740 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.740 INFO:teuthology.orchestra.run.smithi090.stdout: "prefix": { 2024-04-24T09:24:42.740 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "prefix", 2024-04-24T09:24:42.741 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.741 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.741 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.741 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.741 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.741 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.741 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.741 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "name prefix for any created local pool", 2024-04-24T09:24:42.741 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.742 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.742 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.742 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.742 INFO:teuthology.orchestra.run.smithi090.stdout: "subtree": { 2024-04-24T09:24:42.742 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "subtree", 2024-04-24T09:24:42.742 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.742 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.742 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.742 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "rack", 2024-04-24T09:24:42.742 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.743 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.743 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.743 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-24T09:24:42.743 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-24T09:24:42.743 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.743 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.743 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.743 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.743 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.744 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.744 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mds_autoscaler", 2024-04-24T09:24:42.744 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.744 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.744 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.744 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.744 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.744 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.744 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.744 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.745 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.745 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.745 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.745 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.745 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.745 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.745 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.745 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.745 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.746 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.746 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.746 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.746 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.746 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.746 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.746 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.746 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.747 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.747 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.747 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.747 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.747 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.747 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.747 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.747 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.747 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.748 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.748 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.748 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.748 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.748 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.748 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.748 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.748 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.748 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.749 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.749 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.749 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.749 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.749 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.749 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.749 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.749 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.750 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.750 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.750 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.750 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.750 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.750 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.750 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.750 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.751 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.751 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.751 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.751 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.751 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.751 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.751 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.751 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.751 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.752 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.752 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.752 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.752 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.752 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.752 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.752 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.752 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.753 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mirroring", 2024-04-24T09:24:42.753 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.753 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.753 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.753 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.753 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.753 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.753 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.754 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.754 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.754 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.754 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.754 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.754 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.754 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.754 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.755 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.755 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.755 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.755 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.755 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.755 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.755 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.755 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.756 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.756 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.756 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.756 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.756 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.756 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.756 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.756 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.756 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.757 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.757 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.757 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.757 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.757 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.757 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.757 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.757 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.757 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.758 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.758 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.758 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.758 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.758 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.758 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.758 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.758 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.758 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.759 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.759 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.759 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.759 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.759 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.759 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.759 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.759 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.760 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.760 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.760 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.760 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.760 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.760 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.760 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.760 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.760 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.761 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.761 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.761 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.761 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.761 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.761 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.761 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.761 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.761 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.762 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "nfs", 2024-04-24T09:24:42.762 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.762 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.762 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.762 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.762 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.762 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.762 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.762 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.763 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.763 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.763 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.763 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.763 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.763 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.763 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.763 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.763 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.764 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.764 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.764 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.764 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.764 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.764 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.764 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.764 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.764 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.765 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.765 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.765 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.765 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.765 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.765 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.765 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.765 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.765 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.765 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.766 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.766 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.766 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.766 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.766 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.766 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.766 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.766 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.767 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.767 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.767 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.767 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.767 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.767 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.767 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.767 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.767 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.768 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.768 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.768 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.768 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.768 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.768 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.768 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.768 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.768 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.769 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.769 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.769 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.769 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.769 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.769 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.769 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.769 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.769 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.769 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.770 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.770 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.770 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.770 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.770 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "orchestrator", 2024-04-24T09:24:42.770 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.770 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.770 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.771 INFO:teuthology.orchestra.run.smithi090.stdout: "fail_fs": { 2024-04-24T09:24:42.771 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "fail_fs", 2024-04-24T09:24:42.771 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.771 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.771 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.771 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.771 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.771 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.771 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.772 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-24T09:24:42.772 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.772 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.772 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.772 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.772 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.772 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.772 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.772 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.773 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.773 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.773 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.773 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.773 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.773 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.773 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.773 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.773 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.773 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.774 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.774 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.774 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.774 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.774 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.774 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.774 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.774 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.775 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.775 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.775 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.775 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.775 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.775 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.775 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.775 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.776 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.776 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.776 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.776 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.776 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.776 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.776 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.776 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.777 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.777 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.777 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.777 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.777 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.777 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.777 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.777 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.777 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.778 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.778 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.778 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.778 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.778 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.778 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.778 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.778 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.778 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.779 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.779 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.779 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.779 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.779 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.779 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.779 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.779 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.779 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.780 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.780 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.780 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.780 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.780 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.780 INFO:teuthology.orchestra.run.smithi090.stdout: "orchestrator": { 2024-04-24T09:24:42.780 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "orchestrator", 2024-04-24T09:24:42.780 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.781 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.781 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.781 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.781 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.781 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.781 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.781 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm", 2024-04-24T09:24:42.781 INFO:teuthology.orchestra.run.smithi090.stdout: "rook", 2024-04-24T09:24:42.781 INFO:teuthology.orchestra.run.smithi090.stdout: "test_orchestrator" 2024-04-24T09:24:42.782 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.782 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Orchestrator backend", 2024-04-24T09:24:42.782 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.782 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.782 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.782 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.782 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.782 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.782 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.783 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "osd_perf_query", 2024-04-24T09:24:42.783 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.783 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.783 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.783 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.783 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.783 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.783 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.784 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.784 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.784 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.784 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.784 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.784 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.784 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.784 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.784 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.785 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.785 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.785 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.785 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.785 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.785 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.785 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.786 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.786 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.786 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.786 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.786 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.786 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.786 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.786 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.786 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.787 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.787 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.787 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.787 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.787 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.787 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.787 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.787 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.787 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.788 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.788 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.788 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.788 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.788 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.788 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.788 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.788 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.788 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.789 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.789 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.789 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.789 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.789 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.789 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.789 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.789 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.789 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.790 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.790 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.790 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.790 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.790 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.790 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.790 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.790 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.791 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.791 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.791 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.791 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.791 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.791 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.791 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.791 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.792 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.792 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "osd_support", 2024-04-24T09:24:42.792 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.792 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.792 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.792 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.792 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.792 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.792 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.792 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.793 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.793 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.793 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.793 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.793 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.793 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.793 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.793 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.793 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.794 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.794 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.794 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.794 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.794 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.794 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.794 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.794 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.794 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.795 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.795 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.795 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.795 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.795 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.795 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.795 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.795 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.795 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.796 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.796 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.796 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.796 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.796 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.796 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.796 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.796 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.796 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.797 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.797 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.797 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.797 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.797 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.797 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.797 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.797 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.797 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.798 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.798 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.798 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.798 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.798 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.798 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.798 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.798 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.798 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.799 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.799 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.799 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.799 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.799 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.799 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.799 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.799 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.800 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.800 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.800 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.800 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.800 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.800 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.800 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pg_autoscaler", 2024-04-24T09:24:42.800 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.801 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.801 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.801 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.801 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.801 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.801 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.801 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.801 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.801 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.802 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.802 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.802 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.802 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.802 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.802 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.802 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.802 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.802 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.803 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.803 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.803 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.803 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.803 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.803 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.803 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.803 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.803 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.804 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.804 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.804 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.804 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.804 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.804 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.804 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.804 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.805 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.805 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.805 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.805 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.805 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.805 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.805 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.805 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.806 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.806 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.806 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.806 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.806 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.806 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.806 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.806 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.807 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.807 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.807 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.807 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.807 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.807 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.807 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.807 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.807 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.808 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.808 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.808 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.808 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.808 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.808 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.808 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.808 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.808 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.809 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.809 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.809 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.809 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-04-24T09:24:42.809 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-04-24T09:24:42.809 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:42.809 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.809 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.810 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-04-24T09:24:42.810 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.810 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.810 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.810 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.810 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.810 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.810 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.810 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.811 INFO:teuthology.orchestra.run.smithi090.stdout: "threshold": { 2024-04-24T09:24:42.811 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "threshold", 2024-04-24T09:24:42.811 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-04-24T09:24:42.811 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.811 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.811 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "3.0", 2024-04-24T09:24:42.811 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1.0", 2024-04-24T09:24:42.811 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.812 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.812 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "scaling threshold", 2024-04-24T09:24:42.812 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-24T09:24:42.812 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.812 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.812 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.812 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.812 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.812 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.813 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "progress", 2024-04-24T09:24:42.813 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.813 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.813 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.813 INFO:teuthology.orchestra.run.smithi090.stdout: "allow_pg_recovery_event": { 2024-04-24T09:24:42.813 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "allow_pg_recovery_event", 2024-04-24T09:24:42.813 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.813 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.813 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.814 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.814 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.814 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.814 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.814 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-24T09:24:42.814 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.814 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.814 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.814 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.815 INFO:teuthology.orchestra.run.smithi090.stdout: "enabled": { 2024-04-24T09:24:42.815 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enabled", 2024-04-24T09:24:42.815 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.815 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.815 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.815 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:42.815 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.815 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.815 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.816 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.816 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.816 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.816 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.816 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.816 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.816 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.816 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.816 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.816 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.817 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.817 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.817 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.817 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.817 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.817 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.817 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.817 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.817 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.818 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.818 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.818 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.818 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.818 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.818 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.818 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.818 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.818 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.819 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.819 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.819 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.819 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.819 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.819 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.819 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.819 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.819 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.820 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.820 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.820 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.820 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.820 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.820 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.820 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.820 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.820 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.821 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.821 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.821 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.821 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.821 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.821 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.821 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.821 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.822 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.822 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.822 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.822 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.822 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.822 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.822 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.822 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.823 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.823 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.823 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.823 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.823 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.823 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.823 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.823 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.824 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.824 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.824 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.824 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.824 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.824 INFO:teuthology.orchestra.run.smithi090.stdout: "max_completed_events": { 2024-04-24T09:24:42.824 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_completed_events", 2024-04-24T09:24:42.824 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.825 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.825 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.825 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "50", 2024-04-24T09:24:42.825 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.825 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.825 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.825 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "number of past completed events to remember", 2024-04-24T09:24:42.825 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.825 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.825 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.826 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.826 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-04-24T09:24:42.826 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-04-24T09:24:42.826 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:42.826 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.826 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.826 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-04-24T09:24:42.826 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.827 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.827 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.827 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how long the module is going to sleep", 2024-04-24T09:24:42.827 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.827 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.827 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.827 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.827 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.827 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.828 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.828 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "prometheus", 2024-04-24T09:24:42.828 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.828 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.828 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.828 INFO:teuthology.orchestra.run.smithi090.stdout: "cache": { 2024-04-24T09:24:42.828 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cache", 2024-04-24T09:24:42.828 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.828 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.828 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.829 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:42.829 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.829 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.829 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.829 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.829 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.829 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.829 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.829 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.830 INFO:teuthology.orchestra.run.smithi090.stdout: "exclude_perf_counters": { 2024-04-24T09:24:42.830 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "exclude_perf_counters", 2024-04-24T09:24:42.830 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.830 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.830 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.830 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:42.830 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.830 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.830 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.831 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-24T09:24:42.831 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-24T09:24:42.831 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.831 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.831 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.831 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.831 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.831 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.832 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.832 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.832 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.832 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.832 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.832 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.832 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.832 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.832 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.833 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.833 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.833 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.833 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.833 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.833 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.833 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.833 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.834 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.834 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.834 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.834 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.834 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.834 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.834 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.834 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.834 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.835 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.835 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.835 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.835 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.835 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.835 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.835 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.835 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.835 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.836 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.836 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.836 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.836 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.836 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.836 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.836 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.836 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.836 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.837 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.837 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.837 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.837 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.837 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.837 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.837 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.837 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.838 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.838 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.838 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.838 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.838 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.838 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.838 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.838 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.838 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.839 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.839 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.839 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.839 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.839 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.839 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.839 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_stats_pools": { 2024-04-24T09:24:42.839 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_stats_pools", 2024-04-24T09:24:42.839 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.840 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.840 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.840 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.840 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.840 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.840 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.840 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.840 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.841 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.841 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.841 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.841 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-24T09:24:42.841 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-24T09:24:42.841 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.841 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.841 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.841 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "300", 2024-04-24T09:24:42.841 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.842 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.842 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.842 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.842 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.842 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.842 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.842 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.842 INFO:teuthology.orchestra.run.smithi090.stdout: "scrape_interval": { 2024-04-24T09:24:42.842 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "scrape_interval", 2024-04-24T09:24:42.843 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-04-24T09:24:42.843 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.843 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.843 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "15.0", 2024-04-24T09:24:42.843 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.843 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.843 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.843 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.843 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.843 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.844 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.844 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.844 INFO:teuthology.orchestra.run.smithi090.stdout: "server_addr": { 2024-04-24T09:24:42.844 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_addr", 2024-04-24T09:24:42.844 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.844 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.844 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.844 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "::", 2024-04-24T09:24:42.844 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.845 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.845 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.845 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-24T09:24:42.845 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.845 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.845 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.845 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.845 INFO:teuthology.orchestra.run.smithi090.stdout: "server_port": { 2024-04-24T09:24:42.845 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_port", 2024-04-24T09:24:42.846 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.846 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.846 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.846 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "9283", 2024-04-24T09:24:42.846 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.846 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.846 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.846 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-24T09:24:42.846 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.847 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.847 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.847 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.847 INFO:teuthology.orchestra.run.smithi090.stdout: "stale_cache_strategy": { 2024-04-24T09:24:42.847 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "stale_cache_strategy", 2024-04-24T09:24:42.847 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.847 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.847 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.848 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "log", 2024-04-24T09:24:42.848 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.848 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.848 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.848 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.848 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.848 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.848 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.848 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.849 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_behaviour": { 2024-04-24T09:24:42.849 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_behaviour", 2024-04-24T09:24:42.849 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.849 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.849 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.849 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "default", 2024-04-24T09:24:42.849 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.849 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.849 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.850 INFO:teuthology.orchestra.run.smithi090.stdout: "default", 2024-04-24T09:24:42.850 INFO:teuthology.orchestra.run.smithi090.stdout: "error" 2024-04-24T09:24:42.850 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.850 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.850 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.850 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.850 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.850 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.851 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_error_status_code": { 2024-04-24T09:24:42.851 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_error_status_code", 2024-04-24T09:24:42.851 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.851 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.851 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.851 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "500", 2024-04-24T09:24:42.851 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "400", 2024-04-24T09:24:42.851 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "599", 2024-04-24T09:24:42.852 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.852 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.852 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.852 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.852 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.852 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.852 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.853 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.853 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.853 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_support", 2024-04-24T09:24:42.853 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.853 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.853 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.853 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.853 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.853 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.854 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.854 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.854 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.854 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.854 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.854 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.854 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.854 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.854 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.855 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.855 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.855 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.855 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.855 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.855 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.855 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.855 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.856 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.856 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.856 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.856 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.856 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.856 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.856 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.856 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.856 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.857 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.857 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.857 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.857 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.857 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.857 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.857 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.857 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.857 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.858 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.858 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.858 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.858 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.858 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.858 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.858 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.858 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.859 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.859 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.859 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.859 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.859 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.859 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.859 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.859 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.859 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.860 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.860 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.860 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.860 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.860 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.860 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.860 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.860 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.860 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.861 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.861 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.861 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.861 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.861 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.861 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.861 INFO:teuthology.orchestra.run.smithi090.stdout: "max_concurrent_snap_create": { 2024-04-24T09:24:42.861 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_concurrent_snap_create", 2024-04-24T09:24:42.862 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.862 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.862 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.862 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-04-24T09:24:42.862 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.862 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.862 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.862 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.863 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.863 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.863 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.863 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.863 INFO:teuthology.orchestra.run.smithi090.stdout: "mirror_snapshot_schedule": { 2024-04-24T09:24:42.863 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mirror_snapshot_schedule", 2024-04-24T09:24:42.863 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.863 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.863 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.864 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.864 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.864 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.864 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.864 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.864 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.864 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.864 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.864 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.864 INFO:teuthology.orchestra.run.smithi090.stdout: "trash_purge_schedule": { 2024-04-24T09:24:42.865 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "trash_purge_schedule", 2024-04-24T09:24:42.865 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.865 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.865 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.865 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.865 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.865 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.865 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.865 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.866 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.866 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.866 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.866 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.866 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.866 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.866 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.866 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "restful", 2024-04-24T09:24:42.866 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.867 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.867 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.867 INFO:teuthology.orchestra.run.smithi090.stdout: "enable_auth": { 2024-04-24T09:24:42.867 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enable_auth", 2024-04-24T09:24:42.867 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.867 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.867 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.867 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:42.867 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.868 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.868 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.868 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.868 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.868 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.868 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.868 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.868 INFO:teuthology.orchestra.run.smithi090.stdout: "key_file": { 2024-04-24T09:24:42.868 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "key_file", 2024-04-24T09:24:42.869 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.869 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.869 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.869 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.869 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.869 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.869 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.869 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.869 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.870 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.870 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.870 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.870 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.870 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.870 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.870 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.870 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.871 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.871 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.871 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.871 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.871 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.871 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.871 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.871 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.871 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.871 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.872 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.872 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.872 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.872 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.872 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.872 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.872 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.872 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.872 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.873 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.873 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.873 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.873 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.873 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.873 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.873 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.873 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.873 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.874 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.874 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.874 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.874 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.874 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.874 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.874 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.874 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.874 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.875 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.875 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.875 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.875 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.875 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.875 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.875 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.875 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.875 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.876 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.876 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.876 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.876 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.876 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.876 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.876 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.876 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.877 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.877 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.877 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.877 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.877 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.877 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.877 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.877 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.878 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.878 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.878 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.878 INFO:teuthology.orchestra.run.smithi090.stdout: "server_addr": { 2024-04-24T09:24:42.878 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_addr", 2024-04-24T09:24:42.878 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.878 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.878 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.879 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.879 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.879 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.879 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.879 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.879 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.879 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.880 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.880 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.880 INFO:teuthology.orchestra.run.smithi090.stdout: "server_port": { 2024-04-24T09:24:42.880 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_port", 2024-04-24T09:24:42.880 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.880 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.880 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.880 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.881 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.881 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.881 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.881 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.881 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.881 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.881 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.881 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.881 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.882 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.882 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.882 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rgw", 2024-04-24T09:24:42.882 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.882 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.882 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.882 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.882 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.882 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.883 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.883 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.883 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.883 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.883 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.883 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.883 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.883 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.884 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.884 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.884 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.884 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.884 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.884 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.884 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.884 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.885 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.885 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.885 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.885 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.885 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.885 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.885 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.885 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.885 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.886 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.886 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.886 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.886 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.886 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.886 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.886 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.886 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.886 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.887 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.887 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.887 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.887 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.887 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.887 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.887 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.887 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.888 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.888 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.888 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.888 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.888 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.888 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.888 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.888 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.888 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.889 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.889 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.889 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.889 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.889 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.889 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.889 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.889 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.889 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.890 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.890 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.890 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.890 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.890 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.890 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.890 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.890 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.891 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.891 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.891 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rook", 2024-04-24T09:24:42.891 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.891 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.891 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.891 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.891 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.891 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.892 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.892 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.892 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.892 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.892 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.892 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.892 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.892 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.892 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.893 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.893 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.893 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.893 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.893 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.893 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.893 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.893 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.893 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.894 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.894 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.894 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.894 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.894 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.894 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.894 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.894 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.894 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.895 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.895 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.895 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.895 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.895 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.895 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.895 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.895 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.895 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.896 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.896 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.896 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.896 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.896 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.896 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.896 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.896 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.896 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.896 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.897 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.897 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.897 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.897 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.897 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.897 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.897 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.897 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.897 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.898 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.898 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.898 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.898 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.898 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.898 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.898 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.898 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.898 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.899 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.899 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.899 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.899 INFO:teuthology.orchestra.run.smithi090.stdout: "storage_class": { 2024-04-24T09:24:42.899 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "storage_class", 2024-04-24T09:24:42.899 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.899 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.899 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.899 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "local", 2024-04-24T09:24:42.899 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.900 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.900 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.900 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-24T09:24:42.900 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.900 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.900 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.900 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.900 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.900 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.901 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.901 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "selftest", 2024-04-24T09:24:42.901 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.901 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.901 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.901 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.901 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.901 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.901 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.901 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.902 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.902 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.902 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.902 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.902 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.902 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.902 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.902 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.902 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.903 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.903 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.903 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.903 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.903 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.903 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.903 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.903 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.903 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.903 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.904 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.904 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.904 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.904 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.904 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.904 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.904 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.904 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.904 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.905 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.905 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.905 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.905 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.905 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.905 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.905 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.905 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.905 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.905 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.906 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.906 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.906 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.906 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.906 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.906 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.906 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.906 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.906 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.907 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.907 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.907 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.907 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.907 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.907 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.907 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.907 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.907 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.908 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.908 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.908 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.908 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.908 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.908 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.908 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.908 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.908 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.909 INFO:teuthology.orchestra.run.smithi090.stdout: "roption1": { 2024-04-24T09:24:42.909 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "roption1", 2024-04-24T09:24:42.909 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.909 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.909 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.909 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.909 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.909 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.910 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.910 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.910 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.910 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.910 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.910 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.910 INFO:teuthology.orchestra.run.smithi090.stdout: "roption2": { 2024-04-24T09:24:42.911 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "roption2", 2024-04-24T09:24:42.911 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.911 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.911 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.911 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "xyz", 2024-04-24T09:24:42.911 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.911 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.911 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.911 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.912 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.912 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.912 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.912 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.912 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption1": { 2024-04-24T09:24:42.912 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption1", 2024-04-24T09:24:42.912 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.912 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.912 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.913 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.913 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.913 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.913 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.913 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.913 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.913 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.913 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.914 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.914 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption2": { 2024-04-24T09:24:42.914 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption2", 2024-04-24T09:24:42.914 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.914 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.914 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.914 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.915 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.915 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.915 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.915 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.915 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.915 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.915 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.915 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.915 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption3": { 2024-04-24T09:24:42.915 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption3", 2024-04-24T09:24:42.916 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-04-24T09:24:42.916 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.916 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.916 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.916 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.916 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.916 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.916 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.916 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.917 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.917 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.917 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.917 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption4": { 2024-04-24T09:24:42.917 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption4", 2024-04-24T09:24:42.917 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.917 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.917 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.918 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.918 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.918 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.918 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.918 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.918 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.918 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.918 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.919 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.919 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption5": { 2024-04-24T09:24:42.919 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption5", 2024-04-24T09:24:42.919 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.919 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.919 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.919 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.919 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.919 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.920 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.920 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.920 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.920 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.920 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.920 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.920 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption6": { 2024-04-24T09:24:42.920 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption6", 2024-04-24T09:24:42.920 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.921 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.921 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.921 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:42.921 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.921 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.921 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.921 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.921 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.921 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.922 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.922 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.922 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption7": { 2024-04-24T09:24:42.922 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption7", 2024-04-24T09:24:42.922 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.922 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.922 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.923 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.923 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-04-24T09:24:42.923 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "42", 2024-04-24T09:24:42.923 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.923 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.923 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.923 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.923 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.924 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.924 INFO:teuthology.orchestra.run.smithi090.stdout: "testkey": { 2024-04-24T09:24:42.924 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "testkey", 2024-04-24T09:24:42.924 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.924 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.924 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.924 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.924 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.924 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.925 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.925 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.925 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.925 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.925 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.925 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.925 INFO:teuthology.orchestra.run.smithi090.stdout: "testlkey": { 2024-04-24T09:24:42.925 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "testlkey", 2024-04-24T09:24:42.926 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.926 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.926 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.926 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.926 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.926 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.926 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.926 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.926 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.927 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.927 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.927 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.927 INFO:teuthology.orchestra.run.smithi090.stdout: "testnewline": { 2024-04-24T09:24:42.927 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "testnewline", 2024-04-24T09:24:42.927 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.927 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.927 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.928 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.928 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.928 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.928 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.928 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.928 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.928 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.928 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.928 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.929 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.929 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.929 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.929 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snap_schedule", 2024-04-24T09:24:42.929 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.929 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.929 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.929 INFO:teuthology.orchestra.run.smithi090.stdout: "allow_m_granularity": { 2024-04-24T09:24:42.930 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "allow_m_granularity", 2024-04-24T09:24:42.930 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.930 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.930 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.930 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.930 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.930 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.930 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.930 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "allow minute scheduled snapshots", 2024-04-24T09:24:42.931 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.931 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.931 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.931 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.931 INFO:teuthology.orchestra.run.smithi090.stdout: "dump_on_update": { 2024-04-24T09:24:42.931 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "dump_on_update", 2024-04-24T09:24:42.931 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.932 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.932 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.932 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.932 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.932 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.932 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.932 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "dump database to debug log on update", 2024-04-24T09:24:42.932 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.932 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.933 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.933 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.933 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.933 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.933 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.933 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.933 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.933 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.934 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.934 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.934 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.934 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.934 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.934 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.934 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.934 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.934 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.935 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.935 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.935 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.935 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.935 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.935 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.935 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.935 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.935 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.936 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.936 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.936 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.936 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.936 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.936 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.936 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.936 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.936 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.937 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.937 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.937 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.937 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.937 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.937 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.937 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.937 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.937 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.938 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.938 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.938 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.938 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.938 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.938 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.938 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.938 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.939 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.939 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.939 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.939 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.939 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.939 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.939 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.939 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.939 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.939 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.940 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.940 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.940 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.940 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.940 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.940 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.940 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.940 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.940 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.941 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.941 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.941 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.941 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.941 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "stats", 2024-04-24T09:24:42.941 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.941 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.941 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.941 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.942 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.942 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.942 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.942 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.942 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.942 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.942 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.942 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.942 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.943 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.943 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.943 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.943 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.943 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.943 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.943 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.943 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.943 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.943 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.944 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.944 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.944 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.944 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.944 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.944 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.944 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.944 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.945 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.945 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.945 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.945 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.945 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.945 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.945 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.945 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.945 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.946 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.946 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.946 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.946 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.946 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.946 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.946 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.946 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.946 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.946 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.947 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.947 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.947 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.947 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.947 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.947 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.947 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.947 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.947 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.948 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.948 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.948 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.948 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.948 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.948 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.948 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.948 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.948 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.948 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.949 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.949 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.949 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.949 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.949 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.949 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.949 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.949 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "status", 2024-04-24T09:24:42.949 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.950 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.950 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.950 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.950 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.950 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.950 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.950 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.950 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.951 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.951 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.951 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.951 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.951 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.951 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.951 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.951 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.952 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.952 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.952 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.952 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.952 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.952 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.952 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.952 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.952 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.952 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.953 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.953 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.953 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.953 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.953 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.953 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.953 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.953 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.954 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.954 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.954 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.954 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.954 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.954 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.954 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.954 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.954 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.955 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.955 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.955 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.955 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.955 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.955 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.955 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.955 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.956 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.956 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.956 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.956 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.956 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.956 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.956 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.956 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.957 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.957 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.957 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.957 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.957 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.957 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.957 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.957 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.958 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.958 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.958 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.958 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.958 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.958 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.958 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.958 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.959 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "telegraf", 2024-04-24T09:24:42.959 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.959 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.959 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.959 INFO:teuthology.orchestra.run.smithi090.stdout: "address": { 2024-04-24T09:24:42.959 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "address", 2024-04-24T09:24:42.959 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.959 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.960 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.960 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-24T09:24:42.960 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.960 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.960 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.960 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.960 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.960 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.961 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.961 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.961 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-04-24T09:24:42.961 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-04-24T09:24:42.961 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:42.961 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.961 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.961 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "15", 2024-04-24T09:24:42.961 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.962 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.962 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.962 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.962 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.962 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.962 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.962 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.962 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.963 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.963 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.963 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.963 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.963 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.963 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.963 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.963 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.963 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.964 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.964 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.964 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.964 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.964 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.964 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.964 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.965 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.965 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.965 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.965 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.965 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.965 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.965 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.965 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.966 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.966 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.966 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.966 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.966 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.966 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.966 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.966 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.966 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.967 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.967 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.967 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.967 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.967 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.967 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.967 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.967 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.967 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.968 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.968 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.968 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.968 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.968 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.968 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.968 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.968 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.968 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.969 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.969 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.969 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.969 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.969 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.969 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:42.969 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.969 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.969 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.970 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.970 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.970 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.970 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.970 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.970 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.970 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.970 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.971 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.971 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:42.971 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.971 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:42.971 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "telemetry", 2024-04-24T09:24:42.971 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:42.971 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:42.971 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:42.971 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_basic": { 2024-04-24T09:24:42.971 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_basic", 2024-04-24T09:24:42.972 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.972 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.972 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.972 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:42.972 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.972 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.972 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.972 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-24T09:24:42.972 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.973 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.973 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.973 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.973 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_crash": { 2024-04-24T09:24:42.973 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_crash", 2024-04-24T09:24:42.973 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.973 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.973 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.973 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:42.973 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.974 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.974 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.974 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-24T09:24:42.974 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.974 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.974 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.974 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.974 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_device": { 2024-04-24T09:24:42.974 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_device", 2024-04-24T09:24:42.975 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.975 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.975 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.975 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:42.975 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.975 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.975 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.975 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-24T09:24:42.975 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.976 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.976 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.976 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.976 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_ident": { 2024-04-24T09:24:42.976 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_ident", 2024-04-24T09:24:42.976 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.976 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.976 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.976 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.976 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.977 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.977 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.977 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-24T09:24:42.977 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.977 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.977 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.977 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.977 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_perf": { 2024-04-24T09:24:42.977 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_perf", 2024-04-24T09:24:42.978 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.978 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.978 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.978 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.978 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.978 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.978 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.978 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-24T09:24:42.979 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.979 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.979 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.979 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.979 INFO:teuthology.orchestra.run.smithi090.stdout: "contact": { 2024-04-24T09:24:42.979 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "contact", 2024-04-24T09:24:42.979 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.979 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.979 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.980 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.980 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.980 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.980 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.980 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.980 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.980 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.981 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.981 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.981 INFO:teuthology.orchestra.run.smithi090.stdout: "description": { 2024-04-24T09:24:42.981 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "description", 2024-04-24T09:24:42.981 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.981 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.981 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.981 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.981 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.982 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.982 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.982 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.982 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.982 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.982 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.982 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.982 INFO:teuthology.orchestra.run.smithi090.stdout: "device_url": { 2024-04-24T09:24:42.982 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "device_url", 2024-04-24T09:24:42.983 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.983 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.983 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.983 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-24T09:24:42.983 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.983 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.983 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.983 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.983 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.984 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.984 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.984 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.984 INFO:teuthology.orchestra.run.smithi090.stdout: "enabled": { 2024-04-24T09:24:42.984 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enabled", 2024-04-24T09:24:42.984 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.984 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.984 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.985 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.985 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.985 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.985 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.985 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.985 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.985 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.985 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.985 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.986 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-04-24T09:24:42.986 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-04-24T09:24:42.986 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.986 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.986 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.986 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "24", 2024-04-24T09:24:42.986 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "8", 2024-04-24T09:24:42.986 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.987 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.987 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.987 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.987 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.987 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.987 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.987 INFO:teuthology.orchestra.run.smithi090.stdout: "last_opt_revision": { 2024-04-24T09:24:42.987 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "last_opt_revision", 2024-04-24T09:24:42.987 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:42.988 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.988 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.988 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "1", 2024-04-24T09:24:42.988 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.988 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.988 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.988 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.988 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.988 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.989 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.989 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.989 INFO:teuthology.orchestra.run.smithi090.stdout: "leaderboard": { 2024-04-24T09:24:42.989 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "leaderboard", 2024-04-24T09:24:42.989 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.989 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.989 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.989 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.990 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.990 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.990 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.990 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.990 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.990 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.990 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.990 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.991 INFO:teuthology.orchestra.run.smithi090.stdout: "leaderboard_description": { 2024-04-24T09:24:42.991 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "leaderboard_description", 2024-04-24T09:24:42.991 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.991 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.991 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:42.991 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.991 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.991 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.991 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.992 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.992 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.992 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.992 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.992 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.992 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:42.992 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:42.992 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.993 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.993 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.993 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:42.993 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.993 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.993 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.993 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.994 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.994 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.994 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.994 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.994 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.994 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.994 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.994 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.994 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.995 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.995 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.995 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:42.995 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:42.995 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:42.995 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.995 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.995 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:42.996 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.996 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.996 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:42.996 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.996 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.996 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.996 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.996 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.997 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:42.997 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:42.997 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:42.997 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:42.997 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:42.997 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:42.997 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:42.997 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:42.998 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:42.998 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:42.998 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:42.998 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:42.998 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:42.998 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:42.998 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:42.999 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:42.999 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:42.999 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:42.999 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:42.999 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:42.999 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:42.999 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:42.999 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:43.000 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.000 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.000 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.000 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.000 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.000 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.000 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.000 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.001 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.001 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.001 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.001 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.001 INFO:teuthology.orchestra.run.smithi090.stdout: "organization": { 2024-04-24T09:24:43.001 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "organization", 2024-04-24T09:24:43.001 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.002 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.002 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.002 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.002 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.002 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.002 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.002 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.002 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.003 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.003 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.003 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.003 INFO:teuthology.orchestra.run.smithi090.stdout: "proxy": { 2024-04-24T09:24:43.003 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "proxy", 2024-04-24T09:24:43.003 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.003 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.003 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.003 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.004 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.004 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.004 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.004 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.004 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.004 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.004 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.004 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.005 INFO:teuthology.orchestra.run.smithi090.stdout: "url": { 2024-04-24T09:24:43.005 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "url", 2024-04-24T09:24:43.005 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.005 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.005 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.005 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-24T09:24:43.005 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.005 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.005 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.006 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.006 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.006 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.006 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.006 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.006 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.006 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.006 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:43.006 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "test_orchestrator", 2024-04-24T09:24:43.007 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:43.007 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:43.007 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:43.007 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:43.007 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:43.007 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.007 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.007 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.008 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.008 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.008 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.008 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.008 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.008 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.008 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.008 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.009 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.009 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.009 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.009 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.009 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.009 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.009 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.009 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.009 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:43.010 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:43.010 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.010 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.010 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.010 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.010 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.010 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.010 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.011 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.011 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.011 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.011 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.011 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.011 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:43.011 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:43.011 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.012 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.012 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.012 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:43.012 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.012 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.012 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.012 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.012 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.012 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.012 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.013 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.013 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.013 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.013 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.013 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.013 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.013 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.013 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.013 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:43.014 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:43.014 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.014 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.014 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.014 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.014 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.014 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.014 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.015 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.015 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.015 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.015 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.015 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.015 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.015 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.015 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:43.015 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "volumes", 2024-04-24T09:24:43.016 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:43.016 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:43.016 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:43.016 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:43.016 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:43.016 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.016 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.016 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.016 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.016 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.017 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.017 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.017 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.017 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.017 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.017 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.017 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.017 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.017 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.018 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.018 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.018 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.018 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.018 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.018 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:43.018 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:43.018 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.018 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.019 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.019 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.019 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.019 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.019 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.019 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.019 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.019 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.020 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.020 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.020 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:43.020 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:43.020 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.020 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.020 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.020 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:43.020 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.021 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.021 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.021 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.021 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.021 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.021 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.021 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.021 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.022 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.022 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.022 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.022 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.022 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.022 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.022 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:43.022 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:43.022 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.023 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.023 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.023 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.023 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.023 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.023 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.023 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.024 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.024 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.024 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.024 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.024 INFO:teuthology.orchestra.run.smithi090.stdout: "max_concurrent_clones": { 2024-04-24T09:24:43.024 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_concurrent_clones", 2024-04-24T09:24:43.024 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:43.024 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.024 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.025 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "4", 2024-04-24T09:24:43.025 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.025 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.025 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.025 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-24T09:24:43.025 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.025 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.025 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.026 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.026 INFO:teuthology.orchestra.run.smithi090.stdout: "periodic_async_work": { 2024-04-24T09:24:43.026 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "periodic_async_work", 2024-04-24T09:24:43.026 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.026 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.026 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.026 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.026 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.026 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.027 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.027 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Periodically check for async work", 2024-04-24T09:24:43.027 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.027 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.027 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.027 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.027 INFO:teuthology.orchestra.run.smithi090.stdout: "snapshot_clone_delay": { 2024-04-24T09:24:43.027 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snapshot_clone_delay", 2024-04-24T09:24:43.028 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:43.028 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.028 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.028 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-04-24T09:24:43.028 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.028 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.028 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.028 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-24T09:24:43.028 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.029 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.029 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.029 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.029 INFO:teuthology.orchestra.run.smithi090.stdout: "snapshot_clone_no_wait": { 2024-04-24T09:24:43.029 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snapshot_clone_no_wait", 2024-04-24T09:24:43.029 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.029 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.029 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.029 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:43.030 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.030 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.030 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.030 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-24T09:24:43.030 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.030 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.030 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.030 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.031 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.031 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.031 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:43.031 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix", 2024-04-24T09:24:43.031 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:43.031 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:43.031 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:43.031 INFO:teuthology.orchestra.run.smithi090.stdout: "discovery_interval": { 2024-04-24T09:24:43.031 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "discovery_interval", 2024-04-24T09:24:43.032 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-04-24T09:24:43.032 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.032 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.032 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "100", 2024-04-24T09:24:43.032 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.032 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.032 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.032 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.032 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.033 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.033 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.033 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.033 INFO:teuthology.orchestra.run.smithi090.stdout: "identifier": { 2024-04-24T09:24:43.033 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "identifier", 2024-04-24T09:24:43.033 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.033 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.033 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.034 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.034 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.034 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.034 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.034 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.034 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.034 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.035 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.035 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.035 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-04-24T09:24:43.035 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-04-24T09:24:43.035 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:43.035 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.035 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.035 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-04-24T09:24:43.035 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.036 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.036 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.036 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.036 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.036 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.036 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.036 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.036 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:43.037 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:43.037 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.037 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.037 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.037 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.037 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.037 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.037 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.037 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.038 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.038 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.038 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.038 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.038 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.038 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.038 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.038 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.039 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.039 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.039 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.039 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:43.039 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:43.039 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.039 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.039 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.040 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.040 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.040 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.040 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.040 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.040 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.040 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.040 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.041 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.041 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:43.041 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:43.041 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.041 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.041 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.041 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:43.041 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.041 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.042 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.042 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.042 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.042 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.042 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.042 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.042 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.042 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.043 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.043 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.043 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.043 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.043 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.043 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:43.043 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:43.043 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.043 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.044 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.044 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.044 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.044 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.044 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.044 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.044 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.044 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.045 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.045 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.045 INFO:teuthology.orchestra.run.smithi090.stdout: "zabbix_host": { 2024-04-24T09:24:43.045 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix_host", 2024-04-24T09:24:43.045 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.045 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.045 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.045 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.045 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.046 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.046 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.046 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.046 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.046 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.046 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.046 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.046 INFO:teuthology.orchestra.run.smithi090.stdout: "zabbix_port": { 2024-04-24T09:24:43.047 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix_port", 2024-04-24T09:24:43.047 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:43.047 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.047 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.047 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10051", 2024-04-24T09:24:43.047 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.047 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.047 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.047 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.048 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.048 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.048 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.048 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.048 INFO:teuthology.orchestra.run.smithi090.stdout: "zabbix_sender": { 2024-04-24T09:24:43.048 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix_sender", 2024-04-24T09:24:43.048 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.048 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.049 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.049 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-24T09:24:43.049 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.049 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.049 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.049 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.049 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.049 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.049 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.050 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.050 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.050 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.050 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-04-24T09:24:43.050 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.050 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.050 INFO:teuthology.orchestra.run.smithi090.stdout: "modules": [ 2024-04-24T09:24:43.050 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm", 2024-04-24T09:24:43.050 INFO:teuthology.orchestra.run.smithi090.stdout: "dashboard", 2024-04-24T09:24:43.051 INFO:teuthology.orchestra.run.smithi090.stdout: "iostat", 2024-04-24T09:24:43.051 INFO:teuthology.orchestra.run.smithi090.stdout: "nfs", 2024-04-24T09:24:43.051 INFO:teuthology.orchestra.run.smithi090.stdout: "restful" 2024-04-24T09:24:43.051 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.051 INFO:teuthology.orchestra.run.smithi090.stdout: "available_modules": [ 2024-04-24T09:24:43.051 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:43.051 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "alerts", 2024-04-24T09:24:43.052 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:43.052 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:43.052 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:43.052 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-04-24T09:24:43.052 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-04-24T09:24:43.052 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:43.052 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.052 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.052 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-04-24T09:24:43.053 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.053 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.053 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.053 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How frequently to reexamine health status", 2024-04-24T09:24:43.053 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.053 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.053 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.053 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.054 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:43.054 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:43.054 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.054 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.054 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.054 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.054 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.054 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.054 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.055 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.055 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.055 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.055 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.055 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.055 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.055 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.055 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.056 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.056 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.056 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.056 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.056 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:43.056 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:43.056 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.056 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.056 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.057 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.057 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.057 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.057 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.057 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.057 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.057 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.057 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.058 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.058 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:43.058 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:43.058 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.058 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.058 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.058 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:43.058 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.058 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.059 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.059 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.059 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.059 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.059 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.059 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.059 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.059 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.060 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.060 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.060 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.060 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.060 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.060 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:43.060 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:43.060 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.061 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.061 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.061 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.061 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.061 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.061 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.061 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.061 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.061 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.062 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.062 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.062 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_destination": { 2024-04-24T09:24:43.062 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_destination", 2024-04-24T09:24:43.062 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.062 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.062 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.062 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.063 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.063 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.063 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.063 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Email address to send alerts to", 2024-04-24T09:24:43.063 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.063 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.063 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.063 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.064 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_from_name": { 2024-04-24T09:24:43.064 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_from_name", 2024-04-24T09:24:43.064 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.064 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.064 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.064 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "Ceph", 2024-04-24T09:24:43.064 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.064 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.064 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.065 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Email From: name", 2024-04-24T09:24:43.065 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.065 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.065 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.065 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.065 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_host": { 2024-04-24T09:24:43.065 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_host", 2024-04-24T09:24:43.066 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.066 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.066 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.066 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.066 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.066 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.066 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.066 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SMTP server", 2024-04-24T09:24:43.067 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.067 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.067 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.067 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.067 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_password": { 2024-04-24T09:24:43.067 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_password", 2024-04-24T09:24:43.067 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.067 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.067 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.068 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.068 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.068 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.068 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.068 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Password to authenticate with", 2024-04-24T09:24:43.068 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.068 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.068 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.068 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.069 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_port": { 2024-04-24T09:24:43.069 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_port", 2024-04-24T09:24:43.069 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:43.069 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.069 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.069 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "465", 2024-04-24T09:24:43.069 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.069 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.070 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.070 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SMTP port", 2024-04-24T09:24:43.070 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.070 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.070 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.070 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.070 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_sender": { 2024-04-24T09:24:43.070 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_sender", 2024-04-24T09:24:43.071 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.071 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.071 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.071 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.071 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.071 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.071 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.071 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SMTP envelope sender", 2024-04-24T09:24:43.071 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.072 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.072 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.072 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.072 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_ssl": { 2024-04-24T09:24:43.072 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_ssl", 2024-04-24T09:24:43.072 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.072 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.072 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.072 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:43.073 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.073 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.073 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.073 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-24T09:24:43.073 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.073 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.073 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.073 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.073 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_user": { 2024-04-24T09:24:43.073 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_user", 2024-04-24T09:24:43.074 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.074 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.074 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.074 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.074 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.074 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.074 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.074 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "User to authenticate as", 2024-04-24T09:24:43.075 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.075 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.075 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.075 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.075 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.075 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.075 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:43.075 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "balancer", 2024-04-24T09:24:43.075 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:43.075 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:43.076 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:43.076 INFO:teuthology.orchestra.run.smithi090.stdout: "active": { 2024-04-24T09:24:43.076 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "active", 2024-04-24T09:24:43.076 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.076 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.076 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.076 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:43.076 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.076 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.077 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.077 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "automatically balance PGs across cluster", 2024-04-24T09:24:43.077 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.077 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.077 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.077 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.077 INFO:teuthology.orchestra.run.smithi090.stdout: "begin_time": { 2024-04-24T09:24:43.077 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "begin_time", 2024-04-24T09:24:43.077 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.078 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.078 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.078 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0000", 2024-04-24T09:24:43.078 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.078 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.078 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.078 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "beginning time of day to automatically balance", 2024-04-24T09:24:43.078 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T09:24:43.078 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.079 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.079 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.079 INFO:teuthology.orchestra.run.smithi090.stdout: "begin_weekday": { 2024-04-24T09:24:43.079 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "begin_weekday", 2024-04-24T09:24:43.079 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-04-24T09:24:43.079 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.079 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.079 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-04-24T09:24:43.079 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "0", 2024-04-24T09:24:43.080 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "6", 2024-04-24T09:24:43.080 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.080 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-24T09:24:43.080 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T09:24:43.080 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.080 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.080 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.080 INFO:teuthology.orchestra.run.smithi090.stdout: "crush_compat_max_iterations": { 2024-04-24T09:24:43.080 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crush_compat_max_iterations", 2024-04-24T09:24:43.081 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-04-24T09:24:43.081 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.081 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.081 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "25", 2024-04-24T09:24:43.081 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-04-24T09:24:43.081 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "250", 2024-04-24T09:24:43.081 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.081 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-24T09:24:43.081 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.082 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.082 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.082 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.082 INFO:teuthology.orchestra.run.smithi090.stdout: "crush_compat_metrics": { 2024-04-24T09:24:43.082 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crush_compat_metrics", 2024-04-24T09:24:43.082 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.082 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.083 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.083 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "pgs,objects,bytes", 2024-04-24T09:24:43.083 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.083 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.083 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.083 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-24T09:24:43.083 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-24T09:24:43.083 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.083 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.084 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.084 INFO:teuthology.orchestra.run.smithi090.stdout: "crush_compat_step": { 2024-04-24T09:24:43.084 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crush_compat_step", 2024-04-24T09:24:43.084 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-04-24T09:24:43.084 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.084 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.084 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0.5", 2024-04-24T09:24:43.084 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "0.001", 2024-04-24T09:24:43.085 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "0.999", 2024-04-24T09:24:43.085 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.085 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "aggressiveness of optimization", 2024-04-24T09:24:43.085 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-24T09:24:43.085 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.085 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.085 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.085 INFO:teuthology.orchestra.run.smithi090.stdout: "end_time": { 2024-04-24T09:24:43.085 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "end_time", 2024-04-24T09:24:43.086 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.086 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.086 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.086 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "2359", 2024-04-24T09:24:43.086 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.086 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.086 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.086 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "ending time of day to automatically balance", 2024-04-24T09:24:43.086 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T09:24:43.087 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.087 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.087 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.087 INFO:teuthology.orchestra.run.smithi090.stdout: "end_weekday": { 2024-04-24T09:24:43.087 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "end_weekday", 2024-04-24T09:24:43.087 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-04-24T09:24:43.087 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.087 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.088 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-04-24T09:24:43.088 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "0", 2024-04-24T09:24:43.088 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "6", 2024-04-24T09:24:43.088 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.088 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-24T09:24:43.088 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T09:24:43.088 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.088 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.088 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.089 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:43.089 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:43.089 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.089 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.089 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.089 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.089 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.089 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.089 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.090 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.090 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.090 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.090 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.090 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.090 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.090 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.090 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.091 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.091 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.091 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.091 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.091 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:43.091 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:43.091 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.091 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.092 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.092 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.092 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.092 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.092 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.092 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.092 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.092 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.092 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.093 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.093 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:43.093 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:43.093 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.093 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.093 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.093 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:43.093 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.094 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.094 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.094 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.094 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.094 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.094 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.094 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.094 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.094 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.095 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.095 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.095 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.095 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.095 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.095 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:43.095 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:43.095 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.095 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.096 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.096 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.096 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.096 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.096 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.096 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.096 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.096 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.096 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.097 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.097 INFO:teuthology.orchestra.run.smithi090.stdout: "min_score": { 2024-04-24T09:24:43.097 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "min_score", 2024-04-24T09:24:43.097 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-04-24T09:24:43.097 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.097 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.097 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-04-24T09:24:43.097 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.097 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.098 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.098 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-24T09:24:43.098 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.098 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.098 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.098 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.098 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": { 2024-04-24T09:24:43.098 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mode", 2024-04-24T09:24:43.098 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.099 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.099 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.099 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "upmap", 2024-04-24T09:24:43.099 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.099 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.099 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.099 INFO:teuthology.orchestra.run.smithi090.stdout: "crush-compat", 2024-04-24T09:24:43.099 INFO:teuthology.orchestra.run.smithi090.stdout: "none", 2024-04-24T09:24:43.100 INFO:teuthology.orchestra.run.smithi090.stdout: "read", 2024-04-24T09:24:43.100 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap", 2024-04-24T09:24:43.100 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap-read" 2024-04-24T09:24:43.100 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.100 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Balancer mode", 2024-04-24T09:24:43.100 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.100 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.100 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.101 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.101 INFO:teuthology.orchestra.run.smithi090.stdout: "pool_ids": { 2024-04-24T09:24:43.101 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pool_ids", 2024-04-24T09:24:43.101 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.101 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.101 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.101 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.101 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.101 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.102 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.102 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-24T09:24:43.102 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.102 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.102 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.102 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.102 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-04-24T09:24:43.102 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-04-24T09:24:43.102 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:43.103 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.103 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.103 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-04-24T09:24:43.103 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.103 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.103 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.103 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-24T09:24:43.103 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.103 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.104 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.104 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.104 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap_max_deviation": { 2024-04-24T09:24:43.104 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "upmap_max_deviation", 2024-04-24T09:24:43.104 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:43.104 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.104 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.104 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-04-24T09:24:43.104 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-04-24T09:24:43.104 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.105 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.105 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-24T09:24:43.105 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-24T09:24:43.105 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.105 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.105 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.105 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap_max_optimizations": { 2024-04-24T09:24:43.105 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "upmap_max_optimizations", 2024-04-24T09:24:43.105 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-04-24T09:24:43.106 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.106 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.106 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-04-24T09:24:43.106 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.106 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.106 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.106 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-24T09:24:43.106 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.106 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.107 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.107 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.107 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.107 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.107 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:43.107 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cephadm", 2024-04-24T09:24:43.107 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:43.107 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:43.107 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:43.107 INFO:teuthology.orchestra.run.smithi090.stdout: "agent_down_multiplier": { 2024-04-24T09:24:43.108 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "agent_down_multiplier", 2024-04-24T09:24:43.108 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-04-24T09:24:43.108 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.108 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.108 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "3.0", 2024-04-24T09:24:43.108 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.108 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.108 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.108 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-24T09:24:43.109 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.109 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.109 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.109 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.109 INFO:teuthology.orchestra.run.smithi090.stdout: "agent_refresh_rate": { 2024-04-24T09:24:43.109 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "agent_refresh_rate", 2024-04-24T09:24:43.109 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:43.109 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.109 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.109 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "20", 2024-04-24T09:24:43.110 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.110 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.110 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.110 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-24T09:24:43.110 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.110 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.110 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.110 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.110 INFO:teuthology.orchestra.run.smithi090.stdout: "agent_starting_port": { 2024-04-24T09:24:43.111 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "agent_starting_port", 2024-04-24T09:24:43.111 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:43.111 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.111 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.111 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "4721", 2024-04-24T09:24:43.111 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.111 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.111 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.111 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-24T09:24:43.112 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.112 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.112 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.112 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.112 INFO:teuthology.orchestra.run.smithi090.stdout: "allow_ptrace": { 2024-04-24T09:24:43.112 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "allow_ptrace", 2024-04-24T09:24:43.112 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.112 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.112 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.113 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.113 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.113 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.113 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.113 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-24T09:24:43.113 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-24T09:24:43.113 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.113 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.113 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.113 INFO:teuthology.orchestra.run.smithi090.stdout: "autotune_interval": { 2024-04-24T09:24:43.114 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "autotune_interval", 2024-04-24T09:24:43.114 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:43.114 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.114 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.114 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-04-24T09:24:43.114 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.114 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.114 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.115 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-24T09:24:43.115 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.115 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.115 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.115 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.115 INFO:teuthology.orchestra.run.smithi090.stdout: "autotune_memory_target_ratio": { 2024-04-24T09:24:43.115 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "autotune_memory_target_ratio", 2024-04-24T09:24:43.115 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-04-24T09:24:43.115 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.116 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.116 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0.7", 2024-04-24T09:24:43.116 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.116 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.116 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.116 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-24T09:24:43.116 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.116 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.116 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.116 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.117 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm_log_destination": { 2024-04-24T09:24:43.117 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cephadm_log_destination", 2024-04-24T09:24:43.117 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.117 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.117 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.117 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.117 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.117 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.118 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.118 INFO:teuthology.orchestra.run.smithi090.stdout: "file", 2024-04-24T09:24:43.118 INFO:teuthology.orchestra.run.smithi090.stdout: "file,syslog", 2024-04-24T09:24:43.118 INFO:teuthology.orchestra.run.smithi090.stdout: "syslog" 2024-04-24T09:24:43.118 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.118 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-24T09:24:43.118 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.118 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.118 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.118 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.119 INFO:teuthology.orchestra.run.smithi090.stdout: "cgroups_split": { 2024-04-24T09:24:43.119 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cgroups_split", 2024-04-24T09:24:43.119 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.119 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.119 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.119 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:43.119 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.119 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.119 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.120 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-24T09:24:43.120 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.120 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.120 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.120 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.120 INFO:teuthology.orchestra.run.smithi090.stdout: "config_checks_enabled": { 2024-04-24T09:24:43.120 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "config_checks_enabled", 2024-04-24T09:24:43.120 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.121 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.121 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.121 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.121 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.121 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.121 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.121 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-24T09:24:43.121 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.121 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.122 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.122 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.122 INFO:teuthology.orchestra.run.smithi090.stdout: "config_dashboard": { 2024-04-24T09:24:43.122 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "config_dashboard", 2024-04-24T09:24:43.122 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.122 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.122 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.122 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:43.122 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.123 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.123 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.123 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-24T09:24:43.123 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.123 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.123 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.123 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.123 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_alertmanager": { 2024-04-24T09:24:43.124 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_alertmanager", 2024-04-24T09:24:43.124 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.124 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.124 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.124 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-24T09:24:43.124 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.124 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.124 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.124 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-04-24T09:24:43.125 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.125 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.125 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.125 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.125 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_base": { 2024-04-24T09:24:43.125 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_base", 2024-04-24T09:24:43.125 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.125 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.125 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.126 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-24T09:24:43.126 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.126 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.126 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.126 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Container image name, without the tag", 2024-04-24T09:24:43.126 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.126 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.126 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.127 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.127 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_elasticsearch": { 2024-04-24T09:24:43.127 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_elasticsearch", 2024-04-24T09:24:43.127 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.127 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.127 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.127 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-24T09:24:43.127 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.127 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.128 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.128 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "elasticsearch container image", 2024-04-24T09:24:43.128 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.128 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.128 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.128 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.128 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_grafana": { 2024-04-24T09:24:43.128 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_grafana", 2024-04-24T09:24:43.129 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.129 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.129 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.129 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-24T09:24:43.129 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.129 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.129 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.129 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-04-24T09:24:43.129 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.130 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.130 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.130 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.130 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_haproxy": { 2024-04-24T09:24:43.130 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_haproxy", 2024-04-24T09:24:43.130 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.130 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.130 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.131 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-24T09:24:43.131 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.131 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.131 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.131 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "HAproxy container image", 2024-04-24T09:24:43.131 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.131 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.131 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.131 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.132 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_jaeger_agent": { 2024-04-24T09:24:43.132 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_jaeger_agent", 2024-04-24T09:24:43.132 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.132 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.132 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.132 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-24T09:24:43.132 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.132 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.133 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.133 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Jaeger agent container image", 2024-04-24T09:24:43.133 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.133 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.133 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.133 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.133 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_jaeger_collector": { 2024-04-24T09:24:43.133 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_jaeger_collector", 2024-04-24T09:24:43.133 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.134 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.134 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.134 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-24T09:24:43.134 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.134 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.134 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.134 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Jaeger collector container image", 2024-04-24T09:24:43.135 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.135 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.135 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.135 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.135 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_jaeger_query": { 2024-04-24T09:24:43.135 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_jaeger_query", 2024-04-24T09:24:43.135 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.135 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.136 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.136 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-24T09:24:43.136 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.136 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.136 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.136 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Jaeger query container image", 2024-04-24T09:24:43.136 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.136 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.137 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.137 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.137 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_keepalived": { 2024-04-24T09:24:43.137 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_keepalived", 2024-04-24T09:24:43.137 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.137 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.137 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.137 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-24T09:24:43.137 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.138 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.138 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.138 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Keepalived container image", 2024-04-24T09:24:43.138 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.138 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.138 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.138 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.138 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_loki": { 2024-04-24T09:24:43.139 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_loki", 2024-04-24T09:24:43.139 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.139 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.139 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.139 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-24T09:24:43.139 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.139 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.140 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.140 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Loki container image", 2024-04-24T09:24:43.140 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.140 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.140 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.140 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.140 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_node_exporter": { 2024-04-24T09:24:43.140 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_node_exporter", 2024-04-24T09:24:43.140 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.141 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.141 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.141 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-24T09:24:43.141 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.141 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.141 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.141 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-04-24T09:24:43.141 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.141 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.142 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.142 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.142 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_nvmeof": { 2024-04-24T09:24:43.142 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_nvmeof", 2024-04-24T09:24:43.142 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.142 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.142 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.142 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-24T09:24:43.142 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.143 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.143 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.143 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Nvme-of container image", 2024-04-24T09:24:43.143 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.143 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.143 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.143 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.143 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_prometheus": { 2024-04-24T09:24:43.144 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_prometheus", 2024-04-24T09:24:43.144 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.144 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.144 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.144 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-24T09:24:43.144 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.144 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.144 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.144 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-04-24T09:24:43.144 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.145 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.145 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.145 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.145 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_promtail": { 2024-04-24T09:24:43.145 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_promtail", 2024-04-24T09:24:43.145 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.145 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.145 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.145 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-24T09:24:43.146 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.146 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.146 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.146 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Promtail container image", 2024-04-24T09:24:43.146 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.146 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.146 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.146 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.147 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_samba": { 2024-04-24T09:24:43.147 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_samba", 2024-04-24T09:24:43.147 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.147 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.147 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.147 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-24T09:24:43.147 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.147 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.147 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.147 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Samba/SMB container image", 2024-04-24T09:24:43.148 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.148 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.148 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.148 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.148 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_snmp_gateway": { 2024-04-24T09:24:43.148 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_snmp_gateway", 2024-04-24T09:24:43.148 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.148 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.149 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.149 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-24T09:24:43.149 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.149 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.149 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.149 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SNMP Gateway container image", 2024-04-24T09:24:43.149 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.149 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.149 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.150 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.150 INFO:teuthology.orchestra.run.smithi090.stdout: "container_init": { 2024-04-24T09:24:43.150 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_init", 2024-04-24T09:24:43.150 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.150 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.150 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.150 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:43.150 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.151 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.151 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.151 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Run podman/docker with `--init`", 2024-04-24T09:24:43.151 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.151 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.151 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.151 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.151 INFO:teuthology.orchestra.run.smithi090.stdout: "daemon_cache_timeout": { 2024-04-24T09:24:43.151 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "daemon_cache_timeout", 2024-04-24T09:24:43.152 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:43.152 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.152 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.152 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-04-24T09:24:43.152 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.152 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.152 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.152 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-24T09:24:43.153 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.153 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.153 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.153 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.153 INFO:teuthology.orchestra.run.smithi090.stdout: "default_cephadm_command_timeout": { 2024-04-24T09:24:43.153 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "default_cephadm_command_timeout", 2024-04-24T09:24:43.153 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:43.153 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.153 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.154 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "900", 2024-04-24T09:24:43.154 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.154 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.154 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.154 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-24T09:24:43.154 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.154 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.154 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.154 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.155 INFO:teuthology.orchestra.run.smithi090.stdout: "default_registry": { 2024-04-24T09:24:43.155 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "default_registry", 2024-04-24T09:24:43.155 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.155 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.155 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.155 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io", 2024-04-24T09:24:43.155 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.155 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.156 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.156 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-24T09:24:43.156 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.156 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.156 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.156 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.156 INFO:teuthology.orchestra.run.smithi090.stdout: "device_cache_timeout": { 2024-04-24T09:24:43.156 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "device_cache_timeout", 2024-04-24T09:24:43.156 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:43.157 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.157 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.157 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "1800", 2024-04-24T09:24:43.157 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.157 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.157 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.157 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "seconds to cache device inventory", 2024-04-24T09:24:43.157 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.158 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.158 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.158 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.158 INFO:teuthology.orchestra.run.smithi090.stdout: "device_enhanced_scan": { 2024-04-24T09:24:43.158 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "device_enhanced_scan", 2024-04-24T09:24:43.158 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.158 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.158 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.159 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.159 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.159 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.159 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.159 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-24T09:24:43.159 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.159 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.159 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.159 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.160 INFO:teuthology.orchestra.run.smithi090.stdout: "facts_cache_timeout": { 2024-04-24T09:24:43.160 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "facts_cache_timeout", 2024-04-24T09:24:43.160 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:43.160 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.160 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.160 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-04-24T09:24:43.160 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.160 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.160 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.161 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "seconds to cache host facts data", 2024-04-24T09:24:43.161 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.161 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.161 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.161 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.161 INFO:teuthology.orchestra.run.smithi090.stdout: "grafana_dashboards_path": { 2024-04-24T09:24:43.161 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "grafana_dashboards_path", 2024-04-24T09:24:43.161 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.161 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.162 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.162 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-24T09:24:43.162 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.162 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.162 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.162 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-24T09:24:43.162 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.162 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.163 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.163 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.163 INFO:teuthology.orchestra.run.smithi090.stdout: "host_check_interval": { 2024-04-24T09:24:43.163 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "host_check_interval", 2024-04-24T09:24:43.163 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:43.163 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.163 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.163 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-04-24T09:24:43.164 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.164 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.164 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.164 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to perform a host check", 2024-04-24T09:24:43.164 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.164 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.164 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.164 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.164 INFO:teuthology.orchestra.run.smithi090.stdout: "hw_monitoring": { 2024-04-24T09:24:43.165 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "hw_monitoring", 2024-04-24T09:24:43.165 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.165 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.165 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.165 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.165 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.165 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.165 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.165 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-24T09:24:43.166 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.166 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.166 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.166 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.166 INFO:teuthology.orchestra.run.smithi090.stdout: "inventory_list_all": { 2024-04-24T09:24:43.166 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "inventory_list_all", 2024-04-24T09:24:43.166 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.166 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.167 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.167 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.167 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.167 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.167 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.167 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-24T09:24:43.167 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.168 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.168 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.168 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.168 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:43.168 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:43.168 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.168 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.168 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.169 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.169 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.169 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.169 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.169 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.169 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.169 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.169 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.170 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.170 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.170 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.170 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.170 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.170 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.170 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.170 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.170 INFO:teuthology.orchestra.run.smithi090.stdout: "log_refresh_metadata": { 2024-04-24T09:24:43.171 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_refresh_metadata", 2024-04-24T09:24:43.171 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.171 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.171 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.171 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.171 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.171 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.171 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.172 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-24T09:24:43.172 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.172 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.172 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.172 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.172 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:43.172 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:43.172 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.172 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.173 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.173 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:43.173 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.173 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.173 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.173 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-24T09:24:43.173 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.173 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.174 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.174 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.174 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:43.174 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:43.174 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.174 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.174 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.174 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:43.175 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.175 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.175 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.175 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.175 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.175 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.175 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.175 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.176 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.176 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.176 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.176 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.176 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.176 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.176 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.176 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:43.176 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:43.177 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.177 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.177 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.177 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.177 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.177 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.177 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.177 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.178 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.178 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.178 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.178 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.178 INFO:teuthology.orchestra.run.smithi090.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-24T09:24:43.178 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-24T09:24:43.178 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.178 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.178 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.179 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.179 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.179 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.179 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.179 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-24T09:24:43.179 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.179 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.179 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.179 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.179 INFO:teuthology.orchestra.run.smithi090.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-24T09:24:43.180 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-24T09:24:43.180 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.180 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.180 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.180 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "*", 2024-04-24T09:24:43.180 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.180 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.180 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.181 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-24T09:24:43.181 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.181 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.181 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.181 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.181 INFO:teuthology.orchestra.run.smithi090.stdout: "max_count_per_host": { 2024-04-24T09:24:43.181 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_count_per_host", 2024-04-24T09:24:43.181 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:43.182 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.182 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.182 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-04-24T09:24:43.182 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.182 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.182 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.182 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "max number of daemons per service per host", 2024-04-24T09:24:43.182 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.182 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.183 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.183 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.183 INFO:teuthology.orchestra.run.smithi090.stdout: "max_osd_draining_count": { 2024-04-24T09:24:43.183 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_osd_draining_count", 2024-04-24T09:24:43.183 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:43.183 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.183 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.183 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-04-24T09:24:43.184 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.184 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.184 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.184 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-24T09:24:43.184 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.184 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.184 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.184 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.184 INFO:teuthology.orchestra.run.smithi090.stdout: "migration_current": { 2024-04-24T09:24:43.185 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "migration_current", 2024-04-24T09:24:43.185 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:43.185 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.185 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.185 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.185 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.185 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.185 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.186 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "internal - do not modify", 2024-04-24T09:24:43.186 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.186 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.186 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.186 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.186 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": { 2024-04-24T09:24:43.186 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mode", 2024-04-24T09:24:43.186 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.187 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.187 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.187 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "root", 2024-04-24T09:24:43.187 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.187 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.187 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.187 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm-package", 2024-04-24T09:24:43.187 INFO:teuthology.orchestra.run.smithi090.stdout: "root" 2024-04-24T09:24:43.188 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.188 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "mode for remote execution of cephadm", 2024-04-24T09:24:43.188 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.188 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.188 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.188 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.188 INFO:teuthology.orchestra.run.smithi090.stdout: "oob_default_addr": { 2024-04-24T09:24:43.188 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "oob_default_addr", 2024-04-24T09:24:43.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.668+0000 7feab7e00640 1 -- 172.21.15.90:0/4109344076 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fea9c0a3bb0 msgr2=0x7fea9c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:24:43.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.668+0000 7feab7e00640 1 --2- 172.21.15.90:0/4109344076 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fea9c0a3bb0 0x7fea9c0a6070 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7feaa80096c0 tx=0x7feaa8009260 comp rx=0 tx=0).stop 2024-04-24T09:24:43.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.668+0000 7feab7e00640 1 -- 172.21.15.90:0/4109344076 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feab8107ad0 msgr2=0x7feab8101a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:24:43.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.668+0000 7feab7e00640 1 --2- 172.21.15.90:0/4109344076 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feab8107ad0 0x7feab8101a10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7feab0009b30 tx=0x7feab00305d0 comp rx=0 tx=0).stop 2024-04-24T09:24:43.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.668+0000 7feab7e00640 1 -- 172.21.15.90:0/4109344076 shutdown_connections 2024-04-24T09:24:43.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.668+0000 7feab7e00640 1 --2- 172.21.15.90:0/4109344076 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fea9c0a3bb0 0x7fea9c0a6070 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:24:43.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.668+0000 7feab7e00640 1 --2- 172.21.15.90:0/4109344076 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feab8107ad0 0x7feab8101a10 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:24:43.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.668+0000 7feab7e00640 1 --2- 172.21.15.90:0/4109344076 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feab8100540 0x7feab81014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:24:43.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.668+0000 7feab7e00640 1 --2- 172.21.15.90:0/4109344076 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feab80ffbe0 0x7feab8102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:24:43.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.668+0000 7feab7e00640 1 -- 172.21.15.90:0/4109344076 >> 172.21.15.90:0/4109344076 conn(0x7feab80fb630 msgr2=0x7feab8103520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:24:43.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.668+0000 7feab7e00640 1 -- 172.21.15.90:0/4109344076 shutdown_connections 2024-04-24T09:24:43.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:24:41.668+0000 7feab7e00640 1 -- 172.21.15.90:0/4109344076 wait complete. 2024-04-24T09:24:43.194 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:24:42 smithi114 ceph-mon[32501]: pgmap v490: 97 pgs: 97 active+clean; 68 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 237 KiB/s wr, 60 op/s 2024-04-24T09:24:43.194 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:24:42 smithi149 ceph-mon[32592]: pgmap v490: 97 pgs: 97 active+clean; 68 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 237 KiB/s wr, 60 op/s 2024-04-24T09:24:43.195 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.195 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.195 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.195 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "169.254.1.1", 2024-04-24T09:24:43.196 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.196 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.196 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.196 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-24T09:24:43.196 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.196 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.196 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.196 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.196 INFO:teuthology.orchestra.run.smithi090.stdout: "prometheus_alerts_path": { 2024-04-24T09:24:43.197 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "prometheus_alerts_path", 2024-04-24T09:24:43.197 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.197 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.197 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.197 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-24T09:24:43.197 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.197 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.197 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.198 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-24T09:24:43.198 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.198 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.198 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.198 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.198 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_insecure": { 2024-04-24T09:24:43.198 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_insecure", 2024-04-24T09:24:43.198 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.199 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.199 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.199 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.199 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.199 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.199 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.199 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-24T09:24:43.199 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.200 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.200 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.200 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.200 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_password": { 2024-04-24T09:24:43.200 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_password", 2024-04-24T09:24:43.200 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.200 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.200 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.201 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.201 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.201 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.201 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.201 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-24T09:24:43.201 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.201 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.201 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.202 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.202 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_url": { 2024-04-24T09:24:43.202 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_url", 2024-04-24T09:24:43.202 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.202 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.202 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.202 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.202 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.202 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.203 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.203 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-24T09:24:43.203 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.203 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.203 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.203 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.203 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_username": { 2024-04-24T09:24:43.203 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_username", 2024-04-24T09:24:43.204 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.204 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.204 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.204 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.204 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.204 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.204 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.204 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-24T09:24:43.204 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.205 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.205 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.205 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.205 INFO:teuthology.orchestra.run.smithi090.stdout: "secure_monitoring_stack": { 2024-04-24T09:24:43.205 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "secure_monitoring_stack", 2024-04-24T09:24:43.205 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.205 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.205 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.206 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.206 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.206 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.206 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.206 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-24T09:24:43.206 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.206 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.207 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.207 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.207 INFO:teuthology.orchestra.run.smithi090.stdout: "service_discovery_port": { 2024-04-24T09:24:43.207 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "service_discovery_port", 2024-04-24T09:24:43.207 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:43.207 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.207 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.207 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8765", 2024-04-24T09:24:43.207 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.208 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.208 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.208 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "cephadm service discovery port", 2024-04-24T09:24:43.208 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.208 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.208 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.208 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.208 INFO:teuthology.orchestra.run.smithi090.stdout: "ssh_config_file": { 2024-04-24T09:24:43.208 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssh_config_file", 2024-04-24T09:24:43.209 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.209 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.209 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.209 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.209 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.209 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.209 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.209 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-24T09:24:43.210 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.210 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.210 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.210 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.210 INFO:teuthology.orchestra.run.smithi090.stdout: "use_agent": { 2024-04-24T09:24:43.210 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "use_agent", 2024-04-24T09:24:43.210 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.210 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.211 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.211 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.211 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.211 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.211 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.211 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-24T09:24:43.211 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.211 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.211 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.212 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.212 INFO:teuthology.orchestra.run.smithi090.stdout: "use_repo_digest": { 2024-04-24T09:24:43.212 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "use_repo_digest", 2024-04-24T09:24:43.212 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.212 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.212 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.212 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:43.212 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.213 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.213 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.213 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-24T09:24:43.213 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.213 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.213 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.213 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.213 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_on_failed_host_check": { 2024-04-24T09:24:43.213 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_on_failed_host_check", 2024-04-24T09:24:43.213 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.214 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.214 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.214 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:43.214 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.214 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.214 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.214 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise a health warning if the host check fails", 2024-04-24T09:24:43.214 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.214 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.215 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.215 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.215 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_on_stray_daemons": { 2024-04-24T09:24:43.215 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_on_stray_daemons", 2024-04-24T09:24:43.215 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.215 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.215 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.215 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:43.216 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.216 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.216 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.216 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-24T09:24:43.216 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.216 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.216 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.216 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.216 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_on_stray_hosts": { 2024-04-24T09:24:43.217 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_on_stray_hosts", 2024-04-24T09:24:43.217 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.217 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.217 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.217 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:43.217 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.217 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.217 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.218 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-24T09:24:43.218 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.218 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.218 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.218 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.218 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.218 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.218 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:43.219 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crash", 2024-04-24T09:24:43.219 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:43.219 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:43.219 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:43.219 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:43.219 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:43.219 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.219 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.220 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.220 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.220 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.220 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.220 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.220 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.220 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.221 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.221 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.221 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.221 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.221 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.221 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.221 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.221 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.222 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.222 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.222 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:43.222 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:43.222 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.222 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.222 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.222 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.222 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.223 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.223 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.223 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.223 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.223 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.223 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.223 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.223 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:43.223 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:43.224 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.224 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.224 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.224 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:43.224 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.224 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.224 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.224 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.225 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.225 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.225 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.225 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.225 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.225 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.225 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.225 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.225 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.226 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.226 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.226 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:43.226 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:43.226 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.226 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.226 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.226 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.227 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.227 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.227 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.227 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.227 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.227 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.227 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.227 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.227 INFO:teuthology.orchestra.run.smithi090.stdout: "retain_interval": { 2024-04-24T09:24:43.228 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "retain_interval", 2024-04-24T09:24:43.228 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:43.228 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.228 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.228 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "31536000", 2024-04-24T09:24:43.228 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.228 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.228 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.229 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-24T09:24:43.229 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.229 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.229 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.229 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.229 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_recent_interval": { 2024-04-24T09:24:43.229 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_recent_interval", 2024-04-24T09:24:43.229 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:43.229 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.230 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.230 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "1209600", 2024-04-24T09:24:43.230 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.230 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.230 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.230 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-24T09:24:43.230 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.230 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.231 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.231 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.231 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.231 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.231 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:43.231 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "dashboard", 2024-04-24T09:24:43.231 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:43.231 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:43.232 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:43.232 INFO:teuthology.orchestra.run.smithi090.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-24T09:24:43.232 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-24T09:24:43.232 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:43.232 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.232 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.232 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-04-24T09:24:43.232 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.232 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.233 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.233 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.233 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.233 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.233 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.233 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.233 INFO:teuthology.orchestra.run.smithi090.stdout: "ALERTMANAGER_API_HOST": { 2024-04-24T09:24:43.233 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-24T09:24:43.234 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.234 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.234 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.234 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.234 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.234 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.234 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.234 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.235 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.235 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.235 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.235 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.235 INFO:teuthology.orchestra.run.smithi090.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-24T09:24:43.235 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-24T09:24:43.235 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.235 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.236 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.236 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:43.236 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.236 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.236 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.236 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.236 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.236 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.237 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.237 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.237 INFO:teuthology.orchestra.run.smithi090.stdout: "AUDIT_API_ENABLED": { 2024-04-24T09:24:43.237 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "AUDIT_API_ENABLED", 2024-04-24T09:24:43.237 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.237 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.237 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.237 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.238 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.238 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.238 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.238 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.238 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.238 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.238 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.238 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.239 INFO:teuthology.orchestra.run.smithi090.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-24T09:24:43.239 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-24T09:24:43.239 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.239 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.239 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.239 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:43.239 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.239 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.240 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.240 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.240 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.240 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.240 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.240 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.240 INFO:teuthology.orchestra.run.smithi090.stdout: "ENABLE_BROWSABLE_API": { 2024-04-24T09:24:43.240 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-24T09:24:43.240 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.241 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.241 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.241 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:43.241 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.241 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.241 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.241 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.241 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.241 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.242 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.242 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.242 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-24T09:24:43.242 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-24T09:24:43.242 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.242 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.242 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.242 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:43.243 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.243 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.243 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.243 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.243 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.243 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.243 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.243 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.243 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-24T09:24:43.244 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-24T09:24:43.244 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.244 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.244 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.244 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:43.244 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.244 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.244 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.244 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.245 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.245 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.245 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.245 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.245 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-24T09:24:43.245 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-24T09:24:43.245 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.245 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.245 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.245 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:43.246 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.246 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.246 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.246 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.246 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.246 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.246 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.246 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.247 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-24T09:24:43.247 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-24T09:24:43.247 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.247 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.247 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.247 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:43.247 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.247 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.247 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.248 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.248 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.248 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.248 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.248 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.248 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-24T09:24:43.248 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-24T09:24:43.249 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.249 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.249 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.249 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:43.249 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.249 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.249 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.249 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.250 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.250 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.250 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.250 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.250 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-24T09:24:43.250 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-24T09:24:43.250 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.250 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.251 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.251 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:43.251 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.251 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.251 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.251 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.251 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.251 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.251 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.251 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.252 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-24T09:24:43.252 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-24T09:24:43.252 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.252 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.252 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.252 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:43.252 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.252 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.252 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.253 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.253 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.253 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.253 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.253 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.253 INFO:teuthology.orchestra.run.smithi090.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-24T09:24:43.253 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-24T09:24:43.253 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.253 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.254 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.254 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.254 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.254 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.254 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.254 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.254 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.254 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.254 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.255 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.255 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_PASSWORD": { 2024-04-24T09:24:43.255 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-24T09:24:43.255 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.255 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.255 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.255 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "admin", 2024-04-24T09:24:43.255 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.256 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.256 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.256 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.256 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.256 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.256 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.256 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.256 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-24T09:24:43.256 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-24T09:24:43.257 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.257 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.257 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.257 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:43.257 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.257 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.257 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.257 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.257 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.258 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.258 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.258 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.258 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_URL": { 2024-04-24T09:24:43.258 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_URL", 2024-04-24T09:24:43.258 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.258 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.258 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.259 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.259 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.259 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.259 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.259 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.259 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.259 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.259 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.259 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.260 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_USERNAME": { 2024-04-24T09:24:43.260 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-24T09:24:43.260 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.260 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.260 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.260 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "admin", 2024-04-24T09:24:43.260 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.260 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.261 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.261 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.261 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.261 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.261 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.261 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.261 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-24T09:24:43.261 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-24T09:24:43.262 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.262 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.262 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.262 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.262 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.262 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.262 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.262 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.263 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.263 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.263 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.263 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.263 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-24T09:24:43.263 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-24T09:24:43.263 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.263 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.263 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.264 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.264 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.264 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.264 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.264 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.264 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.264 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.264 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.265 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.265 INFO:teuthology.orchestra.run.smithi090.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-24T09:24:43.265 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-24T09:24:43.265 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.265 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.265 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.265 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:43.265 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.266 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.266 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.266 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.266 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.266 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.266 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.266 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.266 INFO:teuthology.orchestra.run.smithi090.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-24T09:24:43.266 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-24T09:24:43.267 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.267 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.267 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.267 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.267 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.267 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.267 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.267 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.267 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.268 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.268 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.268 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.268 INFO:teuthology.orchestra.run.smithi090.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-24T09:24:43.268 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-24T09:24:43.268 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.268 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.268 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.269 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "[]", 2024-04-24T09:24:43.269 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.269 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.269 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.269 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.269 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.269 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.269 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.269 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.270 INFO:teuthology.orchestra.run.smithi090.stdout: "MULTICLUSTER_CONFIG": { 2024-04-24T09:24:43.270 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-24T09:24:43.270 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.270 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.270 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.270 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "{}", 2024-04-24T09:24:43.270 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.270 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.271 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.271 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.271 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.271 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.271 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.271 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.271 INFO:teuthology.orchestra.run.smithi090.stdout: "PROMETHEUS_API_HOST": { 2024-04-24T09:24:43.271 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-24T09:24:43.272 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.272 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.272 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.272 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.272 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.272 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.272 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.272 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.272 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.273 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.273 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.273 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.273 INFO:teuthology.orchestra.run.smithi090.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-24T09:24:43.273 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-24T09:24:43.273 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.273 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.273 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.274 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:43.274 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.274 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.274 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.274 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.274 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.274 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.274 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.274 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.275 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-24T09:24:43.275 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-24T09:24:43.275 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.275 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.275 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.275 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.275 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.276 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.276 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.276 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.276 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.276 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.276 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.276 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.276 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-24T09:24:43.276 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-24T09:24:43.277 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.277 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.277 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.277 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.277 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.277 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.277 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.277 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.277 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.277 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.278 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.278 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.278 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-24T09:24:43.278 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-24T09:24:43.278 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.278 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.278 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.278 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:43.279 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.279 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.279 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.279 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.279 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.279 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.279 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.279 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.280 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-24T09:24:43.280 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-24T09:24:43.280 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.280 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.280 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.280 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:43.280 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.280 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.280 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.281 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.281 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.281 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.281 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.281 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.281 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-24T09:24:43.281 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-24T09:24:43.281 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.281 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.282 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.282 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.282 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.282 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.282 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.282 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.282 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.283 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.283 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.283 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.283 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-24T09:24:43.283 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-24T09:24:43.283 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.283 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.283 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.283 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.284 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.284 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.284 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.284 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.284 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.284 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.284 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.284 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.285 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-24T09:24:43.285 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-24T09:24:43.285 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.285 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.285 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.285 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.285 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.285 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.286 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.286 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.286 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.286 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.286 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.286 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.286 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_ENABLED": { 2024-04-24T09:24:43.286 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-24T09:24:43.286 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.287 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.287 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.287 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:43.287 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.287 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.287 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.287 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.287 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.288 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.288 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.288 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.288 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-24T09:24:43.288 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-24T09:24:43.288 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.288 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.288 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.288 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-24T09:24:43.289 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.289 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.289 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.289 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.289 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.289 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.289 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.290 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.290 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-24T09:24:43.290 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-24T09:24:43.290 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:43.290 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.290 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.290 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-04-24T09:24:43.290 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.291 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.291 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.291 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.291 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.291 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.291 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.291 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.291 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-24T09:24:43.292 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-24T09:24:43.292 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:43.292 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.292 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.292 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8", 2024-04-24T09:24:43.292 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.292 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.292 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.292 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.293 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.293 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.293 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.293 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.293 INFO:teuthology.orchestra.run.smithi090.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-24T09:24:43.293 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-24T09:24:43.293 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:43.293 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.293 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.294 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "45", 2024-04-24T09:24:43.294 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.294 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.294 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.294 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.294 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.294 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.295 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.295 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.295 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_ACCESS_KEY": { 2024-04-24T09:24:43.295 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-24T09:24:43.295 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.295 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.295 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.296 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.296 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.296 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.296 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.296 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.296 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.296 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.296 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.297 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.297 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-24T09:24:43.297 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-24T09:24:43.297 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.297 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.297 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.297 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "admin", 2024-04-24T09:24:43.297 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.298 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.298 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.298 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.298 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.298 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.298 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.298 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.299 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_SECRET_KEY": { 2024-04-24T09:24:43.299 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-24T09:24:43.299 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.299 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.299 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.299 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.299 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.299 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.300 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.300 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.300 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.300 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.300 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.300 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.300 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_SSL_VERIFY": { 2024-04-24T09:24:43.300 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-24T09:24:43.301 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.301 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.301 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.301 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:43.301 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.301 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.301 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.301 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.302 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.302 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.302 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.302 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.302 INFO:teuthology.orchestra.run.smithi090.stdout: "UNSAFE_TLS_v1_2": { 2024-04-24T09:24:43.302 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-24T09:24:43.302 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.302 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.303 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.303 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.303 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.303 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.303 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.303 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.303 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.303 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.303 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.304 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.304 INFO:teuthology.orchestra.run.smithi090.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-24T09:24:43.304 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-24T09:24:43.304 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:43.304 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.304 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.304 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-04-24T09:24:43.304 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.304 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.305 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.305 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.305 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.305 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.305 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.305 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.305 INFO:teuthology.orchestra.run.smithi090.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-24T09:24:43.305 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-24T09:24:43.306 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:43.306 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.306 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.306 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-04-24T09:24:43.306 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.306 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.306 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.306 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.307 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.307 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.307 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.307 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.307 INFO:teuthology.orchestra.run.smithi090.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-24T09:24:43.307 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-24T09:24:43.307 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:43.307 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.308 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.308 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-04-24T09:24:43.308 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.308 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.308 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.308 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.308 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.308 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.309 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.309 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.309 INFO:teuthology.orchestra.run.smithi090.stdout: "cross_origin_url": { 2024-04-24T09:24:43.309 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cross_origin_url", 2024-04-24T09:24:43.309 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.309 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.309 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.309 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.310 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.310 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.310 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.310 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.310 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.310 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.310 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.310 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.311 INFO:teuthology.orchestra.run.smithi090.stdout: "crt_file": { 2024-04-24T09:24:43.311 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crt_file", 2024-04-24T09:24:43.311 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.311 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.311 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.311 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.311 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.311 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.312 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.312 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.312 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.312 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.312 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.312 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.312 INFO:teuthology.orchestra.run.smithi090.stdout: "debug": { 2024-04-24T09:24:43.312 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "debug", 2024-04-24T09:24:43.312 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.313 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.313 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.313 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.313 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.313 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.313 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.313 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Enable/disable debug options", 2024-04-24T09:24:43.313 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.313 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.314 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.314 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.314 INFO:teuthology.orchestra.run.smithi090.stdout: "jwt_token_ttl": { 2024-04-24T09:24:43.314 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "jwt_token_ttl", 2024-04-24T09:24:43.314 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:43.314 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.314 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.314 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "28800", 2024-04-24T09:24:43.314 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.315 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.315 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.315 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.315 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.315 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.315 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.315 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.315 INFO:teuthology.orchestra.run.smithi090.stdout: "key_file": { 2024-04-24T09:24:43.316 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "key_file", 2024-04-24T09:24:43.316 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.316 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.316 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.316 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.316 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.316 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.316 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.316 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.317 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.317 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.317 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.317 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.317 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:43.317 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:43.317 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.317 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.317 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.318 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.318 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.318 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.318 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.318 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.318 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.318 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.318 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.319 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.319 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.319 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.319 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.319 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.319 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.319 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.319 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.319 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:43.320 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:43.320 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.320 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.320 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.320 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.320 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.320 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.320 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.321 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.321 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.321 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.321 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.321 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.321 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:43.321 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:43.321 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.321 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.322 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.322 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:43.322 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.322 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.322 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.322 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.322 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.322 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.322 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.323 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.323 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.323 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.323 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.323 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.323 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.323 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.323 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.324 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:43.324 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:43.324 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.324 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.324 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.324 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.324 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.324 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.325 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.325 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.325 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.325 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.325 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.325 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.325 INFO:teuthology.orchestra.run.smithi090.stdout: "motd": { 2024-04-24T09:24:43.325 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "motd", 2024-04-24T09:24:43.325 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.325 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.326 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.326 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.326 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.326 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.326 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.326 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "The message of the day", 2024-04-24T09:24:43.326 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.326 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.326 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.327 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.327 INFO:teuthology.orchestra.run.smithi090.stdout: "redirect_resolve_ip_addr": { 2024-04-24T09:24:43.327 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "redirect_resolve_ip_addr", 2024-04-24T09:24:43.327 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.327 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.327 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.327 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.327 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.328 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.328 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.328 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.328 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.328 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.328 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.328 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.328 INFO:teuthology.orchestra.run.smithi090.stdout: "server_addr": { 2024-04-24T09:24:43.328 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_addr", 2024-04-24T09:24:43.329 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.329 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.329 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.329 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "::", 2024-04-24T09:24:43.329 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.329 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.329 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.329 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.330 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.330 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.330 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.330 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.330 INFO:teuthology.orchestra.run.smithi090.stdout: "server_port": { 2024-04-24T09:24:43.330 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_port", 2024-04-24T09:24:43.330 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:43.330 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.331 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.331 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8080", 2024-04-24T09:24:43.331 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.331 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.331 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.331 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.331 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.331 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.331 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.332 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.332 INFO:teuthology.orchestra.run.smithi090.stdout: "ssl": { 2024-04-24T09:24:43.332 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssl", 2024-04-24T09:24:43.332 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.332 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.332 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.332 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:43.332 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.332 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.333 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.333 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.333 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.333 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.333 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.333 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.333 INFO:teuthology.orchestra.run.smithi090.stdout: "ssl_server_port": { 2024-04-24T09:24:43.333 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssl_server_port", 2024-04-24T09:24:43.334 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:43.334 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.334 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.334 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8443", 2024-04-24T09:24:43.334 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.334 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.334 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.335 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.335 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.335 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.335 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.335 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.335 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_behaviour": { 2024-04-24T09:24:43.335 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_behaviour", 2024-04-24T09:24:43.335 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.336 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.336 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.336 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "redirect", 2024-04-24T09:24:43.336 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.336 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.336 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.336 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.337 INFO:teuthology.orchestra.run.smithi090.stdout: "redirect" 2024-04-24T09:24:43.337 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.337 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.337 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.337 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.337 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.337 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.337 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_error_status_code": { 2024-04-24T09:24:43.337 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_error_status_code", 2024-04-24T09:24:43.338 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:43.338 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.338 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.338 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "500", 2024-04-24T09:24:43.338 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "400", 2024-04-24T09:24:43.338 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "599", 2024-04-24T09:24:43.338 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.338 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.339 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.339 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.339 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.339 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.339 INFO:teuthology.orchestra.run.smithi090.stdout: "url_prefix": { 2024-04-24T09:24:43.339 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "url_prefix", 2024-04-24T09:24:43.339 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.339 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.340 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.340 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.340 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.340 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.340 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.340 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.340 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.340 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.341 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.341 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.341 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.341 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.341 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:43.341 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "devicehealth", 2024-04-24T09:24:43.341 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:43.341 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:43.341 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:43.342 INFO:teuthology.orchestra.run.smithi090.stdout: "enable_monitoring": { 2024-04-24T09:24:43.342 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enable_monitoring", 2024-04-24T09:24:43.342 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.342 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.342 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.342 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:43.342 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.342 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.343 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.343 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "monitor device health metrics", 2024-04-24T09:24:43.343 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.343 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.343 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.343 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.343 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:43.343 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:43.344 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.344 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.344 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.344 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.344 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.344 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.344 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.344 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.344 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.345 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.345 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.345 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.345 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.345 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.345 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.345 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.345 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.346 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.346 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.346 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:43.346 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:43.346 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.346 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.346 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.346 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.347 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.347 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.347 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.347 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.347 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.347 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.347 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.347 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.348 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:43.348 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:43.348 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.348 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.348 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.348 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:43.348 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.348 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.348 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.349 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.349 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.349 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.349 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.349 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.349 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.349 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.349 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.349 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.350 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.350 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.350 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.350 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:43.350 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:43.350 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.350 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.351 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.351 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.351 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.351 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.351 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.351 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.351 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.351 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.351 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.352 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.352 INFO:teuthology.orchestra.run.smithi090.stdout: "mark_out_threshold": { 2024-04-24T09:24:43.352 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mark_out_threshold", 2024-04-24T09:24:43.352 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:43.352 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.352 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.352 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "2419200", 2024-04-24T09:24:43.352 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.352 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.352 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.353 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-24T09:24:43.353 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.353 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.353 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.353 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.353 INFO:teuthology.orchestra.run.smithi090.stdout: "pool_name": { 2024-04-24T09:24:43.353 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pool_name", 2024-04-24T09:24:43.354 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.354 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.354 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.354 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "device_health_metrics", 2024-04-24T09:24:43.354 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.354 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.354 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.354 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-24T09:24:43.354 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.355 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.355 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.355 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.355 INFO:teuthology.orchestra.run.smithi090.stdout: "retention_period": { 2024-04-24T09:24:43.355 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "retention_period", 2024-04-24T09:24:43.355 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:43.355 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.355 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.355 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "15552000", 2024-04-24T09:24:43.355 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.356 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.356 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.356 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how long to retain device health metrics", 2024-04-24T09:24:43.356 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.356 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.356 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.356 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.356 INFO:teuthology.orchestra.run.smithi090.stdout: "scrape_frequency": { 2024-04-24T09:24:43.357 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "scrape_frequency", 2024-04-24T09:24:43.357 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:43.357 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.357 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.357 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "86400", 2024-04-24T09:24:43.357 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.357 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.357 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.357 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-24T09:24:43.358 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.358 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.358 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.358 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.358 INFO:teuthology.orchestra.run.smithi090.stdout: "self_heal": { 2024-04-24T09:24:43.358 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "self_heal", 2024-04-24T09:24:43.358 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.358 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.359 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.359 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:43.359 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.359 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.359 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.359 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-24T09:24:43.359 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.359 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.360 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.360 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.360 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-04-24T09:24:43.360 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-04-24T09:24:43.360 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:43.360 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.360 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.360 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-04-24T09:24:43.361 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.361 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.361 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.361 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to wake up and check device health", 2024-04-24T09:24:43.361 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.361 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.361 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.361 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.361 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_threshold": { 2024-04-24T09:24:43.362 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_threshold", 2024-04-24T09:24:43.362 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:43.362 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.362 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.362 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "7257600", 2024-04-24T09:24:43.362 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.362 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.363 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.363 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-24T09:24:43.363 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.363 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.363 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.363 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.363 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.363 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.363 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:43.364 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "diskprediction_local", 2024-04-24T09:24:43.364 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:43.364 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:43.364 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:43.364 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:43.364 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:43.364 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.364 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.364 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.365 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.365 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.365 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.365 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.365 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.365 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.365 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.365 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.366 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.366 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.366 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.366 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.366 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.366 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.366 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.366 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.367 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:43.367 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:43.367 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.367 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.367 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.367 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.367 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.367 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.367 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.367 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.368 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.368 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.368 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.368 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.368 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:43.368 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:43.368 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.368 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.369 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.369 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:43.369 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.369 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.369 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.369 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.369 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.369 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.370 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.370 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.370 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.370 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.370 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.370 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.370 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.370 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.371 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.371 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:43.371 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:43.371 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.371 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.371 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.371 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.371 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.371 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.372 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.372 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.372 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.372 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.372 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.372 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.372 INFO:teuthology.orchestra.run.smithi090.stdout: "predict_interval": { 2024-04-24T09:24:43.372 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "predict_interval", 2024-04-24T09:24:43.373 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.373 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.373 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.373 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "86400", 2024-04-24T09:24:43.373 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.373 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.373 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.373 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.374 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.374 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.374 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.374 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.374 INFO:teuthology.orchestra.run.smithi090.stdout: "predictor_model": { 2024-04-24T09:24:43.374 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "predictor_model", 2024-04-24T09:24:43.374 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.374 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.374 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.375 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "prophetstor", 2024-04-24T09:24:43.375 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.375 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.375 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.375 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.375 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.375 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.375 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.375 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.376 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-04-24T09:24:43.376 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-04-24T09:24:43.376 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.376 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.376 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.376 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-04-24T09:24:43.376 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.376 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.377 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.377 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.377 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.377 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.377 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.377 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.377 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.377 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.377 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:43.378 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "influx", 2024-04-24T09:24:43.378 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": false, 2024-04-24T09:24:43.378 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "influxdb python module not found", 2024-04-24T09:24:43.378 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:43.378 INFO:teuthology.orchestra.run.smithi090.stdout: "batch_size": { 2024-04-24T09:24:43.378 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "batch_size", 2024-04-24T09:24:43.378 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:43.378 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.379 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.379 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5000", 2024-04-24T09:24:43.379 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.379 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.379 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.379 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-24T09:24:43.379 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.379 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.379 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.380 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.380 INFO:teuthology.orchestra.run.smithi090.stdout: "database": { 2024-04-24T09:24:43.380 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "database", 2024-04-24T09:24:43.380 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.380 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.380 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.380 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "ceph", 2024-04-24T09:24:43.380 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.381 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.381 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.381 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-24T09:24:43.381 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.381 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.381 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.381 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.381 INFO:teuthology.orchestra.run.smithi090.stdout: "hostname": { 2024-04-24T09:24:43.382 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "hostname", 2024-04-24T09:24:43.382 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.382 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.382 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.382 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.382 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.382 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.382 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.383 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "InfluxDB server hostname", 2024-04-24T09:24:43.383 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.383 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.383 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.383 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.383 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-04-24T09:24:43.383 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-04-24T09:24:43.383 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:43.383 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.384 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.384 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "30", 2024-04-24T09:24:43.384 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "5", 2024-04-24T09:24:43.384 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.384 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.384 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-24T09:24:43.384 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.384 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.384 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.385 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.385 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:43.385 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:43.385 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.385 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.385 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.385 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.385 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.385 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.386 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.386 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.386 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.386 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.386 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.386 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.386 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.386 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.386 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.387 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.387 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.387 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.387 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.387 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:43.387 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:43.387 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.387 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.387 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.388 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.388 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.388 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.388 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.388 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.388 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.388 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.388 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.389 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.389 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:43.389 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:43.389 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.389 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.389 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.389 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:43.389 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.389 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.389 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.390 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.390 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.390 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.390 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.390 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.390 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.390 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.390 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.391 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.391 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.391 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.391 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.391 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:43.391 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:43.391 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.391 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.392 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.392 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.392 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.392 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.392 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.392 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.392 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.392 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.392 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.393 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.393 INFO:teuthology.orchestra.run.smithi090.stdout: "password": { 2024-04-24T09:24:43.393 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "password", 2024-04-24T09:24:43.393 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.393 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.393 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.393 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.393 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.393 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.394 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.394 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "password of InfluxDB server user", 2024-04-24T09:24:43.394 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.394 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.394 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.394 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.394 INFO:teuthology.orchestra.run.smithi090.stdout: "port": { 2024-04-24T09:24:43.394 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "port", 2024-04-24T09:24:43.395 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:43.395 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.395 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.395 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8086", 2024-04-24T09:24:43.395 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.395 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.395 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.395 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "InfluxDB server port", 2024-04-24T09:24:43.396 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.396 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.396 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.396 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.396 INFO:teuthology.orchestra.run.smithi090.stdout: "ssl": { 2024-04-24T09:24:43.396 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssl", 2024-04-24T09:24:43.396 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.396 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.396 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.397 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "false", 2024-04-24T09:24:43.397 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.397 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.397 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.397 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T09:24:43.397 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.397 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.397 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.398 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.398 INFO:teuthology.orchestra.run.smithi090.stdout: "threads": { 2024-04-24T09:24:43.398 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "threads", 2024-04-24T09:24:43.398 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:43.398 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.398 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.398 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-04-24T09:24:43.398 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-04-24T09:24:43.398 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "32", 2024-04-24T09:24:43.399 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.399 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-24T09:24:43.399 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.399 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.399 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.399 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.399 INFO:teuthology.orchestra.run.smithi090.stdout: "username": { 2024-04-24T09:24:43.399 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "username", 2024-04-24T09:24:43.400 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.400 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.400 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.400 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.400 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.400 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.400 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.400 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "username of InfluxDB server user", 2024-04-24T09:24:43.400 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.401 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.401 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.401 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.401 INFO:teuthology.orchestra.run.smithi090.stdout: "verify_ssl": { 2024-04-24T09:24:43.401 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "verify_ssl", 2024-04-24T09:24:43.401 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.401 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.401 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.402 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "true", 2024-04-24T09:24:43.402 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.402 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.402 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.402 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T09:24:43.402 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.402 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.402 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.402 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.403 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.403 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.403 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:43.403 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "insights", 2024-04-24T09:24:43.403 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:43.403 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:43.403 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:43.403 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:43.403 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:43.404 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.404 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.404 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.404 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.404 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.404 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.404 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.404 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.405 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.405 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.405 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.405 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.405 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.405 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.405 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.405 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.405 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.406 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.406 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.406 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:43.406 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:43.406 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.406 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.406 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.406 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.407 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.407 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.407 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.407 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.407 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.407 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.407 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.407 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.407 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:43.408 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:43.408 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.408 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.408 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.408 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:43.408 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.408 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.408 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.408 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.409 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.409 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.409 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.409 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.409 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.409 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.409 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.409 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.409 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.410 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.410 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.410 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:43.410 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:43.410 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.410 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.410 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.411 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.411 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.411 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.411 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.411 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.411 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.411 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.411 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.411 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.412 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.412 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.412 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:43.412 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "iostat", 2024-04-24T09:24:43.412 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:43.412 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:43.412 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:43.412 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:43.412 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:43.413 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.413 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.413 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.413 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.413 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.413 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.413 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.413 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.413 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.414 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.414 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.414 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.414 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.414 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.414 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.414 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.414 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.414 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.415 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.415 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:43.415 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:43.415 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.415 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.415 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.415 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.415 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.416 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.416 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.416 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.416 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.416 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.416 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.416 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.416 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:43.416 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:43.417 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.417 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.417 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.417 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:43.417 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.417 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.417 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.417 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.417 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.418 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.418 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.418 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.418 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.418 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.418 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.418 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.418 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.418 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.419 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.419 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:43.419 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:43.419 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.419 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.419 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.419 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.419 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.420 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.420 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.420 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.420 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.420 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.420 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.420 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.420 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.420 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.421 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:43.421 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "k8sevents", 2024-04-24T09:24:43.421 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:43.421 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:43.421 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:43.421 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_event_retention_days": { 2024-04-24T09:24:43.421 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ceph_event_retention_days", 2024-04-24T09:24:43.421 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:43.421 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.422 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.422 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "7", 2024-04-24T09:24:43.422 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.422 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.422 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.422 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-24T09:24:43.422 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.422 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.423 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.423 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.423 INFO:teuthology.orchestra.run.smithi090.stdout: "config_check_secs": { 2024-04-24T09:24:43.423 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "config_check_secs", 2024-04-24T09:24:43.423 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:43.423 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.423 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.423 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-04-24T09:24:43.423 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "10", 2024-04-24T09:24:43.424 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.424 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.424 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-24T09:24:43.424 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.424 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.424 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.424 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.424 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:43.424 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:43.425 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.425 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.425 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.425 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.425 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.425 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.425 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.425 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.425 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.426 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.426 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.426 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.426 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.426 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.426 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.426 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.426 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.427 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.427 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.427 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:43.427 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:43.427 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.427 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.427 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.427 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.427 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.428 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.428 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.428 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.428 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.428 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.428 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.428 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.428 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:43.428 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:43.429 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.429 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.429 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.429 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:43.429 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.429 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.429 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.429 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.430 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.430 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.430 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.430 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.430 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.430 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.430 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.430 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.431 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.431 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.431 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.431 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:43.431 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:43.431 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.431 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.431 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.432 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.432 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.432 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.432 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.432 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.432 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.432 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.432 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.432 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.433 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.433 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.433 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:43.433 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "localpool", 2024-04-24T09:24:43.433 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:43.433 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:43.433 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:43.433 INFO:teuthology.orchestra.run.smithi090.stdout: "failure_domain": { 2024-04-24T09:24:43.433 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "failure_domain", 2024-04-24T09:24:43.434 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.434 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.434 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.434 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "host", 2024-04-24T09:24:43.434 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.434 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.434 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.434 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "failure domain for any created local pool", 2024-04-24T09:24:43.435 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-24T09:24:43.435 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.435 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.435 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.435 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:43.435 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:43.435 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.435 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.436 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.436 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.436 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.436 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.436 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.436 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.436 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.436 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.436 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.437 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.437 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.437 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.437 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.437 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.437 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.437 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.437 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.438 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:43.438 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:43.438 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.438 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.438 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.438 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.438 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.438 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.439 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.439 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.439 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.439 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.439 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.439 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.439 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:43.439 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:43.439 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.440 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.440 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.440 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:43.440 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.440 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.440 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.440 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.440 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.441 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.441 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.441 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.441 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.441 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.441 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.441 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.441 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.442 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.442 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.442 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:43.442 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:43.442 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.442 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.442 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.442 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.443 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.443 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.443 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.443 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.443 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.443 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.443 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.443 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.443 INFO:teuthology.orchestra.run.smithi090.stdout: "min_size": { 2024-04-24T09:24:43.444 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "min_size", 2024-04-24T09:24:43.444 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:43.444 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.444 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.444 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.444 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.444 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.444 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.444 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "default min_size for any created local pool", 2024-04-24T09:24:43.445 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-24T09:24:43.445 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.445 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.445 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.445 INFO:teuthology.orchestra.run.smithi090.stdout: "num_rep": { 2024-04-24T09:24:43.445 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "num_rep", 2024-04-24T09:24:43.445 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:43.445 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.446 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.446 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "3", 2024-04-24T09:24:43.446 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.446 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.446 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.446 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "default replica count for any created local pool", 2024-04-24T09:24:43.446 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.446 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.446 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.447 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.447 INFO:teuthology.orchestra.run.smithi090.stdout: "pg_num": { 2024-04-24T09:24:43.447 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pg_num", 2024-04-24T09:24:43.447 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:43.447 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.447 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.447 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "128", 2024-04-24T09:24:43.447 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.447 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.448 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.448 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "default pg_num for any created local pool", 2024-04-24T09:24:43.448 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.448 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.448 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.448 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.448 INFO:teuthology.orchestra.run.smithi090.stdout: "prefix": { 2024-04-24T09:24:43.448 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "prefix", 2024-04-24T09:24:43.448 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.449 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.449 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.449 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.449 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.449 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.449 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.449 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "name prefix for any created local pool", 2024-04-24T09:24:43.449 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.450 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.450 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.450 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.450 INFO:teuthology.orchestra.run.smithi090.stdout: "subtree": { 2024-04-24T09:24:43.450 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "subtree", 2024-04-24T09:24:43.450 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.450 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.450 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.450 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "rack", 2024-04-24T09:24:43.451 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.451 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.451 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.451 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-24T09:24:43.451 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-24T09:24:43.451 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.451 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.451 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.451 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.452 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.452 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:43.452 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mds_autoscaler", 2024-04-24T09:24:43.452 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:43.452 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:43.452 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:43.452 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:43.453 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:43.453 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.453 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.453 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.453 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.453 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.453 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.453 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.453 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.454 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.454 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.454 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.454 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.454 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.454 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.454 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.454 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.455 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.455 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.455 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.455 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:43.455 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:43.455 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.455 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.455 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.455 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.456 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.456 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.456 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.456 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.456 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.456 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.456 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.456 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.456 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:43.457 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:43.457 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.457 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.457 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.457 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:43.457 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.457 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.457 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.457 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.458 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.458 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.458 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.458 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.458 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.458 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.458 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.458 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.458 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.459 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.459 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.459 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:43.459 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:43.459 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.459 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.459 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.459 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.459 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.460 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.460 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.460 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.460 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.460 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.460 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.460 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.460 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.460 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.461 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:43.461 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mirroring", 2024-04-24T09:24:43.461 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:43.461 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:43.461 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:43.461 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:43.461 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:43.461 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.461 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.462 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.462 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.462 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.462 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.462 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.462 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.462 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.462 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.463 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.463 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.463 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.463 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.463 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.463 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.463 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.463 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.463 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.464 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:43.464 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:43.464 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.464 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.464 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.464 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.464 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.464 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.464 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.465 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.465 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.465 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.465 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.465 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.465 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:43.465 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:43.465 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.465 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.466 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.466 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:43.466 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.466 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.466 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.466 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.466 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.466 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.466 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.467 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.467 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.467 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.467 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.467 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.467 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.467 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.467 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.467 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:43.468 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:43.468 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.468 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.468 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.468 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.468 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.468 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.468 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.468 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.469 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.469 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.469 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.469 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.469 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.469 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.469 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:43.469 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "nfs", 2024-04-24T09:24:43.470 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:43.470 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:43.470 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:43.470 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:43.470 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:43.470 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.470 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.470 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.470 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.471 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.471 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.471 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.471 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.471 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.471 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.471 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.471 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.472 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.472 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.472 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.472 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.472 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.472 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.472 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.472 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:43.473 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:43.473 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.473 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.473 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.473 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.473 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.473 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.474 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.474 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.474 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.474 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.474 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.474 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.474 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:43.474 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:43.474 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.475 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.475 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.475 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:43.475 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.475 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.475 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.475 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.475 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.475 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.476 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.476 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.476 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.476 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.476 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.476 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.476 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.476 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.476 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.477 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:43.477 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:43.477 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.477 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.477 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.477 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.477 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.477 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.477 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.478 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.478 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.478 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.478 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.478 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.478 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.478 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.478 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:43.479 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "orchestrator", 2024-04-24T09:24:43.479 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:43.479 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:43.479 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:43.479 INFO:teuthology.orchestra.run.smithi090.stdout: "fail_fs": { 2024-04-24T09:24:43.479 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "fail_fs", 2024-04-24T09:24:43.479 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.479 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.479 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.480 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.480 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.480 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.480 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.480 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-24T09:24:43.480 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.480 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.480 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.480 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.481 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:43.481 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:43.481 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.481 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.481 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.481 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.481 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.481 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.482 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.482 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.482 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.482 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.482 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.482 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.482 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.482 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.483 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.483 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.483 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.483 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.483 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.483 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:43.483 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:43.483 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.483 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.484 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.484 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.484 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.484 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.484 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.484 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.484 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.484 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.485 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.485 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.485 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:43.485 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:43.485 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.485 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.485 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.485 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:43.485 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.486 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.486 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.486 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.486 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.486 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.486 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.486 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.486 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.486 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.487 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.487 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.487 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.487 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.487 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.487 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:43.487 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:43.487 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.488 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.488 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.488 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.488 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.488 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.488 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.488 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.488 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.488 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.489 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.489 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.489 INFO:teuthology.orchestra.run.smithi090.stdout: "orchestrator": { 2024-04-24T09:24:43.489 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "orchestrator", 2024-04-24T09:24:43.489 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.489 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.489 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.489 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.489 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.490 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.490 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.490 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm", 2024-04-24T09:24:43.490 INFO:teuthology.orchestra.run.smithi090.stdout: "rook", 2024-04-24T09:24:43.490 INFO:teuthology.orchestra.run.smithi090.stdout: "test_orchestrator" 2024-04-24T09:24:43.490 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.490 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Orchestrator backend", 2024-04-24T09:24:43.490 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.490 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.491 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.491 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.491 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.491 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.491 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:43.491 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "osd_perf_query", 2024-04-24T09:24:43.491 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:43.491 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:43.492 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:43.492 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:43.492 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:43.492 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.492 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.492 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.492 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.492 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.492 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.492 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.493 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.493 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.493 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.493 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.493 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.493 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.493 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.493 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.493 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.494 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.494 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.494 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.494 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:43.494 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:43.494 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.494 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.494 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.494 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.495 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.495 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.495 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.495 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.495 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.495 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.495 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.495 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.495 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:43.496 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:43.496 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.496 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.496 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.496 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:43.496 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.496 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.496 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.497 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.497 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.497 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.497 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.497 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.497 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.497 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.497 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.497 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.498 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.498 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.498 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.498 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:43.498 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:43.498 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.498 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.498 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.498 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.499 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.499 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.499 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.499 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.499 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.499 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.499 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.499 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.499 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.500 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.500 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:43.500 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "osd_support", 2024-04-24T09:24:43.500 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:43.500 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:43.500 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:43.500 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:43.500 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:43.500 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.501 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.501 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.501 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.501 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.501 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.501 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.501 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.501 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.501 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.502 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.502 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.502 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.502 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.502 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.502 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.502 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.502 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.502 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.503 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:43.503 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:43.503 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.503 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.503 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.503 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.503 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.503 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.503 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.504 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.504 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.504 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.504 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.504 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.504 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:43.504 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:43.504 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.504 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.505 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.505 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:43.505 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.505 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.505 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.505 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.505 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.505 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.505 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.506 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.506 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.506 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.506 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.506 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.506 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.506 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.506 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.506 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:43.507 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:43.507 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.507 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.507 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.507 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.507 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.507 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.507 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.508 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.508 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.508 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.508 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.508 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.508 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.508 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.508 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:43.508 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pg_autoscaler", 2024-04-24T09:24:43.509 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:43.509 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:43.509 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:43.509 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:43.509 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:43.509 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.509 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.509 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.510 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.510 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.510 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.510 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.510 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.510 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.510 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.510 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.511 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.511 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.511 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.511 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.511 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.511 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.511 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.511 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.511 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:43.512 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:43.512 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.512 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.512 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.512 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.512 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.512 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.512 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.513 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.513 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.513 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.513 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.513 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.513 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:43.513 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:43.513 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.513 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.514 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.514 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:43.514 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.514 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.514 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.514 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.514 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.514 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.515 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.515 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.515 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.515 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.515 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.515 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.515 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.515 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.516 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.516 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:43.516 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:43.516 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.516 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.516 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.516 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.516 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.516 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.516 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.517 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.517 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.517 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.517 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.517 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.517 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-04-24T09:24:43.517 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-04-24T09:24:43.517 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:43.517 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.518 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.518 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-04-24T09:24:43.518 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.518 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.518 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.518 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.518 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.518 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.519 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.519 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.519 INFO:teuthology.orchestra.run.smithi090.stdout: "threshold": { 2024-04-24T09:24:43.519 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "threshold", 2024-04-24T09:24:43.519 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-04-24T09:24:43.519 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.519 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.519 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "3.0", 2024-04-24T09:24:43.520 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1.0", 2024-04-24T09:24:43.520 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.520 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.520 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "scaling threshold", 2024-04-24T09:24:43.520 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-24T09:24:43.520 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.520 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.520 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.520 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.521 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.521 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:43.521 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "progress", 2024-04-24T09:24:43.521 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:43.521 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:43.521 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:43.521 INFO:teuthology.orchestra.run.smithi090.stdout: "allow_pg_recovery_event": { 2024-04-24T09:24:43.521 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "allow_pg_recovery_event", 2024-04-24T09:24:43.522 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.522 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.522 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.522 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.522 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.522 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.522 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.522 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-24T09:24:43.523 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.523 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.523 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.523 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.523 INFO:teuthology.orchestra.run.smithi090.stdout: "enabled": { 2024-04-24T09:24:43.523 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enabled", 2024-04-24T09:24:43.523 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.523 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.523 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.524 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:43.524 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.524 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.524 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.524 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.524 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.524 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.524 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.524 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.525 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:43.525 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:43.525 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.525 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.525 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.525 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.525 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.525 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.525 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.525 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.526 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.526 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.526 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.526 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.526 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.526 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.526 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.526 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.527 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.527 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.527 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.527 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:43.527 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:43.527 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.527 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.527 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.527 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.528 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.528 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.528 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.528 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.528 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.528 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.528 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.528 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.528 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:43.528 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:43.529 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.529 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.529 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.529 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:43.529 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.529 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.529 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.529 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.530 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.530 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.530 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.530 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.530 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.530 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.530 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.530 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.531 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.531 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.531 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.531 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:43.531 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:43.531 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.531 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.531 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.531 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.532 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.532 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.532 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.532 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.532 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.532 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.532 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.532 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.533 INFO:teuthology.orchestra.run.smithi090.stdout: "max_completed_events": { 2024-04-24T09:24:43.533 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_completed_events", 2024-04-24T09:24:43.533 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:43.533 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.533 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.533 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "50", 2024-04-24T09:24:43.533 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.533 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.533 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.534 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "number of past completed events to remember", 2024-04-24T09:24:43.534 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.534 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.534 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.534 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.534 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-04-24T09:24:43.534 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-04-24T09:24:43.534 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:43.535 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.535 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.535 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-04-24T09:24:43.535 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.535 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.535 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.535 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how long the module is going to sleep", 2024-04-24T09:24:43.535 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.536 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.536 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.536 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.536 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.536 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.536 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:43.536 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "prometheus", 2024-04-24T09:24:43.536 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:43.536 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:43.537 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:43.537 INFO:teuthology.orchestra.run.smithi090.stdout: "cache": { 2024-04-24T09:24:43.537 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cache", 2024-04-24T09:24:43.537 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.537 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.537 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.537 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:43.537 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.537 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.538 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.538 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.538 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.538 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.538 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.538 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.538 INFO:teuthology.orchestra.run.smithi090.stdout: "exclude_perf_counters": { 2024-04-24T09:24:43.538 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "exclude_perf_counters", 2024-04-24T09:24:43.539 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.539 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.539 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.539 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:43.539 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.539 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.539 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.539 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-24T09:24:43.540 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-24T09:24:43.540 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.540 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.540 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.540 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:43.540 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:43.540 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.540 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.540 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.541 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.541 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.541 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.541 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.541 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.541 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.541 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.541 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.542 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.542 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.542 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.542 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.542 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.542 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.542 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.542 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.542 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:43.543 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:43.543 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.543 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.543 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.543 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.543 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.543 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.543 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.543 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.544 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.544 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.544 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.544 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.544 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:43.544 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:43.544 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.544 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.545 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.545 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:43.545 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.545 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.545 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.545 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.545 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.545 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.546 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.546 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.546 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.546 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.546 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.546 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.546 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.546 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.546 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.547 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:43.547 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:43.547 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.547 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.547 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.547 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.547 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.547 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.548 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.548 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.548 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.548 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.548 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.548 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.548 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_stats_pools": { 2024-04-24T09:24:43.548 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_stats_pools", 2024-04-24T09:24:43.548 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.549 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.549 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.549 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.549 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.549 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.549 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.549 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.549 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.549 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.550 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.550 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.550 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-24T09:24:43.550 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-24T09:24:43.550 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:43.550 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.550 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.550 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "300", 2024-04-24T09:24:43.551 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.551 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.551 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.551 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.551 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.551 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.551 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.551 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.551 INFO:teuthology.orchestra.run.smithi090.stdout: "scrape_interval": { 2024-04-24T09:24:43.552 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "scrape_interval", 2024-04-24T09:24:43.552 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-04-24T09:24:43.552 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.552 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.552 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "15.0", 2024-04-24T09:24:43.552 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.552 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.552 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.552 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.553 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.553 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.553 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.553 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.553 INFO:teuthology.orchestra.run.smithi090.stdout: "server_addr": { 2024-04-24T09:24:43.553 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_addr", 2024-04-24T09:24:43.553 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.553 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.554 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.554 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "::", 2024-04-24T09:24:43.554 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.554 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.554 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.554 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-24T09:24:43.554 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.554 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.554 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.555 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.555 INFO:teuthology.orchestra.run.smithi090.stdout: "server_port": { 2024-04-24T09:24:43.555 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_port", 2024-04-24T09:24:43.555 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:43.555 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.555 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.555 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "9283", 2024-04-24T09:24:43.555 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.556 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.556 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.556 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-24T09:24:43.556 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.556 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.556 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.556 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.556 INFO:teuthology.orchestra.run.smithi090.stdout: "stale_cache_strategy": { 2024-04-24T09:24:43.556 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "stale_cache_strategy", 2024-04-24T09:24:43.557 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.557 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.557 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.557 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "log", 2024-04-24T09:24:43.557 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.557 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.557 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.557 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.557 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.558 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.558 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.558 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.558 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_behaviour": { 2024-04-24T09:24:43.558 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_behaviour", 2024-04-24T09:24:43.558 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.558 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.558 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.559 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "default", 2024-04-24T09:24:43.559 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.559 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.559 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.559 INFO:teuthology.orchestra.run.smithi090.stdout: "default", 2024-04-24T09:24:43.559 INFO:teuthology.orchestra.run.smithi090.stdout: "error" 2024-04-24T09:24:43.559 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.559 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.559 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.560 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.560 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.560 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.560 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_error_status_code": { 2024-04-24T09:24:43.560 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_error_status_code", 2024-04-24T09:24:43.560 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:43.560 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.560 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.561 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "500", 2024-04-24T09:24:43.561 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "400", 2024-04-24T09:24:43.561 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "599", 2024-04-24T09:24:43.561 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.561 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.561 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.561 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.561 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.561 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.562 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.562 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.562 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:43.562 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_support", 2024-04-24T09:24:43.562 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:43.562 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:43.562 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:43.562 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:43.562 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:43.563 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.563 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.563 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.563 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.563 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.563 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.563 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.563 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.563 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.564 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.564 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.564 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.564 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.564 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.564 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.564 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.564 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.564 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.565 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.565 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:43.565 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:43.565 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.565 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.565 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.565 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.565 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.565 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.566 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.566 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.566 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.566 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.566 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.566 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.566 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:43.566 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:43.566 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.567 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.567 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.567 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:43.567 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.567 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.567 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.567 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.567 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.567 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.567 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.568 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.568 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.568 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.568 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.568 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.568 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.568 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.568 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.568 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:43.569 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:43.569 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.569 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.569 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.569 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.569 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.569 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.569 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.569 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.570 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.570 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.570 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.570 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.570 INFO:teuthology.orchestra.run.smithi090.stdout: "max_concurrent_snap_create": { 2024-04-24T09:24:43.570 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_concurrent_snap_create", 2024-04-24T09:24:43.570 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:43.570 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.570 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.571 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-04-24T09:24:43.571 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.571 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.571 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.571 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.571 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.571 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.571 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.571 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.571 INFO:teuthology.orchestra.run.smithi090.stdout: "mirror_snapshot_schedule": { 2024-04-24T09:24:43.572 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mirror_snapshot_schedule", 2024-04-24T09:24:43.572 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.572 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.572 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.572 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.572 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.572 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.572 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.572 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.573 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.573 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.573 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.573 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.573 INFO:teuthology.orchestra.run.smithi090.stdout: "trash_purge_schedule": { 2024-04-24T09:24:43.573 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "trash_purge_schedule", 2024-04-24T09:24:43.573 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.573 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.573 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.574 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.574 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.574 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.574 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.574 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.574 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.574 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.574 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.574 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.574 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.575 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.575 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:43.575 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "restful", 2024-04-24T09:24:43.575 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:43.575 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:43.575 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:43.575 INFO:teuthology.orchestra.run.smithi090.stdout: "enable_auth": { 2024-04-24T09:24:43.576 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enable_auth", 2024-04-24T09:24:43.576 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.576 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.576 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.576 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:43.576 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.576 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.576 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.577 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.577 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.577 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.577 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.577 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.577 INFO:teuthology.orchestra.run.smithi090.stdout: "key_file": { 2024-04-24T09:24:43.577 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "key_file", 2024-04-24T09:24:43.577 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.577 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.578 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.578 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.578 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.578 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.578 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.578 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.578 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.578 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.578 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.579 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.579 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:43.579 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:43.579 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.579 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.579 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.579 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.579 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.579 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.580 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.580 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.580 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.580 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.580 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.580 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.580 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.580 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.581 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.581 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.581 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.581 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.581 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.581 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:43.581 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:43.581 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.581 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.582 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.582 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.582 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.582 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.582 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.582 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.582 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.582 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.583 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.583 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.583 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:43.583 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:43.583 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.583 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.583 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.583 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:43.583 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.584 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.584 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.584 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.584 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.584 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.584 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.584 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.584 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.585 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.585 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.585 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.585 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.585 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.585 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.585 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:43.585 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:43.586 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.586 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.586 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.586 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.586 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.586 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.586 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.586 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.587 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.587 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.587 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.587 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.587 INFO:teuthology.orchestra.run.smithi090.stdout: "server_addr": { 2024-04-24T09:24:43.587 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_addr", 2024-04-24T09:24:43.587 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.587 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.588 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.588 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.588 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.588 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.588 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.588 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.588 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.588 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.588 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.589 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.589 INFO:teuthology.orchestra.run.smithi090.stdout: "server_port": { 2024-04-24T09:24:43.589 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_port", 2024-04-24T09:24:43.589 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.589 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.589 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.589 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.589 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.589 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.590 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.590 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.590 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.590 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.590 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.590 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.590 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.590 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.590 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:43.591 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rgw", 2024-04-24T09:24:43.591 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:43.591 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:43.591 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:43.591 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:43.591 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:43.591 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.591 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.592 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.592 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.592 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.592 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.592 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.592 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.592 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.592 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.592 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.593 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.593 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.593 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.593 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.593 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.593 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.593 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.593 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.593 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:43.594 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:43.594 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.594 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.594 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.594 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.594 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.594 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.594 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.594 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.595 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.595 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.595 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.595 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.595 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:43.595 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:43.595 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.595 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.595 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.595 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:43.596 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.596 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.596 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.596 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.596 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.596 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.596 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.596 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.597 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.597 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.597 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.597 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.597 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.597 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.597 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.597 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:43.597 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:43.598 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.598 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.598 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.598 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.598 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.598 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.598 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.598 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.598 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.598 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.599 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.599 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.599 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.599 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.599 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:43.599 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rook", 2024-04-24T09:24:43.599 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:43.599 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:43.599 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:43.600 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:43.600 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:43.600 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.600 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.600 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.600 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.600 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.600 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.600 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.601 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.601 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.601 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.601 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.601 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.601 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.601 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.601 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.602 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.602 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.602 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.602 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.602 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:43.602 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:43.602 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.602 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.602 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.603 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.603 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.603 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.603 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.603 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.603 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.603 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.603 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.603 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.604 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:43.604 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:43.604 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.604 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.604 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.604 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:43.604 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.604 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.604 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.605 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.605 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.605 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.605 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.605 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.605 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.605 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.605 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.605 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.606 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.606 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.606 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.606 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:43.606 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:43.606 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.606 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.606 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.606 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.607 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.607 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.607 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.607 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.607 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.607 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.607 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.607 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.607 INFO:teuthology.orchestra.run.smithi090.stdout: "storage_class": { 2024-04-24T09:24:43.608 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "storage_class", 2024-04-24T09:24:43.608 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.608 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.608 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.608 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "local", 2024-04-24T09:24:43.608 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.608 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.608 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.609 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-24T09:24:43.609 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.609 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.609 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.609 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.609 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.609 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.609 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:43.610 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "selftest", 2024-04-24T09:24:43.610 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:43.610 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:43.610 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:43.610 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:43.610 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:43.610 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.610 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.611 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.611 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.611 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.611 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.611 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.611 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.611 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.611 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.611 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.612 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.612 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.612 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.612 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.612 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.612 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.612 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.612 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.612 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:43.613 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:43.613 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.613 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.613 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.613 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.613 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.613 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.613 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.613 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.614 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.614 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.614 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.614 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.614 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:43.614 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:43.614 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.614 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.614 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.615 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:43.615 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.615 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.615 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.615 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.615 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.615 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.615 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.615 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.616 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.616 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.616 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.616 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.616 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.616 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.616 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.616 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:43.616 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:43.616 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.617 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.617 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.617 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.617 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.617 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.617 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.617 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.617 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.617 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.618 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.618 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.618 INFO:teuthology.orchestra.run.smithi090.stdout: "roption1": { 2024-04-24T09:24:43.618 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "roption1", 2024-04-24T09:24:43.618 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.618 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.618 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.618 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.619 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.619 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.619 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.619 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.619 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.619 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.619 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.619 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.620 INFO:teuthology.orchestra.run.smithi090.stdout: "roption2": { 2024-04-24T09:24:43.620 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "roption2", 2024-04-24T09:24:43.620 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.620 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.620 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.620 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "xyz", 2024-04-24T09:24:43.620 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.620 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.620 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.621 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.621 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.621 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.621 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.621 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.621 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption1": { 2024-04-24T09:24:43.621 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption1", 2024-04-24T09:24:43.621 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.621 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.622 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.622 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.622 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.622 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.622 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.622 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.622 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.622 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.622 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.623 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.623 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption2": { 2024-04-24T09:24:43.623 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption2", 2024-04-24T09:24:43.623 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:43.623 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.623 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.623 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.623 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.623 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.624 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.624 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.624 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.624 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.624 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.624 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.624 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption3": { 2024-04-24T09:24:43.624 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption3", 2024-04-24T09:24:43.625 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-04-24T09:24:43.625 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.625 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.625 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.625 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.625 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.625 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.625 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.625 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.626 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.626 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.626 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.626 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption4": { 2024-04-24T09:24:43.626 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption4", 2024-04-24T09:24:43.626 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.626 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.626 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.626 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.627 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.627 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.627 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.627 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.627 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.627 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.627 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.627 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.627 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption5": { 2024-04-24T09:24:43.628 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption5", 2024-04-24T09:24:43.628 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.628 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.628 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.628 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.628 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.628 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.628 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.629 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.629 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.629 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.629 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.629 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.629 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption6": { 2024-04-24T09:24:43.629 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption6", 2024-04-24T09:24:43.629 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.629 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.630 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.630 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:43.630 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.630 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.630 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.630 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.630 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.630 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.631 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.631 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.631 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption7": { 2024-04-24T09:24:43.631 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption7", 2024-04-24T09:24:43.631 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:43.631 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.631 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.631 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.632 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-04-24T09:24:43.632 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "42", 2024-04-24T09:24:43.632 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.632 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.632 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.632 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.632 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.632 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.633 INFO:teuthology.orchestra.run.smithi090.stdout: "testkey": { 2024-04-24T09:24:43.633 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "testkey", 2024-04-24T09:24:43.633 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.633 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.633 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.633 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.633 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.633 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.633 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.634 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.634 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.634 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.634 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.634 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.634 INFO:teuthology.orchestra.run.smithi090.stdout: "testlkey": { 2024-04-24T09:24:43.634 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "testlkey", 2024-04-24T09:24:43.634 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.635 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.635 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.635 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.635 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.635 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.635 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.635 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.635 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.635 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.636 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.636 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.636 INFO:teuthology.orchestra.run.smithi090.stdout: "testnewline": { 2024-04-24T09:24:43.636 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "testnewline", 2024-04-24T09:24:43.636 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.636 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.636 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.636 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.636 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.637 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.637 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.637 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.637 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.637 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.637 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.637 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.637 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.638 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.638 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:43.638 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snap_schedule", 2024-04-24T09:24:43.638 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:43.638 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:43.638 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:43.638 INFO:teuthology.orchestra.run.smithi090.stdout: "allow_m_granularity": { 2024-04-24T09:24:43.638 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "allow_m_granularity", 2024-04-24T09:24:43.638 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.639 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.639 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.639 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.639 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.639 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.639 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.639 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "allow minute scheduled snapshots", 2024-04-24T09:24:43.639 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.639 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.640 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.640 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.640 INFO:teuthology.orchestra.run.smithi090.stdout: "dump_on_update": { 2024-04-24T09:24:43.640 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "dump_on_update", 2024-04-24T09:24:43.640 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.640 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.640 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.640 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.640 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.641 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.641 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.641 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "dump database to debug log on update", 2024-04-24T09:24:43.641 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.641 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.641 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.641 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.641 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:43.642 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:43.642 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.642 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.642 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.642 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.642 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.642 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.642 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.642 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.642 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.643 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.643 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.643 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.643 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.643 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.643 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.643 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.643 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.643 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.644 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.644 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:43.644 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:43.644 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.644 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.644 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.644 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.644 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.644 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.645 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.645 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.645 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.645 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.645 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.645 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.645 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:43.645 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:43.645 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.646 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.646 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.646 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:43.646 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.646 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.646 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.646 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.646 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.647 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.647 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.647 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.647 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.647 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.647 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.647 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.647 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.647 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.648 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.648 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:43.648 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:43.648 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.648 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.648 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.648 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.648 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.648 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.649 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.649 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.649 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.649 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.649 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.649 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.649 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.649 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.649 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:43.650 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "stats", 2024-04-24T09:24:43.650 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:43.650 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:43.650 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:43.650 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:43.650 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:43.650 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.650 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.651 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.651 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.651 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.651 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.651 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.651 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.651 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.651 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.651 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.652 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.652 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.652 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.652 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.652 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.652 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.652 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.652 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.652 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:43.652 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:43.653 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.653 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.653 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.653 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.653 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.653 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.653 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.653 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.654 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.654 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.654 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.654 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.654 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:43.654 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:43.654 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.654 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.654 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.655 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:43.655 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.655 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.655 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.655 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.655 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.655 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.655 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.656 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.656 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.656 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.656 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.656 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.656 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.656 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.656 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.656 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:43.657 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:43.657 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.657 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.657 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.657 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.657 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.657 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.657 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.657 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.658 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.658 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.658 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.658 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.658 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.658 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.658 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:43.658 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "status", 2024-04-24T09:24:43.658 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:43.659 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:43.659 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:43.659 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:43.659 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:43.659 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.659 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.659 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.659 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.659 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.660 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.660 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.660 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.660 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.660 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.660 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.660 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.660 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.661 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.661 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.661 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.661 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.661 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.661 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.661 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:43.661 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:43.661 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.662 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.662 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.662 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.662 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.662 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.662 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.662 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.662 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.662 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.663 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.663 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.663 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:43.663 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:43.663 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.663 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.663 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.663 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:43.663 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.663 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.664 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.664 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.664 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.664 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.664 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.664 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.664 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.664 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.664 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.665 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.665 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.665 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.665 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.665 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:43.665 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:43.665 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.665 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.665 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.666 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.666 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.666 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.666 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.666 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.666 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.666 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.666 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.666 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.667 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.667 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.667 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:43.667 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "telegraf", 2024-04-24T09:24:43.667 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:43.667 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:43.667 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:43.667 INFO:teuthology.orchestra.run.smithi090.stdout: "address": { 2024-04-24T09:24:43.667 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "address", 2024-04-24T09:24:43.668 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.668 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.668 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.668 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-24T09:24:43.668 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.668 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.668 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.668 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.668 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.669 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.669 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.669 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.669 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-04-24T09:24:43.669 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-04-24T09:24:43.669 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:43.669 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.669 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.669 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "15", 2024-04-24T09:24:43.670 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.670 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.670 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.670 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.670 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.670 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.670 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.670 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.671 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:43.671 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:43.671 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.671 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.671 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.671 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.671 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.671 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.671 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.672 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.672 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.672 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.672 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.672 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.672 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.672 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.672 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.672 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.673 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.673 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.673 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.673 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:43.673 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:43.673 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.673 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.673 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.673 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.674 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.674 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.674 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.674 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.674 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.674 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.674 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.674 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.675 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:43.675 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:43.675 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.675 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.675 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.675 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:43.675 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.675 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.675 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.676 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.676 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.676 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.676 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.676 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.676 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.676 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.676 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.676 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.677 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.677 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.677 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.677 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:43.677 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:43.677 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.677 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.677 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.678 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.678 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.678 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.678 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.678 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.678 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.678 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.678 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.679 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.679 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.679 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.679 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:43.679 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "telemetry", 2024-04-24T09:24:43.679 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:43.679 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:43.679 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:43.679 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_basic": { 2024-04-24T09:24:43.680 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_basic", 2024-04-24T09:24:43.680 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.680 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.680 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.680 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:43.680 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.680 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.681 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.681 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-24T09:24:43.681 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.681 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.681 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.681 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.681 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_crash": { 2024-04-24T09:24:43.681 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_crash", 2024-04-24T09:24:43.681 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.682 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.682 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.682 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:43.682 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.682 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.682 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.682 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-24T09:24:43.682 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.683 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.683 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.683 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.683 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_device": { 2024-04-24T09:24:43.683 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_device", 2024-04-24T09:24:43.683 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.683 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.683 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.684 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:43.684 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.684 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.684 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.684 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-24T09:24:43.684 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.684 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.684 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.684 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.685 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_ident": { 2024-04-24T09:24:43.685 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_ident", 2024-04-24T09:24:43.685 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.685 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.685 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.685 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.685 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.685 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.685 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.686 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-24T09:24:43.686 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.686 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.686 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.686 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.686 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_perf": { 2024-04-24T09:24:43.686 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_perf", 2024-04-24T09:24:43.686 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.687 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.687 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.687 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.687 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.687 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.687 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.687 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-24T09:24:43.687 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.687 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.688 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.688 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.688 INFO:teuthology.orchestra.run.smithi090.stdout: "contact": { 2024-04-24T09:24:43.688 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "contact", 2024-04-24T09:24:43.688 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.688 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.688 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.688 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.688 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.689 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.689 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.689 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.689 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.689 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.689 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.689 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.689 INFO:teuthology.orchestra.run.smithi090.stdout: "description": { 2024-04-24T09:24:43.689 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "description", 2024-04-24T09:24:43.690 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.690 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.690 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.690 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.690 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.690 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.690 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.690 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.691 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.691 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.691 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.691 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.691 INFO:teuthology.orchestra.run.smithi090.stdout: "device_url": { 2024-04-24T09:24:43.691 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "device_url", 2024-04-24T09:24:43.691 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.691 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.691 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.692 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-24T09:24:43.692 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.692 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.692 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.692 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.692 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.692 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.692 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.692 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.693 INFO:teuthology.orchestra.run.smithi090.stdout: "enabled": { 2024-04-24T09:24:43.693 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enabled", 2024-04-24T09:24:43.693 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.693 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.693 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.693 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.693 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.693 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.694 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.694 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.694 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.694 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.694 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.694 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.694 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-04-24T09:24:43.694 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-04-24T09:24:43.694 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:43.695 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.695 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.695 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "24", 2024-04-24T09:24:43.695 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "8", 2024-04-24T09:24:43.695 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.695 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.695 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.695 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.695 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.696 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.696 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.696 INFO:teuthology.orchestra.run.smithi090.stdout: "last_opt_revision": { 2024-04-24T09:24:43.696 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "last_opt_revision", 2024-04-24T09:24:43.696 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:43.696 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.696 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.696 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "1", 2024-04-24T09:24:43.696 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.697 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.697 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.697 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.697 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.697 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.697 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.697 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.697 INFO:teuthology.orchestra.run.smithi090.stdout: "leaderboard": { 2024-04-24T09:24:43.698 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "leaderboard", 2024-04-24T09:24:43.698 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.698 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.698 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.698 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.698 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.698 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.698 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.698 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.699 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.699 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.699 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.699 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.699 INFO:teuthology.orchestra.run.smithi090.stdout: "leaderboard_description": { 2024-04-24T09:24:43.699 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "leaderboard_description", 2024-04-24T09:24:43.699 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.699 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.699 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.700 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.700 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.700 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.700 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.700 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.700 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.700 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.700 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.700 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.701 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:43.701 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:43.701 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.701 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.701 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.701 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.701 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.701 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.701 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.702 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.702 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.702 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.702 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.702 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.702 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.702 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.702 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.702 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.703 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.703 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.703 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.703 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:43.703 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:43.703 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.703 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.703 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.703 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.704 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.704 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.704 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.704 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.704 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.704 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.704 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.704 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.704 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:43.704 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:43.705 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.705 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.705 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.705 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:43.705 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.705 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.705 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.705 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.705 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.706 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.706 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.706 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.706 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.706 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.706 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.706 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.706 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.706 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.707 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.707 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:43.707 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:43.707 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.707 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.707 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.707 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.707 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.707 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.707 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.708 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.708 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.708 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.708 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.708 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.708 INFO:teuthology.orchestra.run.smithi090.stdout: "organization": { 2024-04-24T09:24:43.708 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "organization", 2024-04-24T09:24:43.708 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.708 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.708 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.709 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.709 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.709 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.709 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.709 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.709 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.709 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.710 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.710 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.710 INFO:teuthology.orchestra.run.smithi090.stdout: "proxy": { 2024-04-24T09:24:43.710 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "proxy", 2024-04-24T09:24:43.710 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.710 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.710 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.710 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.711 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.711 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.711 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.711 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.711 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.711 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.711 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.711 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.711 INFO:teuthology.orchestra.run.smithi090.stdout: "url": { 2024-04-24T09:24:43.712 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "url", 2024-04-24T09:24:43.712 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.712 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.712 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.712 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-24T09:24:43.712 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.712 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.712 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.712 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.713 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.713 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.713 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.713 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.713 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.713 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.713 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:43.713 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "test_orchestrator", 2024-04-24T09:24:43.714 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:43.714 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:43.714 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:43.714 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:43.714 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:43.714 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.714 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.714 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.714 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.715 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.715 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.715 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.715 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.715 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.715 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.715 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.715 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.715 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.716 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.716 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.716 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.716 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.716 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.716 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.716 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:43.716 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:43.716 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.717 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.717 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.717 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.717 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.717 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.717 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.717 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.717 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.717 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.718 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.718 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.718 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:43.718 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:43.718 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.718 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.718 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.718 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:43.718 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.719 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.719 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.719 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.719 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.719 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.719 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.719 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.719 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.720 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.720 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.720 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.720 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.720 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.720 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.720 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:43.720 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:43.720 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.721 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.721 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.721 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.721 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.721 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.721 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.721 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.721 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.721 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.722 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.722 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.722 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.722 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.722 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:43.722 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "volumes", 2024-04-24T09:24:43.722 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:43.722 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:43.722 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:43.723 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:43.723 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:43.723 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.723 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.723 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.723 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.723 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.723 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.723 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.724 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.724 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.724 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.724 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.724 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.724 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.724 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.724 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.725 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.725 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.725 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.725 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.725 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:43.725 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:43.725 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.725 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.725 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.726 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.726 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.726 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.726 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.726 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.726 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.726 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.726 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.727 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.727 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:43.727 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:43.727 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.727 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.727 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.727 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:43.727 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.727 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.728 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.728 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.728 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.728 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.728 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.728 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.728 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.728 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.728 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.728 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.729 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.729 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.729 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.729 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:43.729 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:43.729 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.729 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.729 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.729 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.730 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.730 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.730 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.730 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.730 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.730 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.730 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.730 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.731 INFO:teuthology.orchestra.run.smithi090.stdout: "max_concurrent_clones": { 2024-04-24T09:24:43.731 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_concurrent_clones", 2024-04-24T09:24:43.731 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:43.731 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.731 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.731 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "4", 2024-04-24T09:24:43.731 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.731 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.732 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.732 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-24T09:24:43.732 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.732 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.732 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.732 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.732 INFO:teuthology.orchestra.run.smithi090.stdout: "periodic_async_work": { 2024-04-24T09:24:43.732 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "periodic_async_work", 2024-04-24T09:24:43.733 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.733 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.733 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.733 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.733 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.733 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.733 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.733 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Periodically check for async work", 2024-04-24T09:24:43.733 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.734 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.734 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.734 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.734 INFO:teuthology.orchestra.run.smithi090.stdout: "snapshot_clone_delay": { 2024-04-24T09:24:43.734 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snapshot_clone_delay", 2024-04-24T09:24:43.734 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:43.734 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.734 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.734 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-04-24T09:24:43.734 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.735 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.735 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.735 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-24T09:24:43.735 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.735 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.735 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.735 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.735 INFO:teuthology.orchestra.run.smithi090.stdout: "snapshot_clone_no_wait": { 2024-04-24T09:24:43.735 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snapshot_clone_no_wait", 2024-04-24T09:24:43.736 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.736 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.736 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.736 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-24T09:24:43.736 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.736 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.736 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.736 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-24T09:24:43.736 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.736 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.737 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.737 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.737 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.737 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.737 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:43.737 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix", 2024-04-24T09:24:43.737 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-24T09:24:43.737 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-24T09:24:43.737 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-24T09:24:43.738 INFO:teuthology.orchestra.run.smithi090.stdout: "discovery_interval": { 2024-04-24T09:24:43.738 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "discovery_interval", 2024-04-24T09:24:43.738 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-04-24T09:24:43.738 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.738 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.738 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "100", 2024-04-24T09:24:43.738 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.738 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.738 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.739 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.739 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.739 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.739 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.739 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.739 INFO:teuthology.orchestra.run.smithi090.stdout: "identifier": { 2024-04-24T09:24:43.739 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "identifier", 2024-04-24T09:24:43.739 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.739 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.740 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.740 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.740 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.740 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.740 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.740 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.740 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.740 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.740 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.741 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.741 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-04-24T09:24:43.741 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-04-24T09:24:43.741 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-24T09:24:43.741 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.741 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.741 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-04-24T09:24:43.741 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.741 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.742 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.742 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.742 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.742 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.742 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.742 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.742 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-24T09:24:43.742 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-24T09:24:43.742 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.742 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.743 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.743 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.743 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.743 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.743 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.743 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.743 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.743 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.743 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.744 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.744 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.744 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.744 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.744 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.744 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.744 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.744 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.744 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-24T09:24:43.744 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-24T09:24:43.745 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.745 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.745 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.745 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.745 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.745 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.745 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.745 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.745 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.746 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.746 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.746 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.746 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-24T09:24:43.746 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-24T09:24:43.746 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.746 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.746 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.746 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-24T09:24:43.747 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.747 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.747 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-24T09:24:43.747 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-24T09:24:43.747 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-24T09:24:43.747 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-24T09:24:43.747 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-24T09:24:43.747 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-24T09:24:43.747 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-24T09:24:43.748 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.748 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.748 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.748 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.748 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.748 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.748 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-24T09:24:43.748 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-24T09:24:43.748 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-24T09:24:43.749 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.749 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-24T09:24:43.749 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-24T09:24:43.749 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.749 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.749 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.749 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.749 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.749 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.750 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.750 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.750 INFO:teuthology.orchestra.run.smithi090.stdout: "zabbix_host": { 2024-04-24T09:24:43.750 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix_host", 2024-04-24T09:24:43.750 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.750 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.750 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.750 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-24T09:24:43.751 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.751 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.751 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.751 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.751 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.751 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.751 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.751 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.751 INFO:teuthology.orchestra.run.smithi090.stdout: "zabbix_port": { 2024-04-24T09:24:43.752 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix_port", 2024-04-24T09:24:43.752 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-24T09:24:43.752 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.752 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.752 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10051", 2024-04-24T09:24:43.752 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.752 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.752 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.752 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.752 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.753 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.753 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.753 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.753 INFO:teuthology.orchestra.run.smithi090.stdout: "zabbix_sender": { 2024-04-24T09:24:43.753 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix_sender", 2024-04-24T09:24:43.753 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-24T09:24:43.753 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-24T09:24:43.753 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-24T09:24:43.753 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-24T09:24:43.754 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-24T09:24:43.754 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-24T09:24:43.754 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-24T09:24:43.754 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-24T09:24:43.754 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-24T09:24:43.754 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-24T09:24:43.754 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-24T09:24:43.754 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.755 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.755 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.755 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.755 INFO:teuthology.orchestra.run.smithi090.stdout: "services": { 2024-04-24T09:24:43.755 INFO:teuthology.orchestra.run.smithi090.stdout: "dashboard": "https://172.21.15.90:8443/" 2024-04-24T09:24:43.755 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.755 INFO:teuthology.orchestra.run.smithi090.stdout: "always_on_modules": { 2024-04-24T09:24:43.755 INFO:teuthology.orchestra.run.smithi090.stdout: "octopus": [ 2024-04-24T09:24:43.756 INFO:teuthology.orchestra.run.smithi090.stdout: "balancer", 2024-04-24T09:24:43.756 INFO:teuthology.orchestra.run.smithi090.stdout: "crash", 2024-04-24T09:24:43.756 INFO:teuthology.orchestra.run.smithi090.stdout: "devicehealth", 2024-04-24T09:24:43.756 INFO:teuthology.orchestra.run.smithi090.stdout: "orchestrator", 2024-04-24T09:24:43.756 INFO:teuthology.orchestra.run.smithi090.stdout: "pg_autoscaler", 2024-04-24T09:24:43.756 INFO:teuthology.orchestra.run.smithi090.stdout: "progress", 2024-04-24T09:24:43.756 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_support", 2024-04-24T09:24:43.756 INFO:teuthology.orchestra.run.smithi090.stdout: "status", 2024-04-24T09:24:43.756 INFO:teuthology.orchestra.run.smithi090.stdout: "telemetry", 2024-04-24T09:24:43.757 INFO:teuthology.orchestra.run.smithi090.stdout: "volumes" 2024-04-24T09:24:43.757 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.757 INFO:teuthology.orchestra.run.smithi090.stdout: "pacific": [ 2024-04-24T09:24:43.757 INFO:teuthology.orchestra.run.smithi090.stdout: "balancer", 2024-04-24T09:24:43.757 INFO:teuthology.orchestra.run.smithi090.stdout: "crash", 2024-04-24T09:24:43.757 INFO:teuthology.orchestra.run.smithi090.stdout: "devicehealth", 2024-04-24T09:24:43.757 INFO:teuthology.orchestra.run.smithi090.stdout: "orchestrator", 2024-04-24T09:24:43.757 INFO:teuthology.orchestra.run.smithi090.stdout: "pg_autoscaler", 2024-04-24T09:24:43.757 INFO:teuthology.orchestra.run.smithi090.stdout: "progress", 2024-04-24T09:24:43.758 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_support", 2024-04-24T09:24:43.758 INFO:teuthology.orchestra.run.smithi090.stdout: "status", 2024-04-24T09:24:43.758 INFO:teuthology.orchestra.run.smithi090.stdout: "telemetry", 2024-04-24T09:24:43.758 INFO:teuthology.orchestra.run.smithi090.stdout: "volumes" 2024-04-24T09:24:43.758 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.758 INFO:teuthology.orchestra.run.smithi090.stdout: "quincy": [ 2024-04-24T09:24:43.758 INFO:teuthology.orchestra.run.smithi090.stdout: "balancer", 2024-04-24T09:24:43.758 INFO:teuthology.orchestra.run.smithi090.stdout: "crash", 2024-04-24T09:24:43.759 INFO:teuthology.orchestra.run.smithi090.stdout: "devicehealth", 2024-04-24T09:24:43.759 INFO:teuthology.orchestra.run.smithi090.stdout: "orchestrator", 2024-04-24T09:24:43.759 INFO:teuthology.orchestra.run.smithi090.stdout: "pg_autoscaler", 2024-04-24T09:24:43.759 INFO:teuthology.orchestra.run.smithi090.stdout: "progress", 2024-04-24T09:24:43.759 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_support", 2024-04-24T09:24:43.759 INFO:teuthology.orchestra.run.smithi090.stdout: "status", 2024-04-24T09:24:43.759 INFO:teuthology.orchestra.run.smithi090.stdout: "telemetry", 2024-04-24T09:24:43.759 INFO:teuthology.orchestra.run.smithi090.stdout: "volumes" 2024-04-24T09:24:43.759 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.760 INFO:teuthology.orchestra.run.smithi090.stdout: "reef": [ 2024-04-24T09:24:43.760 INFO:teuthology.orchestra.run.smithi090.stdout: "balancer", 2024-04-24T09:24:43.760 INFO:teuthology.orchestra.run.smithi090.stdout: "crash", 2024-04-24T09:24:43.760 INFO:teuthology.orchestra.run.smithi090.stdout: "devicehealth", 2024-04-24T09:24:43.760 INFO:teuthology.orchestra.run.smithi090.stdout: "orchestrator", 2024-04-24T09:24:43.760 INFO:teuthology.orchestra.run.smithi090.stdout: "pg_autoscaler", 2024-04-24T09:24:43.760 INFO:teuthology.orchestra.run.smithi090.stdout: "progress", 2024-04-24T09:24:43.760 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_support", 2024-04-24T09:24:43.761 INFO:teuthology.orchestra.run.smithi090.stdout: "status", 2024-04-24T09:24:43.761 INFO:teuthology.orchestra.run.smithi090.stdout: "telemetry", 2024-04-24T09:24:43.761 INFO:teuthology.orchestra.run.smithi090.stdout: "volumes" 2024-04-24T09:24:43.761 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-24T09:24:43.761 INFO:teuthology.orchestra.run.smithi090.stdout: "squid": [ 2024-04-24T09:24:43.761 INFO:teuthology.orchestra.run.smithi090.stdout: "balancer", 2024-04-24T09:24:43.761 INFO:teuthology.orchestra.run.smithi090.stdout: "crash", 2024-04-24T09:24:43.761 INFO:teuthology.orchestra.run.smithi090.stdout: "devicehealth", 2024-04-24T09:24:43.761 INFO:teuthology.orchestra.run.smithi090.stdout: "orchestrator", 2024-04-24T09:24:43.762 INFO:teuthology.orchestra.run.smithi090.stdout: "pg_autoscaler", 2024-04-24T09:24:43.762 INFO:teuthology.orchestra.run.smithi090.stdout: "progress", 2024-04-24T09:24:43.762 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_support", 2024-04-24T09:24:43.762 INFO:teuthology.orchestra.run.smithi090.stdout: "status", 2024-04-24T09:24:43.762 INFO:teuthology.orchestra.run.smithi090.stdout: "telemetry", 2024-04-24T09:24:43.762 INFO:teuthology.orchestra.run.smithi090.stdout: "volumes" 2024-04-24T09:24:43.762 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-04-24T09:24:43.762 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.762 INFO:teuthology.orchestra.run.smithi090.stdout: "last_failure_osd_epoch": 3, 2024-04-24T09:24:43.763 INFO:teuthology.orchestra.run.smithi090.stdout: "active_clients": [ 2024-04-24T09:24:43.763 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:43.763 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "devicehealth", 2024-04-24T09:24:43.763 INFO:teuthology.orchestra.run.smithi090.stdout: "addrvec": [ 2024-04-24T09:24:43.763 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:43.763 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v2", 2024-04-24T09:24:43.763 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "172.21.15.90:0", 2024-04-24T09:24:43.763 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 2457607786 2024-04-24T09:24:43.763 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.763 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-04-24T09:24:43.764 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.764 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:43.764 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "libcephsqlite", 2024-04-24T09:24:43.764 INFO:teuthology.orchestra.run.smithi090.stdout: "addrvec": [ 2024-04-24T09:24:43.764 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:43.764 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v2", 2024-04-24T09:24:43.764 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "172.21.15.90:0", 2024-04-24T09:24:43.764 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 3397803323 2024-04-24T09:24:43.764 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.765 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-04-24T09:24:43.765 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.765 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:43.765 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_support", 2024-04-24T09:24:43.765 INFO:teuthology.orchestra.run.smithi090.stdout: "addrvec": [ 2024-04-24T09:24:43.765 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:43.765 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v2", 2024-04-24T09:24:43.765 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "172.21.15.90:0", 2024-04-24T09:24:43.765 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 1007262564 2024-04-24T09:24:43.766 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.766 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-04-24T09:24:43.766 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:43.766 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:43.766 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "volumes", 2024-04-24T09:24:43.766 INFO:teuthology.orchestra.run.smithi090.stdout: "addrvec": [ 2024-04-24T09:24:43.766 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-24T09:24:43.766 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v2", 2024-04-24T09:24:43.766 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "172.21.15.90:0", 2024-04-24T09:24:43.767 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 2135173549 2024-04-24T09:24:43.767 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.767 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-04-24T09:24:43.767 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:43.767 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-04-24T09:24:43.767 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T09:24:43.773 DEBUG:tasks.check_counter:Getting stats from a 2024-04-24T09:24:43.773 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-24T09:24:43.774 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:24:41 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/2911974189' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:24:43.774 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:24:41 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/4109344076' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-24T09:24:43.774 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:24:42 smithi090 ceph-mon[27825]: pgmap v490: 97 pgs: 97 active+clean; 68 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 237 KiB/s wr, 60 op/s 2024-04-24T09:24:44.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:24:43 smithi090 ceph-mon[27825]: pgmap v491: 97 pgs: 97 active+clean; 68 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 185 KiB/s wr, 37 op/s 2024-04-24T09:24:44.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:24:43 smithi149 ceph-mon[32592]: pgmap v491: 97 pgs: 97 active+clean; 68 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 185 KiB/s wr, 37 op/s 2024-04-24T09:24:44.175 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:24:43 smithi114 ceph-mon[32501]: pgmap v491: 97 pgs: 97 active+clean; 68 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 185 KiB/s wr, 37 op/s 2024-04-24T09:24:44.227 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:24:46.936 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:24:46 smithi090 ceph-mon[27825]: pgmap v492: 97 pgs: 97 active+clean; 68 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-24T09:24:47.026 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T09:24:47.026 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T09:24:47.027 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 182, 2024-04-24T09:24:47.027 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 234, 2024-04-24T09:24:47.027 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 60734, 2024-04-24T09:24:47.027 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 219263, 2024-04-24T09:24:47.027 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 15, 2024-04-24T09:24:47.027 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 6, 2024-04-24T09:24:47.027 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.075379464, 2024-04-24T09:24:47.027 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.037026130, 2024-04-24T09:24:47.027 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.037412306, 2024-04-24T09:24:47.028 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.001936749, 2024-04-24T09:24:47.028 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:24:47.028 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 232, 2024-04-24T09:24:47.028 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.137595064, 2024-04-24T09:24:47.028 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000593082 2024-04-24T09:24:47.028 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.028 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:24:47.028 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 280, 2024-04-24T09:24:47.029 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000089843, 2024-04-24T09:24:47.029 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000320 2024-04-24T09:24:47.029 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.029 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 52112, 2024-04-24T09:24:47.029 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 64432 2024-04-24T09:24:47.029 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.029 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T09:24:47.029 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 28, 2024-04-24T09:24:47.029 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 99, 2024-04-24T09:24:47.029 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 22764, 2024-04-24T09:24:47.030 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 240955, 2024-04-24T09:24:47.030 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 7, 2024-04-24T09:24:47.030 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 5, 2024-04-24T09:24:47.030 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.023311920, 2024-04-24T09:24:47.030 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.012330058, 2024-04-24T09:24:47.030 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.007388785, 2024-04-24T09:24:47.030 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.002126343, 2024-04-24T09:24:47.030 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:24:47.031 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 98, 2024-04-24T09:24:47.031 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.039843266, 2024-04-24T09:24:47.031 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000406563 2024-04-24T09:24:47.031 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.031 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:24:47.031 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 14, 2024-04-24T09:24:47.031 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000003787, 2024-04-24T09:24:47.031 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000270 2024-04-24T09:24:47.031 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.031 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-04-24T09:24:47.032 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 208960 2024-04-24T09:24:47.032 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.032 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T09:24:47.032 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 919, 2024-04-24T09:24:47.032 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 647, 2024-04-24T09:24:47.032 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 1151933, 2024-04-24T09:24:47.032 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 242574, 2024-04-24T09:24:47.032 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 6, 2024-04-24T09:24:47.033 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 5, 2024-04-24T09:24:47.033 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.172483951, 2024-04-24T09:24:47.033 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.066740972, 2024-04-24T09:24:47.033 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.144831957, 2024-04-24T09:24:47.033 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.007114823, 2024-04-24T09:24:47.033 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:24:47.033 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 646, 2024-04-24T09:24:47.033 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.058096415, 2024-04-24T09:24:47.033 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000089932 2024-04-24T09:24:47.034 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.034 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:24:47.034 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 1251, 2024-04-24T09:24:47.034 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000559663, 2024-04-24T09:24:47.034 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000447 2024-04-24T09:24:47.034 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.034 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 798048, 2024-04-24T09:24:47.034 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 64416 2024-04-24T09:24:47.034 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.035 INFO:teuthology.orchestra.run.smithi090.stdout: "cct": { 2024-04-24T09:24:47.035 INFO:teuthology.orchestra.run.smithi090.stdout: "total_workers": 1, 2024-04-24T09:24:47.035 INFO:teuthology.orchestra.run.smithi090.stdout: "unhealthy_workers": 0 2024-04-24T09:24:47.035 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.035 INFO:teuthology.orchestra.run.smithi090.stdout: "finisher-MDSRank": { 2024-04-24T09:24:47.035 INFO:teuthology.orchestra.run.smithi090.stdout: "queue_len": 0, 2024-04-24T09:24:47.035 INFO:teuthology.orchestra.run.smithi090.stdout: "complete_latency": { 2024-04-24T09:24:47.035 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 62, 2024-04-24T09:24:47.035 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.031700162, 2024-04-24T09:24:47.036 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000511292 2024-04-24T09:24:47.036 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:47.036 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.036 INFO:teuthology.orchestra.run.smithi090.stdout: "finisher-PurgeQueue": { 2024-04-24T09:24:47.036 INFO:teuthology.orchestra.run.smithi090.stdout: "queue_len": 0, 2024-04-24T09:24:47.036 INFO:teuthology.orchestra.run.smithi090.stdout: "complete_latency": { 2024-04-24T09:24:47.036 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 2, 2024-04-24T09:24:47.036 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000008417, 2024-04-24T09:24:47.036 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000004208 2024-04-24T09:24:47.036 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:47.037 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.037 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-24T09:24:47.037 INFO:teuthology.orchestra.run.smithi090.stdout: "request": 87, 2024-04-24T09:24:47.037 INFO:teuthology.orchestra.run.smithi090.stdout: "reply": 1, 2024-04-24T09:24:47.037 INFO:teuthology.orchestra.run.smithi090.stdout: "reply_latency": { 2024-04-24T09:24:47.037 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 1, 2024-04-24T09:24:47.037 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000363123, 2024-04-24T09:24:47.037 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000363123 2024-04-24T09:24:47.037 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.037 INFO:teuthology.orchestra.run.smithi090.stdout: "slow_reply": 0, 2024-04-24T09:24:47.038 INFO:teuthology.orchestra.run.smithi090.stdout: "forward": 86, 2024-04-24T09:24:47.038 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_fetch_complete": 0, 2024-04-24T09:24:47.038 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_fetch_keys": 0, 2024-04-24T09:24:47.038 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_commit": 11, 2024-04-24T09:24:47.038 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_split": 0, 2024-04-24T09:24:47.038 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_merge": 0, 2024-04-24T09:24:47.038 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes": 91, 2024-04-24T09:24:47.038 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_top": 10, 2024-04-24T09:24:47.038 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_bottom": 81, 2024-04-24T09:24:47.039 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_pin_tail": 0, 2024-04-24T09:24:47.039 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_pinned": 75, 2024-04-24T09:24:47.039 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_expired": 8, 2024-04-24T09:24:47.039 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_with_caps": 50, 2024-04-24T09:24:47.039 INFO:teuthology.orchestra.run.smithi090.stdout: "caps": 50, 2024-04-24T09:24:47.039 INFO:teuthology.orchestra.run.smithi090.stdout: "subtrees": 4, 2024-04-24T09:24:47.039 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse": 139, 2024-04-24T09:24:47.039 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_hit": 31, 2024-04-24T09:24:47.039 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_forward": 84, 2024-04-24T09:24:47.040 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_discover": 22, 2024-04-24T09:24:47.040 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_dir_fetch": 0, 2024-04-24T09:24:47.040 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_remote_ino": 0, 2024-04-24T09:24:47.040 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_lock": 0, 2024-04-24T09:24:47.040 INFO:teuthology.orchestra.run.smithi090.stdout: "load_cent": 0, 2024-04-24T09:24:47.040 INFO:teuthology.orchestra.run.smithi090.stdout: "q": 0, 2024-04-24T09:24:47.040 INFO:teuthology.orchestra.run.smithi090.stdout: "exported": 0, 2024-04-24T09:24:47.040 INFO:teuthology.orchestra.run.smithi090.stdout: "exported_inodes": 0, 2024-04-24T09:24:47.040 INFO:teuthology.orchestra.run.smithi090.stdout: "imported": 1, 2024-04-24T09:24:47.040 INFO:teuthology.orchestra.run.smithi090.stdout: "imported_inodes": 51, 2024-04-24T09:24:47.041 INFO:teuthology.orchestra.run.smithi090.stdout: "openino_dir_fetch": 0, 2024-04-24T09:24:47.041 INFO:teuthology.orchestra.run.smithi090.stdout: "openino_backtrace_fetch": 0, 2024-04-24T09:24:47.041 INFO:teuthology.orchestra.run.smithi090.stdout: "openino_peer_discover": 0, 2024-04-24T09:24:47.041 INFO:teuthology.orchestra.run.smithi090.stdout: "root_rfiles": 4, 2024-04-24T09:24:47.041 INFO:teuthology.orchestra.run.smithi090.stdout: "root_rbytes": 234, 2024-04-24T09:24:47.041 INFO:teuthology.orchestra.run.smithi090.stdout: "root_rsnaps": 0, 2024-04-24T09:24:47.041 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_backtrace_fetch": 0, 2024-04-24T09:24:47.041 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_set_tag": 0, 2024-04-24T09:24:47.041 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T09:24:47.042 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_inotable_repaired": 0, 2024-04-24T09:24:47.042 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_dir_inodes": 0, 2024-04-24T09:24:47.042 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T09:24:47.042 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_dirfrag_rstats": 0, 2024-04-24T09:24:47.042 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_file_inodes": 0, 2024-04-24T09:24:47.042 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_inode_file_caps": 0, 2024-04-24T09:24:47.042 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_revoke": 0, 2024-04-24T09:24:47.043 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_grant": 0, 2024-04-24T09:24:47.043 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T09:24:47.043 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T09:24:47.043 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T09:24:47.043 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_caps": 50, 2024-04-24T09:24:47.043 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_caps_dirty": 50, 2024-04-24T09:24:47.043 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_cap_release": 0, 2024-04-24T09:24:47.043 INFO:teuthology.orchestra.run.smithi090.stdout: "process_request_cap_release": 2 2024-04-24T09:24:47.043 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.044 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_cache": { 2024-04-24T09:24:47.044 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_update": 0, 2024-04-24T09:24:47.044 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_update_receipt": 77, 2024-04-24T09:24:47.044 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_try_discover": 48, 2024-04-24T09:24:47.044 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_send_discover": 59, 2024-04-24T09:24:47.044 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_handle_discover": 0, 2024-04-24T09:24:47.044 INFO:teuthology.orchestra.run.smithi090.stdout: "num_strays": 0, 2024-04-24T09:24:47.044 INFO:teuthology.orchestra.run.smithi090.stdout: "num_strays_delayed": 0, 2024-04-24T09:24:47.044 INFO:teuthology.orchestra.run.smithi090.stdout: "num_strays_enqueuing": 0, 2024-04-24T09:24:47.045 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_created": 0, 2024-04-24T09:24:47.045 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_enqueued": 0, 2024-04-24T09:24:47.045 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_reintegrated": 0, 2024-04-24T09:24:47.045 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_migrated": 0, 2024-04-24T09:24:47.045 INFO:teuthology.orchestra.run.smithi090.stdout: "num_recovering_processing": 0, 2024-04-24T09:24:47.045 INFO:teuthology.orchestra.run.smithi090.stdout: "num_recovering_enqueued": 0, 2024-04-24T09:24:47.045 INFO:teuthology.orchestra.run.smithi090.stdout: "num_recovering_prioritized": 0, 2024-04-24T09:24:47.045 INFO:teuthology.orchestra.run.smithi090.stdout: "recovery_started": 0, 2024-04-24T09:24:47.045 INFO:teuthology.orchestra.run.smithi090.stdout: "recovery_completed": 0, 2024-04-24T09:24:47.046 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_quiesce_path": 0, 2024-04-24T09:24:47.046 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_quiesce_inode": 0, 2024-04-24T09:24:47.046 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T09:24:47.046 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_exportdir": 0, 2024-04-24T09:24:47.046 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_flush": 0, 2024-04-24T09:24:47.046 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_fragmentdir": 0, 2024-04-24T09:24:47.046 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_fragstats": 0, 2024-04-24T09:24:47.046 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_inodestats": 0 2024-04-24T09:24:47.047 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.047 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_log": { 2024-04-24T09:24:47.047 INFO:teuthology.orchestra.run.smithi090.stdout: "evlrg": 0, 2024-04-24T09:24:47.047 INFO:teuthology.orchestra.run.smithi090.stdout: "evadd": 60, 2024-04-24T09:24:47.047 INFO:teuthology.orchestra.run.smithi090.stdout: "evex": 0, 2024-04-24T09:24:47.047 INFO:teuthology.orchestra.run.smithi090.stdout: "evtrm": 0, 2024-04-24T09:24:47.047 INFO:teuthology.orchestra.run.smithi090.stdout: "ev": 60, 2024-04-24T09:24:47.047 INFO:teuthology.orchestra.run.smithi090.stdout: "evexg": 0, 2024-04-24T09:24:47.047 INFO:teuthology.orchestra.run.smithi090.stdout: "evexd": 0, 2024-04-24T09:24:47.047 INFO:teuthology.orchestra.run.smithi090.stdout: "segadd": 2, 2024-04-24T09:24:47.048 INFO:teuthology.orchestra.run.smithi090.stdout: "segex": 0, 2024-04-24T09:24:47.048 INFO:teuthology.orchestra.run.smithi090.stdout: "segtrm": 0, 2024-04-24T09:24:47.048 INFO:teuthology.orchestra.run.smithi090.stdout: "seg": 2, 2024-04-24T09:24:47.048 INFO:teuthology.orchestra.run.smithi090.stdout: "segmjr": 2, 2024-04-24T09:24:47.048 INFO:teuthology.orchestra.run.smithi090.stdout: "segexg": 0, 2024-04-24T09:24:47.048 INFO:teuthology.orchestra.run.smithi090.stdout: "segexd": 0, 2024-04-24T09:24:47.048 INFO:teuthology.orchestra.run.smithi090.stdout: "expos": 4194304, 2024-04-24T09:24:47.048 INFO:teuthology.orchestra.run.smithi090.stdout: "wrpos": 4296190, 2024-04-24T09:24:47.048 INFO:teuthology.orchestra.run.smithi090.stdout: "rdpos": 0, 2024-04-24T09:24:47.048 INFO:teuthology.orchestra.run.smithi090.stdout: "jlat": { 2024-04-24T09:24:47.049 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 10, 2024-04-24T09:24:47.049 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.027686144, 2024-04-24T09:24:47.049 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.002768614 2024-04-24T09:24:47.049 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.049 INFO:teuthology.orchestra.run.smithi090.stdout: "replayed": 0 2024-04-24T09:24:47.049 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.049 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_mem": { 2024-04-24T09:24:47.049 INFO:teuthology.orchestra.run.smithi090.stdout: "ino": 94, 2024-04-24T09:24:47.049 INFO:teuthology.orchestra.run.smithi090.stdout: "ino+": 99, 2024-04-24T09:24:47.050 INFO:teuthology.orchestra.run.smithi090.stdout: "ino-": 5, 2024-04-24T09:24:47.050 INFO:teuthology.orchestra.run.smithi090.stdout: "dir": 47, 2024-04-24T09:24:47.050 INFO:teuthology.orchestra.run.smithi090.stdout: "dir+": 58, 2024-04-24T09:24:47.050 INFO:teuthology.orchestra.run.smithi090.stdout: "dir-": 11, 2024-04-24T09:24:47.050 INFO:teuthology.orchestra.run.smithi090.stdout: "dn": 91, 2024-04-24T09:24:47.050 INFO:teuthology.orchestra.run.smithi090.stdout: "dn+": 99, 2024-04-24T09:24:47.050 INFO:teuthology.orchestra.run.smithi090.stdout: "dn-": 8, 2024-04-24T09:24:47.050 INFO:teuthology.orchestra.run.smithi090.stdout: "cap": 50, 2024-04-24T09:24:47.051 INFO:teuthology.orchestra.run.smithi090.stdout: "cap+": 51, 2024-04-24T09:24:47.051 INFO:teuthology.orchestra.run.smithi090.stdout: "cap-": 1, 2024-04-24T09:24:47.051 INFO:teuthology.orchestra.run.smithi090.stdout: "rss": 53120, 2024-04-24T09:24:47.051 INFO:teuthology.orchestra.run.smithi090.stdout: "heap": 223608 2024-04-24T09:24:47.051 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.051 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_server": { 2024-04-24T09:24:47.051 INFO:teuthology.orchestra.run.smithi090.stdout: "dispatch_client_request": 89, 2024-04-24T09:24:47.051 INFO:teuthology.orchestra.run.smithi090.stdout: "dispatch_server_request": 0, 2024-04-24T09:24:47.051 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_request": 87, 2024-04-24T09:24:47.052 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_session": 10, 2024-04-24T09:24:47.052 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_peer_request": 0, 2024-04-24T09:24:47.052 INFO:teuthology.orchestra.run.smithi090.stdout: "req_create_latency": { 2024-04-24T09:24:47.052 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:47.052 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:47.052 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:47.052 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.052 INFO:teuthology.orchestra.run.smithi090.stdout: "req_getattr_latency": { 2024-04-24T09:24:47.052 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 1, 2024-04-24T09:24:47.053 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000363123, 2024-04-24T09:24:47.053 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000363123 2024-04-24T09:24:47.053 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.053 INFO:teuthology.orchestra.run.smithi090.stdout: "req_getfilelock_latency": { 2024-04-24T09:24:47.053 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:47.053 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:47.053 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:47.053 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.053 INFO:teuthology.orchestra.run.smithi090.stdout: "req_link_latency": { 2024-04-24T09:24:47.053 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:47.054 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:47.054 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:47.054 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.054 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookup_latency": { 2024-04-24T09:24:47.054 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:47.054 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:47.054 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:47.054 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.054 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookuphash_latency": { 2024-04-24T09:24:47.055 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:47.055 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:47.055 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:47.055 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.055 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupino_latency": { 2024-04-24T09:24:47.055 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:47.055 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:47.055 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:47.055 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.056 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupname_latency": { 2024-04-24T09:24:47.056 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:47.056 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:47.056 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:47.056 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.056 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupparent_latency": { 2024-04-24T09:24:47.056 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:47.056 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:47.056 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:47.056 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.057 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupsnap_latency": { 2024-04-24T09:24:47.057 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:47.057 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:47.057 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:47.057 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.057 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lssnap_latency": { 2024-04-24T09:24:47.057 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:47.057 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:47.057 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:47.057 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.058 INFO:teuthology.orchestra.run.smithi090.stdout: "req_mkdir_latency": { 2024-04-24T09:24:47.058 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:47.058 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:47.059 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:47.059 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.059 INFO:teuthology.orchestra.run.smithi090.stdout: "req_mknod_latency": { 2024-04-24T09:24:47.059 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:47.059 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:47.059 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:47.059 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.059 INFO:teuthology.orchestra.run.smithi090.stdout: "req_mksnap_latency": { 2024-04-24T09:24:47.059 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:47.060 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:47.060 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:47.060 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.060 INFO:teuthology.orchestra.run.smithi090.stdout: "req_open_latency": { 2024-04-24T09:24:47.060 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:47.060 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:47.060 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:47.060 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.060 INFO:teuthology.orchestra.run.smithi090.stdout: "req_readdir_latency": { 2024-04-24T09:24:47.061 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:47.061 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:47.061 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:47.061 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.061 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rename_latency": { 2024-04-24T09:24:47.061 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:47.061 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:47.061 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:47.061 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.062 INFO:teuthology.orchestra.run.smithi090.stdout: "req_renamesnap_latency": { 2024-04-24T09:24:47.062 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:47.062 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:47.062 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:47.062 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.062 INFO:teuthology.orchestra.run.smithi090.stdout: "req_snapdiff_latency": { 2024-04-24T09:24:47.062 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:47.062 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:47.062 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:47.063 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.063 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rmdir_latency": { 2024-04-24T09:24:47.063 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:47.063 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:47.063 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:47.063 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.063 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rmsnap_latency": { 2024-04-24T09:24:47.063 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:47.063 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:47.064 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:47.064 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.064 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rmxattr_latency": { 2024-04-24T09:24:47.064 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:47.064 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:47.064 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:47.064 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.064 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setattr_latency": { 2024-04-24T09:24:47.064 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:47.065 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:47.065 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:47.065 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.065 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setdirlayout_latency": { 2024-04-24T09:24:47.065 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:47.065 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:47.065 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:47.065 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.066 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setfilelock_latency": { 2024-04-24T09:24:47.066 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:47.066 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:47.066 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:47.066 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.066 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setlayout_latency": { 2024-04-24T09:24:47.066 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:47.066 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:47.066 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:47.067 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.067 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setxattr_latency": { 2024-04-24T09:24:47.067 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:47.067 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:47.067 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:47.067 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.067 INFO:teuthology.orchestra.run.smithi090.stdout: "req_symlink_latency": { 2024-04-24T09:24:47.067 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:47.067 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:47.068 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:47.068 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.068 INFO:teuthology.orchestra.run.smithi090.stdout: "req_unlink_latency": { 2024-04-24T09:24:47.068 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:47.068 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:47.068 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:47.068 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.068 INFO:teuthology.orchestra.run.smithi090.stdout: "cap_revoke_eviction": 0, 2024-04-24T09:24:47.068 INFO:teuthology.orchestra.run.smithi090.stdout: "cap_acquisition_throttle": 0, 2024-04-24T09:24:47.068 INFO:teuthology.orchestra.run.smithi090.stdout: "req_getvxattr_latency": { 2024-04-24T09:24:47.069 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:47.069 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:47.069 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:47.069 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:47.069 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.069 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_sessions": { 2024-04-24T09:24:47.069 INFO:teuthology.orchestra.run.smithi090.stdout: "session_count": 1, 2024-04-24T09:24:47.069 INFO:teuthology.orchestra.run.smithi090.stdout: "session_add": 3, 2024-04-24T09:24:47.069 INFO:teuthology.orchestra.run.smithi090.stdout: "session_remove": 2, 2024-04-24T09:24:47.070 INFO:teuthology.orchestra.run.smithi090.stdout: "sessions_open": 1, 2024-04-24T09:24:47.070 INFO:teuthology.orchestra.run.smithi090.stdout: "sessions_stale": 0, 2024-04-24T09:24:47.070 INFO:teuthology.orchestra.run.smithi090.stdout: "total_load": 1, 2024-04-24T09:24:47.070 INFO:teuthology.orchestra.run.smithi090.stdout: "average_load": 1, 2024-04-24T09:24:47.070 INFO:teuthology.orchestra.run.smithi090.stdout: "avg_session_uptime": 75, 2024-04-24T09:24:47.070 INFO:teuthology.orchestra.run.smithi090.stdout: "mdthresh_evicted": 0 2024-04-24T09:24:47.070 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.070 INFO:teuthology.orchestra.run.smithi090.stdout: "mempool": { 2024-04-24T09:24:47.071 INFO:teuthology.orchestra.run.smithi090.stdout: "bloom_filter_bytes": 0, 2024-04-24T09:24:47.071 INFO:teuthology.orchestra.run.smithi090.stdout: "bloom_filter_items": 0, 2024-04-24T09:24:47.071 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T09:24:47.071 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_alloc_items": 0, 2024-04-24T09:24:47.071 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T09:24:47.071 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_data_items": 0, 2024-04-24T09:24:47.071 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T09:24:47.071 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T09:24:47.071 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T09:24:47.071 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T09:24:47.072 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T09:24:47.072 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_other_items": 0, 2024-04-24T09:24:47.072 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T09:24:47.072 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T09:24:47.072 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_extent_bytes": 0, 2024-04-24T09:24:47.072 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_extent_items": 0, 2024-04-24T09:24:47.072 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_blob_bytes": 0, 2024-04-24T09:24:47.072 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_blob_items": 0, 2024-04-24T09:24:47.072 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T09:24:47.072 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T09:24:47.073 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T09:24:47.073 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T09:24:47.073 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T09:24:47.073 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_items": 0, 2024-04-24T09:24:47.073 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_txc_bytes": 0, 2024-04-24T09:24:47.073 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_txc_items": 0, 2024-04-24T09:24:47.073 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T09:24:47.073 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T09:24:47.073 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_bytes": 0, 2024-04-24T09:24:47.073 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_items": 0, 2024-04-24T09:24:47.074 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_bytes": 0, 2024-04-24T09:24:47.074 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_items": 0, 2024-04-24T09:24:47.074 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T09:24:47.074 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_reader_items": 0, 2024-04-24T09:24:47.074 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T09:24:47.074 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_writer_items": 0, 2024-04-24T09:24:47.074 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_anon_bytes": 253892, 2024-04-24T09:24:47.074 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_anon_items": 181, 2024-04-24T09:24:47.074 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_meta_bytes": 0, 2024-04-24T09:24:47.075 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_meta_items": 0, 2024-04-24T09:24:47.075 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_bytes": 0, 2024-04-24T09:24:47.075 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_items": 0, 2024-04-24T09:24:47.075 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mapbl_bytes": 0, 2024-04-24T09:24:47.075 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mapbl_items": 0, 2024-04-24T09:24:47.075 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pglog_bytes": 0, 2024-04-24T09:24:47.075 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pglog_items": 0, 2024-04-24T09:24:47.075 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_bytes": 7624, 2024-04-24T09:24:47.075 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_items": 160, 2024-04-24T09:24:47.075 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T09:24:47.076 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_mapping_items": 0, 2024-04-24T09:24:47.076 INFO:teuthology.orchestra.run.smithi090.stdout: "pgmap_bytes": 0, 2024-04-24T09:24:47.076 INFO:teuthology.orchestra.run.smithi090.stdout: "pgmap_items": 0, 2024-04-24T09:24:47.076 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_co_bytes": 363186, 2024-04-24T09:24:47.076 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_co_items": 3122, 2024-04-24T09:24:47.076 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_1_bytes": 0, 2024-04-24T09:24:47.076 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_1_items": 0, 2024-04-24T09:24:47.076 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_2_bytes": 0, 2024-04-24T09:24:47.076 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_2_items": 0 2024-04-24T09:24:47.077 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.077 INFO:teuthology.orchestra.run.smithi090.stdout: "objecter": { 2024-04-24T09:24:47.077 INFO:teuthology.orchestra.run.smithi090.stdout: "op_active": 0, 2024-04-24T09:24:47.077 INFO:teuthology.orchestra.run.smithi090.stdout: "op_laggy": 0, 2024-04-24T09:24:47.077 INFO:teuthology.orchestra.run.smithi090.stdout: "op_send": 52, 2024-04-24T09:24:47.077 INFO:teuthology.orchestra.run.smithi090.stdout: "op_send_bytes": 117536, 2024-04-24T09:24:47.077 INFO:teuthology.orchestra.run.smithi090.stdout: "op_resend": 33, 2024-04-24T09:24:47.077 INFO:teuthology.orchestra.run.smithi090.stdout: "op_reply": 52, 2024-04-24T09:24:47.077 INFO:teuthology.orchestra.run.smithi090.stdout: "op_latency": { 2024-04-24T09:24:47.078 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 52, 2024-04-24T09:24:47.078 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.481992906, 2024-04-24T09:24:47.078 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.009269094 2024-04-24T09:24:47.078 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.078 INFO:teuthology.orchestra.run.smithi090.stdout: "op_inflight": 0, 2024-04-24T09:24:47.078 INFO:teuthology.orchestra.run.smithi090.stdout: "oplen_avg": { 2024-04-24T09:24:47.078 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 52, 2024-04-24T09:24:47.078 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 76 2024-04-24T09:24:47.078 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.078 INFO:teuthology.orchestra.run.smithi090.stdout: "op": 52, 2024-04-24T09:24:47.079 INFO:teuthology.orchestra.run.smithi090.stdout: "op_r": 0, 2024-04-24T09:24:47.079 INFO:teuthology.orchestra.run.smithi090.stdout: "op_w": 52, 2024-04-24T09:24:47.079 INFO:teuthology.orchestra.run.smithi090.stdout: "op_rmw": 0, 2024-04-24T09:24:47.079 INFO:teuthology.orchestra.run.smithi090.stdout: "op_pg": 0, 2024-04-24T09:24:47.079 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_stat": 0, 2024-04-24T09:24:47.079 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_create": 10, 2024-04-24T09:24:47.079 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_read": 0, 2024-04-24T09:24:47.079 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_write": 10, 2024-04-24T09:24:47.079 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_writefull": 11, 2024-04-24T09:24:47.080 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_writesame": 0, 2024-04-24T09:24:47.080 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_append": 0, 2024-04-24T09:24:47.080 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_zero": 0, 2024-04-24T09:24:47.080 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_truncate": 0, 2024-04-24T09:24:47.080 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_delete": 6, 2024-04-24T09:24:47.080 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_mapext": 0, 2024-04-24T09:24:47.080 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_sparse_read": 0, 2024-04-24T09:24:47.080 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_clonerange": 0, 2024-04-24T09:24:47.080 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_getxattr": 0, 2024-04-24T09:24:47.081 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_setxattr": 20, 2024-04-24T09:24:47.081 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_cmpxattr": 0, 2024-04-24T09:24:47.081 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_rmxattr": 0, 2024-04-24T09:24:47.081 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_resetxattrs": 0, 2024-04-24T09:24:47.081 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_call": 0, 2024-04-24T09:24:47.081 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_watch": 0, 2024-04-24T09:24:47.081 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_notify": 0, 2024-04-24T09:24:47.081 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T09:24:47.081 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_pgls": 0, 2024-04-24T09:24:47.081 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_pgls_filter": 0, 2024-04-24T09:24:47.082 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_other": 0, 2024-04-24T09:24:47.082 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_active": 0, 2024-04-24T09:24:47.082 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_send": 0, 2024-04-24T09:24:47.082 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_resend": 0, 2024-04-24T09:24:47.082 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_ping": 0, 2024-04-24T09:24:47.082 INFO:teuthology.orchestra.run.smithi090.stdout: "poolop_active": 0, 2024-04-24T09:24:47.082 INFO:teuthology.orchestra.run.smithi090.stdout: "poolop_send": 0, 2024-04-24T09:24:47.082 INFO:teuthology.orchestra.run.smithi090.stdout: "poolop_resend": 0, 2024-04-24T09:24:47.082 INFO:teuthology.orchestra.run.smithi090.stdout: "poolstat_active": 0, 2024-04-24T09:24:47.083 INFO:teuthology.orchestra.run.smithi090.stdout: "poolstat_send": 0, 2024-04-24T09:24:47.083 INFO:teuthology.orchestra.run.smithi090.stdout: "poolstat_resend": 0, 2024-04-24T09:24:47.083 INFO:teuthology.orchestra.run.smithi090.stdout: "statfs_active": 0, 2024-04-24T09:24:47.083 INFO:teuthology.orchestra.run.smithi090.stdout: "statfs_send": 0, 2024-04-24T09:24:47.083 INFO:teuthology.orchestra.run.smithi090.stdout: "statfs_resend": 0, 2024-04-24T09:24:47.083 INFO:teuthology.orchestra.run.smithi090.stdout: "command_active": 0, 2024-04-24T09:24:47.083 INFO:teuthology.orchestra.run.smithi090.stdout: "command_send": 0, 2024-04-24T09:24:47.083 INFO:teuthology.orchestra.run.smithi090.stdout: "command_resend": 0, 2024-04-24T09:24:47.083 INFO:teuthology.orchestra.run.smithi090.stdout: "map_epoch": 88, 2024-04-24T09:24:47.083 INFO:teuthology.orchestra.run.smithi090.stdout: "map_full": 0, 2024-04-24T09:24:47.084 INFO:teuthology.orchestra.run.smithi090.stdout: "map_inc": 2, 2024-04-24T09:24:47.084 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_sessions": 8, 2024-04-24T09:24:47.084 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_session_open": 8, 2024-04-24T09:24:47.084 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_session_close": 0, 2024-04-24T09:24:47.084 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_laggy": 0, 2024-04-24T09:24:47.084 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_wr": 17, 2024-04-24T09:24:47.084 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_rd": 0, 2024-04-24T09:24:47.084 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_del": 2 2024-04-24T09:24:47.084 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.085 INFO:teuthology.orchestra.run.smithi090.stdout: "oft": { 2024-04-24T09:24:47.085 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_objs": 1, 2024-04-24T09:24:47.085 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_kv_pairs": 0, 2024-04-24T09:24:47.085 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_updates": 0, 2024-04-24T09:24:47.085 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_removes": 0 2024-04-24T09:24:47.085 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.085 INFO:teuthology.orchestra.run.smithi090.stdout: "osd-slow-ops": { 2024-04-24T09:24:47.085 INFO:teuthology.orchestra.run.smithi090.stdout: "slow_ops_count": 0 2024-04-24T09:24:47.085 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.086 INFO:teuthology.orchestra.run.smithi090.stdout: "purge_queue": { 2024-04-24T09:24:47.086 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing_ops": 0, 2024-04-24T09:24:47.086 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing_ops_high_water": 0, 2024-04-24T09:24:47.086 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing": 0, 2024-04-24T09:24:47.086 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing_high_water": 0, 2024-04-24T09:24:47.086 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executed_ops": 0, 2024-04-24T09:24:47.086 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executed": 0, 2024-04-24T09:24:47.086 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_item_in_journal": 0 2024-04-24T09:24:47.086 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.087 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T09:24:47.087 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-04-24T09:24:47.087 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 104857600, 2024-04-24T09:24:47.087 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-04-24T09:24:47.087 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 1125, 2024-04-24T09:24:47.087 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 1117397, 2024-04-24T09:24:47.087 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-04-24T09:24:47.087 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 1125, 2024-04-24T09:24:47.087 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 0, 2024-04-24T09:24:47.088 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 0, 2024-04-24T09:24:47.088 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 1125, 2024-04-24T09:24:47.088 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 1117397, 2024-04-24T09:24:47.088 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-04-24T09:24:47.088 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:47.088 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:47.088 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:47.088 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:47.088 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.089 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-objecter_bytes": { 2024-04-24T09:24:47.089 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-04-24T09:24:47.089 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 104857600, 2024-04-24T09:24:47.089 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-04-24T09:24:47.089 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 0, 2024-04-24T09:24:47.089 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 0, 2024-04-24T09:24:47.089 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-04-24T09:24:47.089 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 0, 2024-04-24T09:24:47.089 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 52, 2024-04-24T09:24:47.089 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 117536, 2024-04-24T09:24:47.090 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 52, 2024-04-24T09:24:47.090 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 117536, 2024-04-24T09:24:47.090 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-04-24T09:24:47.090 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:47.090 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:47.090 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:47.090 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:47.090 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.090 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-objecter_ops": { 2024-04-24T09:24:47.091 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-04-24T09:24:47.091 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 1024, 2024-04-24T09:24:47.091 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-04-24T09:24:47.091 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 0, 2024-04-24T09:24:47.091 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 0, 2024-04-24T09:24:47.091 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-04-24T09:24:47.091 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 0, 2024-04-24T09:24:47.091 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 52, 2024-04-24T09:24:47.091 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 52, 2024-04-24T09:24:47.092 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 52, 2024-04-24T09:24:47.092 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 52, 2024-04-24T09:24:47.092 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-04-24T09:24:47.092 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:47.092 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:47.092 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:47.092 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:47.092 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.092 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-write_buf_throttle": { 2024-04-24T09:24:47.092 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-04-24T09:24:47.093 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 3758096384, 2024-04-24T09:24:47.093 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-04-24T09:24:47.093 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 0, 2024-04-24T09:24:47.093 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 0, 2024-04-24T09:24:47.093 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-04-24T09:24:47.093 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 0, 2024-04-24T09:24:47.093 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 0, 2024-04-24T09:24:47.093 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 0, 2024-04-24T09:24:47.093 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 0, 2024-04-24T09:24:47.094 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 0, 2024-04-24T09:24:47.094 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-04-24T09:24:47.094 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:47.094 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:47.094 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:47.094 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:47.094 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:47.094 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-write_buf_throttle-0x5574b36bc500": { 2024-04-24T09:24:47.095 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-04-24T09:24:47.095 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 3758096384, 2024-04-24T09:24:47.095 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-04-24T09:24:47.095 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 60, 2024-04-24T09:24:47.095 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 101886, 2024-04-24T09:24:47.095 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-04-24T09:24:47.095 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 60, 2024-04-24T09:24:47.095 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 0, 2024-04-24T09:24:47.095 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 0, 2024-04-24T09:24:47.096 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 10, 2024-04-24T09:24:47.096 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 101886, 2024-04-24T09:24:47.096 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-04-24T09:24:47.096 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:47.096 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:47.096 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:47.096 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:47.096 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:47.096 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T09:24:47.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:24:46 smithi149 ceph-mon[32592]: pgmap v492: 97 pgs: 97 active+clean; 68 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-24T09:24:47.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:24:46 smithi114 ceph-mon[32501]: pgmap v492: 97 pgs: 97 active+clean; 68 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-24T09:24:48.185 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-04-24T09:24:48.186 INFO:tasks.check_counter:Daemon mds.a mds.imported=1 2024-04-24T09:24:48.186 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-04-24T09:24:48.186 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=77 2024-04-24T09:24:48.186 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=0 2024-04-24T09:24:48.186 DEBUG:tasks.check_counter:Getting stats from d 2024-04-24T09:24:48.186 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-24T09:24:48.638 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:24:49.012 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:24:48 smithi090 ceph-mon[27825]: pgmap v493: 97 pgs: 97 active+clean; 68 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-04-24T09:24:49.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:24:48 smithi149 ceph-mon[32592]: pgmap v493: 97 pgs: 97 active+clean; 68 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-04-24T09:24:49.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:24:48 smithi114 ceph-mon[32501]: pgmap v493: 97 pgs: 97 active+clean; 68 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-04-24T09:24:50.759 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:24:50 smithi090 ceph-mon[27825]: pgmap v494: 97 pgs: 97 active+clean; 70 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 129 KiB/s wr, 21 op/s 2024-04-24T09:24:51.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:24:50 smithi149 ceph-mon[32592]: pgmap v494: 97 pgs: 97 active+clean; 70 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 129 KiB/s wr, 21 op/s 2024-04-24T09:24:51.116 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T09:24:51.116 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T09:24:51.117 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 108, 2024-04-24T09:24:51.117 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 103, 2024-04-24T09:24:51.117 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 726112, 2024-04-24T09:24:51.117 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 62608, 2024-04-24T09:24:51.117 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 4, 2024-04-24T09:24:51.117 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 0, 2024-04-24T09:24:51.117 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.048183262, 2024-04-24T09:24:51.117 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.015146175, 2024-04-24T09:24:51.117 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.036187093, 2024-04-24T09:24:51.118 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.004418321, 2024-04-24T09:24:51.118 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:24:51.118 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 102, 2024-04-24T09:24:51.118 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.011122644, 2024-04-24T09:24:51.118 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000109045 2024-04-24T09:24:51.118 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:51.118 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:24:51.118 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:51.119 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:51.119 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:51.119 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:51.119 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 726112, 2024-04-24T09:24:51.119 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 62608 2024-04-24T09:24:51.119 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:51.119 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T09:24:51.119 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 5, 2024-04-24T09:24:51.119 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 80, 2024-04-24T09:24:51.120 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 3232, 2024-04-24T09:24:51.120 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 92288, 2024-04-24T09:24:51.120 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 3, 2024-04-24T09:24:51.120 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 2, 2024-04-24T09:24:51.120 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.014876936, 2024-04-24T09:24:51.120 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.010844940, 2024-04-24T09:24:51.120 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.002319514, 2024-04-24T09:24:51.120 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T09:24:51.120 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:24:51.121 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 79, 2024-04-24T09:24:51.121 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.009007467, 2024-04-24T09:24:51.121 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000114018 2024-04-24T09:24:51.121 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:51.121 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:24:51.121 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:51.121 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:51.121 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:51.122 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:51.122 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-04-24T09:24:51.122 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 92288 2024-04-24T09:24:51.122 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:51.122 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T09:24:51.122 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 0, 2024-04-24T09:24:51.122 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 2, 2024-04-24T09:24:51.122 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 0, 2024-04-24T09:24:51.122 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 0, 2024-04-24T09:24:51.123 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 2, 2024-04-24T09:24:51.123 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 1, 2024-04-24T09:24:51.123 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.001533140, 2024-04-24T09:24:51.123 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.000000000, 2024-04-24T09:24:51.123 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.001312671, 2024-04-24T09:24:51.123 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T09:24:51.123 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:24:51.123 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:51.123 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:51.124 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:51.124 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:51.124 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:24:51.124 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:51.124 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:51.124 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:51.124 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:51.124 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T09:24:51.124 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T09:24:51.125 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:51.125 INFO:teuthology.orchestra.run.smithi090.stdout: "cct": { 2024-04-24T09:24:51.125 INFO:teuthology.orchestra.run.smithi090.stdout: "total_workers": 0, 2024-04-24T09:24:51.125 INFO:teuthology.orchestra.run.smithi090.stdout: "unhealthy_workers": 0 2024-04-24T09:24:51.125 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:51.125 INFO:teuthology.orchestra.run.smithi090.stdout: "mempool": { 2024-04-24T09:24:51.125 INFO:teuthology.orchestra.run.smithi090.stdout: "bloom_filter_bytes": 0, 2024-04-24T09:24:51.125 INFO:teuthology.orchestra.run.smithi090.stdout: "bloom_filter_items": 0, 2024-04-24T09:24:51.125 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T09:24:51.126 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_alloc_items": 0, 2024-04-24T09:24:51.126 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T09:24:51.126 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_data_items": 0, 2024-04-24T09:24:51.126 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T09:24:51.126 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T09:24:51.126 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T09:24:51.126 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T09:24:51.126 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T09:24:51.127 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_other_items": 0, 2024-04-24T09:24:51.127 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T09:24:51.127 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T09:24:51.127 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_extent_bytes": 0, 2024-04-24T09:24:51.127 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_extent_items": 0, 2024-04-24T09:24:51.127 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_blob_bytes": 0, 2024-04-24T09:24:51.127 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_blob_items": 0, 2024-04-24T09:24:51.127 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T09:24:51.128 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T09:24:51.128 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T09:24:51.128 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T09:24:51.128 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T09:24:51.128 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_items": 0, 2024-04-24T09:24:51.128 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_txc_bytes": 0, 2024-04-24T09:24:51.128 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_txc_items": 0, 2024-04-24T09:24:51.128 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T09:24:51.128 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T09:24:51.129 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_bytes": 0, 2024-04-24T09:24:51.129 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_items": 0, 2024-04-24T09:24:51.129 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_bytes": 0, 2024-04-24T09:24:51.129 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_items": 0, 2024-04-24T09:24:51.129 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T09:24:51.129 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_reader_items": 0, 2024-04-24T09:24:51.129 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T09:24:51.129 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_writer_items": 0, 2024-04-24T09:24:51.129 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_anon_bytes": 149854, 2024-04-24T09:24:51.130 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_anon_items": 28, 2024-04-24T09:24:51.130 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_meta_bytes": 0, 2024-04-24T09:24:51.130 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_meta_items": 0, 2024-04-24T09:24:51.130 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_bytes": 0, 2024-04-24T09:24:51.130 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_items": 0, 2024-04-24T09:24:51.130 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mapbl_bytes": 0, 2024-04-24T09:24:51.130 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mapbl_items": 0, 2024-04-24T09:24:51.130 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pglog_bytes": 0, 2024-04-24T09:24:51.131 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pglog_items": 0, 2024-04-24T09:24:51.131 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_bytes": 0, 2024-04-24T09:24:51.131 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_items": 0, 2024-04-24T09:24:51.131 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T09:24:51.131 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_mapping_items": 0, 2024-04-24T09:24:51.131 INFO:teuthology.orchestra.run.smithi090.stdout: "pgmap_bytes": 0, 2024-04-24T09:24:51.131 INFO:teuthology.orchestra.run.smithi090.stdout: "pgmap_items": 0, 2024-04-24T09:24:51.131 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_co_bytes": 928, 2024-04-24T09:24:51.131 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_co_items": 2, 2024-04-24T09:24:51.132 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_1_bytes": 0, 2024-04-24T09:24:51.132 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_1_items": 0, 2024-04-24T09:24:51.132 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_2_bytes": 0, 2024-04-24T09:24:51.132 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_2_items": 0 2024-04-24T09:24:51.132 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:51.132 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T09:24:51.132 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-04-24T09:24:51.132 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 104857600, 2024-04-24T09:24:51.133 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-04-24T09:24:51.133 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 109, 2024-04-24T09:24:51.133 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 710901, 2024-04-24T09:24:51.133 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-04-24T09:24:51.133 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 109, 2024-04-24T09:24:51.133 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 0, 2024-04-24T09:24:51.133 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 0, 2024-04-24T09:24:51.133 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 109, 2024-04-24T09:24:51.133 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 710901, 2024-04-24T09:24:51.134 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-04-24T09:24:51.134 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:51.134 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:51.134 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:51.134 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:51.134 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:51.134 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T09:24:51.175 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:24:50 smithi114 ceph-mon[32501]: pgmap v494: 97 pgs: 97 active+clean; 70 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 129 KiB/s wr, 21 op/s 2024-04-24T09:24:52.077 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-24T09:24:52.077 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-24T09:24:52.077 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-04-24T09:24:52.077 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-04-24T09:24:52.077 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-04-24T09:24:52.077 DEBUG:tasks.check_counter:Getting stats from g 2024-04-24T09:24:52.077 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-24T09:24:52.530 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:24:52.946 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:24:52 smithi090 ceph-mon[27825]: pgmap v495: 97 pgs: 97 active+clean; 71 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 245 KiB/s wr, 36 op/s 2024-04-24T09:24:53.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:24:52 smithi149 ceph-mon[32592]: pgmap v495: 97 pgs: 97 active+clean; 71 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 245 KiB/s wr, 36 op/s 2024-04-24T09:24:53.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:24:52 smithi114 ceph-mon[32501]: pgmap v495: 97 pgs: 97 active+clean; 71 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 245 KiB/s wr, 36 op/s 2024-04-24T09:24:55.008 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:24:54 smithi090 ceph-mon[27825]: pgmap v496: 97 pgs: 97 active+clean; 71 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 288 KiB/s wr, 47 op/s 2024-04-24T09:24:55.011 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T09:24:55.011 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T09:24:55.011 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 0, 2024-04-24T09:24:55.011 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 2, 2024-04-24T09:24:55.012 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 0, 2024-04-24T09:24:55.012 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 0, 2024-04-24T09:24:55.012 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 4, 2024-04-24T09:24:55.012 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 0, 2024-04-24T09:24:55.012 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.003472379, 2024-04-24T09:24:55.012 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.000000000, 2024-04-24T09:24:55.012 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.000590645, 2024-04-24T09:24:55.012 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T09:24:55.012 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:24:55.013 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:55.013 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:55.013 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:55.013 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:55.013 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:24:55.013 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:55.013 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:55.013 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:55.013 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:55.013 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T09:24:55.014 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T09:24:55.014 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:55.014 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T09:24:55.014 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 107, 2024-04-24T09:24:55.014 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 100, 2024-04-24T09:24:55.014 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 727200, 2024-04-24T09:24:55.014 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 60160, 2024-04-24T09:24:55.014 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 2, 2024-04-24T09:24:55.015 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 1, 2024-04-24T09:24:55.015 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.040724700, 2024-04-24T09:24:55.015 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.014063747, 2024-04-24T09:24:55.015 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.030501676, 2024-04-24T09:24:55.015 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.004230231, 2024-04-24T09:24:55.015 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:24:55.015 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 100, 2024-04-24T09:24:55.015 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.011042596, 2024-04-24T09:24:55.015 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000110425 2024-04-24T09:24:55.016 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:55.016 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:24:55.016 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:55.016 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:55.016 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:55.016 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:55.016 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 727200, 2024-04-24T09:24:55.016 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 60160 2024-04-24T09:24:55.016 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:55.016 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T09:24:55.017 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 1, 2024-04-24T09:24:55.017 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 75, 2024-04-24T09:24:55.017 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 160, 2024-04-24T09:24:55.017 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 91312, 2024-04-24T09:24:55.017 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 3, 2024-04-24T09:24:55.017 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 2, 2024-04-24T09:24:55.017 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.014764193, 2024-04-24T09:24:55.017 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.011046577, 2024-04-24T09:24:55.017 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.001961207, 2024-04-24T09:24:55.017 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T09:24:55.018 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:24:55.018 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 73, 2024-04-24T09:24:55.018 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.006605680, 2024-04-24T09:24:55.018 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000090488 2024-04-24T09:24:55.018 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:55.018 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:24:55.018 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:55.018 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:55.018 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:55.019 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:55.019 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-24T09:24:55.019 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 91312 2024-04-24T09:24:55.019 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:55.019 INFO:teuthology.orchestra.run.smithi090.stdout: "cct": { 2024-04-24T09:24:55.019 INFO:teuthology.orchestra.run.smithi090.stdout: "total_workers": 0, 2024-04-24T09:24:55.019 INFO:teuthology.orchestra.run.smithi090.stdout: "unhealthy_workers": 0 2024-04-24T09:24:55.019 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:55.020 INFO:teuthology.orchestra.run.smithi090.stdout: "mempool": { 2024-04-24T09:24:55.020 INFO:teuthology.orchestra.run.smithi090.stdout: "bloom_filter_bytes": 0, 2024-04-24T09:24:55.020 INFO:teuthology.orchestra.run.smithi090.stdout: "bloom_filter_items": 0, 2024-04-24T09:24:55.020 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T09:24:55.020 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_alloc_items": 0, 2024-04-24T09:24:55.020 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T09:24:55.020 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_data_items": 0, 2024-04-24T09:24:55.020 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T09:24:55.020 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T09:24:55.021 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T09:24:55.021 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T09:24:55.021 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T09:24:55.021 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_other_items": 0, 2024-04-24T09:24:55.021 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T09:24:55.021 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T09:24:55.021 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_extent_bytes": 0, 2024-04-24T09:24:55.021 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_extent_items": 0, 2024-04-24T09:24:55.021 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_blob_bytes": 0, 2024-04-24T09:24:55.022 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_blob_items": 0, 2024-04-24T09:24:55.022 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T09:24:55.022 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T09:24:55.022 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T09:24:55.022 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T09:24:55.022 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T09:24:55.022 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_items": 0, 2024-04-24T09:24:55.022 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_txc_bytes": 0, 2024-04-24T09:24:55.022 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_txc_items": 0, 2024-04-24T09:24:55.023 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T09:24:55.023 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T09:24:55.023 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_bytes": 0, 2024-04-24T09:24:55.023 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_items": 0, 2024-04-24T09:24:55.023 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_bytes": 0, 2024-04-24T09:24:55.023 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_items": 0, 2024-04-24T09:24:55.023 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T09:24:55.023 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_reader_items": 0, 2024-04-24T09:24:55.023 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T09:24:55.023 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_writer_items": 0, 2024-04-24T09:24:55.024 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_anon_bytes": 148823, 2024-04-24T09:24:55.024 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_anon_items": 27, 2024-04-24T09:24:55.024 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_meta_bytes": 0, 2024-04-24T09:24:55.024 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_meta_items": 0, 2024-04-24T09:24:55.024 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_bytes": 0, 2024-04-24T09:24:55.024 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_items": 0, 2024-04-24T09:24:55.024 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mapbl_bytes": 0, 2024-04-24T09:24:55.024 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mapbl_items": 0, 2024-04-24T09:24:55.024 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pglog_bytes": 0, 2024-04-24T09:24:55.025 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pglog_items": 0, 2024-04-24T09:24:55.025 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_bytes": 0, 2024-04-24T09:24:55.025 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_items": 0, 2024-04-24T09:24:55.025 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T09:24:55.025 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_mapping_items": 0, 2024-04-24T09:24:55.025 INFO:teuthology.orchestra.run.smithi090.stdout: "pgmap_bytes": 0, 2024-04-24T09:24:55.025 INFO:teuthology.orchestra.run.smithi090.stdout: "pgmap_items": 0, 2024-04-24T09:24:55.025 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_co_bytes": 928, 2024-04-24T09:24:55.025 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_co_items": 2, 2024-04-24T09:24:55.026 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_1_bytes": 0, 2024-04-24T09:24:55.026 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_1_items": 0, 2024-04-24T09:24:55.026 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_2_bytes": 0, 2024-04-24T09:24:55.026 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_2_items": 0 2024-04-24T09:24:55.026 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:55.026 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T09:24:55.026 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-04-24T09:24:55.026 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 104857600, 2024-04-24T09:24:55.026 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-04-24T09:24:55.027 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 104, 2024-04-24T09:24:55.027 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 709614, 2024-04-24T09:24:55.027 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-04-24T09:24:55.027 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 104, 2024-04-24T09:24:55.027 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 0, 2024-04-24T09:24:55.027 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 0, 2024-04-24T09:24:55.027 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 104, 2024-04-24T09:24:55.027 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 709614, 2024-04-24T09:24:55.028 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-04-24T09:24:55.028 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:55.028 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:55.028 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:55.028 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:55.028 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:55.028 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T09:24:55.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:24:54 smithi149 ceph-mon[32592]: pgmap v496: 97 pgs: 97 active+clean; 71 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 288 KiB/s wr, 47 op/s 2024-04-24T09:24:55.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:24:54 smithi114 ceph-mon[32501]: pgmap v496: 97 pgs: 97 active+clean; 71 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 288 KiB/s wr, 47 op/s 2024-04-24T09:24:55.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:24:55 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:24:56.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:24:55 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:24:56.145 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-04-24T09:24:56.145 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-04-24T09:24:56.145 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-04-24T09:24:56.145 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-04-24T09:24:56.145 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-04-24T09:24:56.145 DEBUG:tasks.check_counter:Getting stats from j 2024-04-24T09:24:56.146 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-24T09:24:56.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:24:55 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:24:56.589 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:24:57.083 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:24:56 smithi090 ceph-mon[27825]: pgmap v497: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 281 KiB/s wr, 47 op/s 2024-04-24T09:24:57.083 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:24:56 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:24:57.083 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:24:56 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:24:57.083 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:24:56 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:24:57.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:24:56 smithi149 ceph-mon[32592]: pgmap v497: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 281 KiB/s wr, 47 op/s 2024-04-24T09:24:57.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:24:56 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:24:57.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:24:56 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:24:57.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:24:56 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:24:57.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:24:56 smithi114 ceph-mon[32501]: pgmap v497: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 281 KiB/s wr, 47 op/s 2024-04-24T09:24:57.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:24:56 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:24:57.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:24:56 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:24:57.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:24:56 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' 2024-04-24T09:24:59.085 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:24:58 smithi090 ceph-mon[27825]: pgmap v498: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 281 KiB/s wr, 46 op/s 2024-04-24T09:24:59.092 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-24T09:24:59.092 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T09:24:59.092 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 766, 2024-04-24T09:24:59.092 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 514, 2024-04-24T09:24:59.093 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 292548, 2024-04-24T09:24:59.093 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 178441, 2024-04-24T09:24:59.093 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 9, 2024-04-24T09:24:59.093 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 4, 2024-04-24T09:24:59.093 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.134694828, 2024-04-24T09:24:59.093 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.055688101, 2024-04-24T09:24:59.093 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.094781458, 2024-04-24T09:24:59.093 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.003529461, 2024-04-24T09:24:59.093 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:24:59.094 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 513, 2024-04-24T09:24:59.094 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.066379826, 2024-04-24T09:24:59.094 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000129395 2024-04-24T09:24:59.094 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.094 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:24:59.094 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 1218, 2024-04-24T09:24:59.094 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000491178, 2024-04-24T09:24:59.094 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000403 2024-04-24T09:24:59.095 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.095 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-04-24T09:24:59.095 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 448 2024-04-24T09:24:59.095 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.095 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T09:24:59.095 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 24, 2024-04-24T09:24:59.095 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 103, 2024-04-24T09:24:59.095 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 18881, 2024-04-24T09:24:59.095 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 283341, 2024-04-24T09:24:59.096 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 13, 2024-04-24T09:24:59.096 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 7, 2024-04-24T09:24:59.096 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.029340323, 2024-04-24T09:24:59.096 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.013307431, 2024-04-24T09:24:59.096 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.014342986, 2024-04-24T09:24:59.096 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.000321850, 2024-04-24T09:24:59.096 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:24:59.096 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 101, 2024-04-24T09:24:59.096 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.054842692, 2024-04-24T09:24:59.097 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000542996 2024-04-24T09:24:59.097 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.097 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:24:59.097 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 21, 2024-04-24T09:24:59.097 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000004420, 2024-04-24T09:24:59.097 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000210 2024-04-24T09:24:59.097 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.097 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-04-24T09:24:59.097 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 212096 2024-04-24T09:24:59.098 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.098 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T09:24:59.098 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 175, 2024-04-24T09:24:59.098 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 164, 2024-04-24T09:24:59.098 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 793633, 2024-04-24T09:24:59.098 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 79285, 2024-04-24T09:24:59.098 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 6, 2024-04-24T09:24:59.098 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 5, 2024-04-24T09:24:59.098 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.062088817, 2024-04-24T09:24:59.099 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.016561199, 2024-04-24T09:24:59.099 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.063655546, 2024-04-24T09:24:59.099 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.005382624, 2024-04-24T09:24:59.099 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:24:59.099 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 163, 2024-04-24T09:24:59.099 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.067644585, 2024-04-24T09:24:59.099 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000414997 2024-04-24T09:24:59.099 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.100 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:24:59.100 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 11, 2024-04-24T09:24:59.100 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000003535, 2024-04-24T09:24:59.100 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000321 2024-04-24T09:24:59.100 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.100 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 789552, 2024-04-24T09:24:59.100 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 71920 2024-04-24T09:24:59.100 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.100 INFO:teuthology.orchestra.run.smithi090.stdout: "cct": { 2024-04-24T09:24:59.101 INFO:teuthology.orchestra.run.smithi090.stdout: "total_workers": 1, 2024-04-24T09:24:59.101 INFO:teuthology.orchestra.run.smithi090.stdout: "unhealthy_workers": 0 2024-04-24T09:24:59.101 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.101 INFO:teuthology.orchestra.run.smithi090.stdout: "finisher-MDSRank": { 2024-04-24T09:24:59.101 INFO:teuthology.orchestra.run.smithi090.stdout: "queue_len": 0, 2024-04-24T09:24:59.101 INFO:teuthology.orchestra.run.smithi090.stdout: "complete_latency": { 2024-04-24T09:24:59.101 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 44, 2024-04-24T09:24:59.101 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.002824051, 2024-04-24T09:24:59.101 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000064182 2024-04-24T09:24:59.102 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:59.102 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.102 INFO:teuthology.orchestra.run.smithi090.stdout: "finisher-PurgeQueue": { 2024-04-24T09:24:59.102 INFO:teuthology.orchestra.run.smithi090.stdout: "queue_len": 0, 2024-04-24T09:24:59.102 INFO:teuthology.orchestra.run.smithi090.stdout: "complete_latency": { 2024-04-24T09:24:59.102 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 2, 2024-04-24T09:24:59.102 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000008602, 2024-04-24T09:24:59.102 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000004301 2024-04-24T09:24:59.103 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:59.103 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.103 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-24T09:24:59.103 INFO:teuthology.orchestra.run.smithi090.stdout: "request": 3, 2024-04-24T09:24:59.103 INFO:teuthology.orchestra.run.smithi090.stdout: "reply": 1, 2024-04-24T09:24:59.103 INFO:teuthology.orchestra.run.smithi090.stdout: "reply_latency": { 2024-04-24T09:24:59.103 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 1, 2024-04-24T09:24:59.103 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000163092, 2024-04-24T09:24:59.104 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000163092 2024-04-24T09:24:59.104 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.104 INFO:teuthology.orchestra.run.smithi090.stdout: "slow_reply": 0, 2024-04-24T09:24:59.104 INFO:teuthology.orchestra.run.smithi090.stdout: "forward": 2, 2024-04-24T09:24:59.104 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_fetch_complete": 0, 2024-04-24T09:24:59.104 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_fetch_keys": 0, 2024-04-24T09:24:59.104 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_commit": 11, 2024-04-24T09:24:59.104 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_split": 0, 2024-04-24T09:24:59.104 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_merge": 0, 2024-04-24T09:24:59.105 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes": 35, 2024-04-24T09:24:59.105 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_top": 9, 2024-04-24T09:24:59.105 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_bottom": 26, 2024-04-24T09:24:59.105 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_pin_tail": 0, 2024-04-24T09:24:59.105 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_pinned": 21, 2024-04-24T09:24:59.105 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_expired": 8, 2024-04-24T09:24:59.105 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_with_caps": 1, 2024-04-24T09:24:59.105 INFO:teuthology.orchestra.run.smithi090.stdout: "caps": 1, 2024-04-24T09:24:59.105 INFO:teuthology.orchestra.run.smithi090.stdout: "subtrees": 3, 2024-04-24T09:24:59.106 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse": 48, 2024-04-24T09:24:59.106 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_hit": 28, 2024-04-24T09:24:59.106 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_forward": 2, 2024-04-24T09:24:59.106 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_discover": 18, 2024-04-24T09:24:59.106 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_dir_fetch": 0, 2024-04-24T09:24:59.106 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_remote_ino": 0, 2024-04-24T09:24:59.106 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_lock": 0, 2024-04-24T09:24:59.106 INFO:teuthology.orchestra.run.smithi090.stdout: "load_cent": 0, 2024-04-24T09:24:59.106 INFO:teuthology.orchestra.run.smithi090.stdout: "q": 0, 2024-04-24T09:24:59.107 INFO:teuthology.orchestra.run.smithi090.stdout: "exported": 0, 2024-04-24T09:24:59.107 INFO:teuthology.orchestra.run.smithi090.stdout: "exported_inodes": 0, 2024-04-24T09:24:59.107 INFO:teuthology.orchestra.run.smithi090.stdout: "imported": 0, 2024-04-24T09:24:59.107 INFO:teuthology.orchestra.run.smithi090.stdout: "imported_inodes": 0, 2024-04-24T09:24:59.107 INFO:teuthology.orchestra.run.smithi090.stdout: "openino_dir_fetch": 0, 2024-04-24T09:24:59.107 INFO:teuthology.orchestra.run.smithi090.stdout: "openino_backtrace_fetch": 0, 2024-04-24T09:24:59.107 INFO:teuthology.orchestra.run.smithi090.stdout: "openino_peer_discover": 0, 2024-04-24T09:24:59.107 INFO:teuthology.orchestra.run.smithi090.stdout: "root_rfiles": 4, 2024-04-24T09:24:59.107 INFO:teuthology.orchestra.run.smithi090.stdout: "root_rbytes": 234, 2024-04-24T09:24:59.108 INFO:teuthology.orchestra.run.smithi090.stdout: "root_rsnaps": 0, 2024-04-24T09:24:59.108 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_backtrace_fetch": 0, 2024-04-24T09:24:59.108 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_set_tag": 0, 2024-04-24T09:24:59.108 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T09:24:59.108 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_inotable_repaired": 0, 2024-04-24T09:24:59.108 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_dir_inodes": 0, 2024-04-24T09:24:59.108 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T09:24:59.108 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_dirfrag_rstats": 0, 2024-04-24T09:24:59.109 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_file_inodes": 0, 2024-04-24T09:24:59.109 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_inode_file_caps": 0, 2024-04-24T09:24:59.109 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_revoke": 0, 2024-04-24T09:24:59.109 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_grant": 0, 2024-04-24T09:24:59.109 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T09:24:59.109 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T09:24:59.109 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T09:24:59.109 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_caps": 0, 2024-04-24T09:24:59.109 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_caps_dirty": 0, 2024-04-24T09:24:59.110 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_cap_release": 0, 2024-04-24T09:24:59.110 INFO:teuthology.orchestra.run.smithi090.stdout: "process_request_cap_release": 0 2024-04-24T09:24:59.110 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.110 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_cache": { 2024-04-24T09:24:59.110 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_update": 0, 2024-04-24T09:24:59.110 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_update_receipt": 74, 2024-04-24T09:24:59.110 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_try_discover": 46, 2024-04-24T09:24:59.110 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_send_discover": 53, 2024-04-24T09:24:59.111 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_handle_discover": 0, 2024-04-24T09:24:59.111 INFO:teuthology.orchestra.run.smithi090.stdout: "num_strays": 0, 2024-04-24T09:24:59.111 INFO:teuthology.orchestra.run.smithi090.stdout: "num_strays_delayed": 0, 2024-04-24T09:24:59.111 INFO:teuthology.orchestra.run.smithi090.stdout: "num_strays_enqueuing": 0, 2024-04-24T09:24:59.111 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_created": 0, 2024-04-24T09:24:59.111 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_enqueued": 0, 2024-04-24T09:24:59.111 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_reintegrated": 0, 2024-04-24T09:24:59.111 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_migrated": 0, 2024-04-24T09:24:59.111 INFO:teuthology.orchestra.run.smithi090.stdout: "num_recovering_processing": 0, 2024-04-24T09:24:59.112 INFO:teuthology.orchestra.run.smithi090.stdout: "num_recovering_enqueued": 0, 2024-04-24T09:24:59.112 INFO:teuthology.orchestra.run.smithi090.stdout: "num_recovering_prioritized": 0, 2024-04-24T09:24:59.112 INFO:teuthology.orchestra.run.smithi090.stdout: "recovery_started": 0, 2024-04-24T09:24:59.112 INFO:teuthology.orchestra.run.smithi090.stdout: "recovery_completed": 0, 2024-04-24T09:24:59.112 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_quiesce_path": 0, 2024-04-24T09:24:59.112 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_quiesce_inode": 0, 2024-04-24T09:24:59.112 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T09:24:59.112 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_exportdir": 0, 2024-04-24T09:24:59.112 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_flush": 0, 2024-04-24T09:24:59.113 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_fragmentdir": 0, 2024-04-24T09:24:59.113 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_fragstats": 0, 2024-04-24T09:24:59.113 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_inodestats": 0 2024-04-24T09:24:59.113 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.113 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_log": { 2024-04-24T09:24:59.113 INFO:teuthology.orchestra.run.smithi090.stdout: "evlrg": 0, 2024-04-24T09:24:59.113 INFO:teuthology.orchestra.run.smithi090.stdout: "evadd": 5, 2024-04-24T09:24:59.113 INFO:teuthology.orchestra.run.smithi090.stdout: "evex": 0, 2024-04-24T09:24:59.113 INFO:teuthology.orchestra.run.smithi090.stdout: "evtrm": 0, 2024-04-24T09:24:59.114 INFO:teuthology.orchestra.run.smithi090.stdout: "ev": 5, 2024-04-24T09:24:59.114 INFO:teuthology.orchestra.run.smithi090.stdout: "evexg": 0, 2024-04-24T09:24:59.114 INFO:teuthology.orchestra.run.smithi090.stdout: "evexd": 0, 2024-04-24T09:24:59.114 INFO:teuthology.orchestra.run.smithi090.stdout: "segadd": 2, 2024-04-24T09:24:59.114 INFO:teuthology.orchestra.run.smithi090.stdout: "segex": 0, 2024-04-24T09:24:59.114 INFO:teuthology.orchestra.run.smithi090.stdout: "segtrm": 0, 2024-04-24T09:24:59.114 INFO:teuthology.orchestra.run.smithi090.stdout: "seg": 2, 2024-04-24T09:24:59.114 INFO:teuthology.orchestra.run.smithi090.stdout: "segmjr": 2, 2024-04-24T09:24:59.115 INFO:teuthology.orchestra.run.smithi090.stdout: "segexg": 0, 2024-04-24T09:24:59.115 INFO:teuthology.orchestra.run.smithi090.stdout: "segexd": 0, 2024-04-24T09:24:59.115 INFO:teuthology.orchestra.run.smithi090.stdout: "expos": 4194304, 2024-04-24T09:24:59.115 INFO:teuthology.orchestra.run.smithi090.stdout: "wrpos": 4195734, 2024-04-24T09:24:59.115 INFO:teuthology.orchestra.run.smithi090.stdout: "rdpos": 0, 2024-04-24T09:24:59.115 INFO:teuthology.orchestra.run.smithi090.stdout: "jlat": { 2024-04-24T09:24:59.115 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 4, 2024-04-24T09:24:59.115 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.009602293, 2024-04-24T09:24:59.115 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.002400573 2024-04-24T09:24:59.116 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.116 INFO:teuthology.orchestra.run.smithi090.stdout: "replayed": 0 2024-04-24T09:24:59.116 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.116 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_mem": { 2024-04-24T09:24:59.116 INFO:teuthology.orchestra.run.smithi090.stdout: "ino": 39, 2024-04-24T09:24:59.116 INFO:teuthology.orchestra.run.smithi090.stdout: "ino+": 44, 2024-04-24T09:24:59.116 INFO:teuthology.orchestra.run.smithi090.stdout: "ino-": 5, 2024-04-24T09:24:59.116 INFO:teuthology.orchestra.run.smithi090.stdout: "dir": 42, 2024-04-24T09:24:59.116 INFO:teuthology.orchestra.run.smithi090.stdout: "dir+": 52, 2024-04-24T09:24:59.117 INFO:teuthology.orchestra.run.smithi090.stdout: "dir-": 10, 2024-04-24T09:24:59.117 INFO:teuthology.orchestra.run.smithi090.stdout: "dn": 35, 2024-04-24T09:24:59.117 INFO:teuthology.orchestra.run.smithi090.stdout: "dn+": 43, 2024-04-24T09:24:59.117 INFO:teuthology.orchestra.run.smithi090.stdout: "dn-": 8, 2024-04-24T09:24:59.117 INFO:teuthology.orchestra.run.smithi090.stdout: "cap": 1, 2024-04-24T09:24:59.117 INFO:teuthology.orchestra.run.smithi090.stdout: "cap+": 1, 2024-04-24T09:24:59.117 INFO:teuthology.orchestra.run.smithi090.stdout: "cap-": 0, 2024-04-24T09:24:59.117 INFO:teuthology.orchestra.run.smithi090.stdout: "rss": 50944, 2024-04-24T09:24:59.117 INFO:teuthology.orchestra.run.smithi090.stdout: "heap": 223608 2024-04-24T09:24:59.118 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.118 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_server": { 2024-04-24T09:24:59.118 INFO:teuthology.orchestra.run.smithi090.stdout: "dispatch_client_request": 3, 2024-04-24T09:24:59.118 INFO:teuthology.orchestra.run.smithi090.stdout: "dispatch_server_request": 0, 2024-04-24T09:24:59.118 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_request": 3, 2024-04-24T09:24:59.118 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_session": 7, 2024-04-24T09:24:59.118 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_peer_request": 0, 2024-04-24T09:24:59.118 INFO:teuthology.orchestra.run.smithi090.stdout: "req_create_latency": { 2024-04-24T09:24:59.118 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:59.119 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:59.119 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:59.119 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.119 INFO:teuthology.orchestra.run.smithi090.stdout: "req_getattr_latency": { 2024-04-24T09:24:59.119 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:59.119 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:59.119 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:59.119 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.119 INFO:teuthology.orchestra.run.smithi090.stdout: "req_getfilelock_latency": { 2024-04-24T09:24:59.120 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:59.120 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:59.120 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:59.120 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.120 INFO:teuthology.orchestra.run.smithi090.stdout: "req_link_latency": { 2024-04-24T09:24:59.120 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:59.120 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:59.120 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:59.121 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.121 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookup_latency": { 2024-04-24T09:24:59.121 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:59.121 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:59.121 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:59.121 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.121 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookuphash_latency": { 2024-04-24T09:24:59.121 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:59.121 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:59.122 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:59.122 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.122 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupino_latency": { 2024-04-24T09:24:59.122 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 1, 2024-04-24T09:24:59.122 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000163092, 2024-04-24T09:24:59.122 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000163092 2024-04-24T09:24:59.122 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.122 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupname_latency": { 2024-04-24T09:24:59.122 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:59.123 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:59.123 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:59.123 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.123 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupparent_latency": { 2024-04-24T09:24:59.123 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:59.123 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:59.123 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:59.123 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.123 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupsnap_latency": { 2024-04-24T09:24:59.123 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:59.124 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:59.124 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:59.124 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.124 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lssnap_latency": { 2024-04-24T09:24:59.124 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:59.124 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:59.124 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:59.124 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.125 INFO:teuthology.orchestra.run.smithi090.stdout: "req_mkdir_latency": { 2024-04-24T09:24:59.125 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:59.125 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:24:58 smithi149 ceph-mon[32592]: pgmap v498: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 281 KiB/s wr, 46 op/s 2024-04-24T09:24:59.126 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:59.126 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:59.126 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.126 INFO:teuthology.orchestra.run.smithi090.stdout: "req_mknod_latency": { 2024-04-24T09:24:59.126 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:59.126 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:59.126 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:59.127 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.127 INFO:teuthology.orchestra.run.smithi090.stdout: "req_mksnap_latency": { 2024-04-24T09:24:59.127 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:59.127 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:59.127 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:59.127 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.127 INFO:teuthology.orchestra.run.smithi090.stdout: "req_open_latency": { 2024-04-24T09:24:59.127 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:59.127 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:59.128 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:59.128 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.128 INFO:teuthology.orchestra.run.smithi090.stdout: "req_readdir_latency": { 2024-04-24T09:24:59.128 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:59.128 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:59.128 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:59.128 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.128 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rename_latency": { 2024-04-24T09:24:59.128 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:59.129 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:59.129 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:59.129 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.129 INFO:teuthology.orchestra.run.smithi090.stdout: "req_renamesnap_latency": { 2024-04-24T09:24:59.129 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:59.129 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:59.129 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:59.129 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.129 INFO:teuthology.orchestra.run.smithi090.stdout: "req_snapdiff_latency": { 2024-04-24T09:24:59.130 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:59.130 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:59.130 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:59.130 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.130 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rmdir_latency": { 2024-04-24T09:24:59.130 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:59.130 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:59.130 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:59.131 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.131 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rmsnap_latency": { 2024-04-24T09:24:59.131 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:59.131 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:59.131 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:59.131 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.131 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rmxattr_latency": { 2024-04-24T09:24:59.131 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:59.131 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:59.132 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:59.132 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.132 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setattr_latency": { 2024-04-24T09:24:59.132 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:59.132 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:59.132 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:59.132 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.132 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setdirlayout_latency": { 2024-04-24T09:24:59.132 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:59.133 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:59.133 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:59.133 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.133 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setfilelock_latency": { 2024-04-24T09:24:59.133 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:59.133 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:59.133 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:59.133 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.134 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setlayout_latency": { 2024-04-24T09:24:59.134 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:59.134 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:59.134 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:59.134 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.134 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setxattr_latency": { 2024-04-24T09:24:59.134 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:59.134 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:59.135 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:59.135 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.135 INFO:teuthology.orchestra.run.smithi090.stdout: "req_symlink_latency": { 2024-04-24T09:24:59.135 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:59.135 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:59.135 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:59.135 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.135 INFO:teuthology.orchestra.run.smithi090.stdout: "req_unlink_latency": { 2024-04-24T09:24:59.135 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:59.136 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:59.136 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:59.136 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.136 INFO:teuthology.orchestra.run.smithi090.stdout: "cap_revoke_eviction": 0, 2024-04-24T09:24:59.136 INFO:teuthology.orchestra.run.smithi090.stdout: "cap_acquisition_throttle": 0, 2024-04-24T09:24:59.136 INFO:teuthology.orchestra.run.smithi090.stdout: "req_getvxattr_latency": { 2024-04-24T09:24:59.136 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:59.136 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:59.137 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:59.137 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:59.137 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.137 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_sessions": { 2024-04-24T09:24:59.137 INFO:teuthology.orchestra.run.smithi090.stdout: "session_count": 1, 2024-04-24T09:24:59.137 INFO:teuthology.orchestra.run.smithi090.stdout: "session_add": 2, 2024-04-24T09:24:59.137 INFO:teuthology.orchestra.run.smithi090.stdout: "session_remove": 1, 2024-04-24T09:24:59.137 INFO:teuthology.orchestra.run.smithi090.stdout: "sessions_open": 1, 2024-04-24T09:24:59.137 INFO:teuthology.orchestra.run.smithi090.stdout: "sessions_stale": 0, 2024-04-24T09:24:59.138 INFO:teuthology.orchestra.run.smithi090.stdout: "total_load": 1, 2024-04-24T09:24:59.138 INFO:teuthology.orchestra.run.smithi090.stdout: "average_load": 0, 2024-04-24T09:24:59.138 INFO:teuthology.orchestra.run.smithi090.stdout: "avg_session_uptime": 186, 2024-04-24T09:24:59.138 INFO:teuthology.orchestra.run.smithi090.stdout: "mdthresh_evicted": 0 2024-04-24T09:24:59.138 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.138 INFO:teuthology.orchestra.run.smithi090.stdout: "mempool": { 2024-04-24T09:24:59.138 INFO:teuthology.orchestra.run.smithi090.stdout: "bloom_filter_bytes": 0, 2024-04-24T09:24:59.138 INFO:teuthology.orchestra.run.smithi090.stdout: "bloom_filter_items": 0, 2024-04-24T09:24:59.139 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T09:24:59.139 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_alloc_items": 0, 2024-04-24T09:24:59.139 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T09:24:59.139 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_data_items": 0, 2024-04-24T09:24:59.139 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T09:24:59.139 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T09:24:59.139 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T09:24:59.139 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T09:24:59.139 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T09:24:59.140 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_other_items": 0, 2024-04-24T09:24:59.140 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T09:24:59.140 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T09:24:59.140 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_extent_bytes": 0, 2024-04-24T09:24:59.140 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_extent_items": 0, 2024-04-24T09:24:59.140 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_blob_bytes": 0, 2024-04-24T09:24:59.140 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_blob_items": 0, 2024-04-24T09:24:59.140 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T09:24:59.141 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T09:24:59.141 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T09:24:59.141 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T09:24:59.141 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T09:24:59.141 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_items": 0, 2024-04-24T09:24:59.141 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_txc_bytes": 0, 2024-04-24T09:24:59.141 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_txc_items": 0, 2024-04-24T09:24:59.141 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T09:24:59.141 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T09:24:59.142 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_bytes": 0, 2024-04-24T09:24:59.142 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_items": 0, 2024-04-24T09:24:59.142 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_bytes": 0, 2024-04-24T09:24:59.142 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_items": 0, 2024-04-24T09:24:59.142 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T09:24:59.142 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_reader_items": 0, 2024-04-24T09:24:59.142 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T09:24:59.142 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_writer_items": 0, 2024-04-24T09:24:59.142 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_anon_bytes": 247974, 2024-04-24T09:24:59.143 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_anon_items": 108, 2024-04-24T09:24:59.143 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_meta_bytes": 0, 2024-04-24T09:24:59.143 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_meta_items": 0, 2024-04-24T09:24:59.143 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_bytes": 0, 2024-04-24T09:24:59.143 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_items": 0, 2024-04-24T09:24:59.143 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mapbl_bytes": 0, 2024-04-24T09:24:59.143 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mapbl_items": 0, 2024-04-24T09:24:59.143 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pglog_bytes": 0, 2024-04-24T09:24:59.144 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pglog_items": 0, 2024-04-24T09:24:59.144 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_bytes": 7624, 2024-04-24T09:24:59.144 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_items": 160, 2024-04-24T09:24:59.144 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T09:24:59.144 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_mapping_items": 0, 2024-04-24T09:24:59.144 INFO:teuthology.orchestra.run.smithi090.stdout: "pgmap_bytes": 0, 2024-04-24T09:24:59.144 INFO:teuthology.orchestra.run.smithi090.stdout: "pgmap_items": 0, 2024-04-24T09:24:59.144 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_co_bytes": 170280, 2024-04-24T09:24:59.144 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_co_items": 915, 2024-04-24T09:24:59.145 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_1_bytes": 0, 2024-04-24T09:24:59.145 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_1_items": 0, 2024-04-24T09:24:59.145 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_2_bytes": 0, 2024-04-24T09:24:59.145 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_2_items": 0 2024-04-24T09:24:59.145 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.145 INFO:teuthology.orchestra.run.smithi090.stdout: "objecter": { 2024-04-24T09:24:59.145 INFO:teuthology.orchestra.run.smithi090.stdout: "op_active": 0, 2024-04-24T09:24:59.145 INFO:teuthology.orchestra.run.smithi090.stdout: "op_laggy": 0, 2024-04-24T09:24:59.145 INFO:teuthology.orchestra.run.smithi090.stdout: "op_send": 40, 2024-04-24T09:24:59.146 INFO:teuthology.orchestra.run.smithi090.stdout: "op_send_bytes": 11732, 2024-04-24T09:24:59.146 INFO:teuthology.orchestra.run.smithi090.stdout: "op_resend": 33, 2024-04-24T09:24:59.146 INFO:teuthology.orchestra.run.smithi090.stdout: "op_reply": 40, 2024-04-24T09:24:59.146 INFO:teuthology.orchestra.run.smithi090.stdout: "op_latency": { 2024-04-24T09:24:59.146 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 40, 2024-04-24T09:24:59.146 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.340994988, 2024-04-24T09:24:59.146 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.008524874 2024-04-24T09:24:59.146 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.146 INFO:teuthology.orchestra.run.smithi090.stdout: "op_inflight": 0, 2024-04-24T09:24:59.146 INFO:teuthology.orchestra.run.smithi090.stdout: "oplen_avg": { 2024-04-24T09:24:59.147 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 40, 2024-04-24T09:24:59.147 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 62 2024-04-24T09:24:59.147 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.147 INFO:teuthology.orchestra.run.smithi090.stdout: "op": 40, 2024-04-24T09:24:59.147 INFO:teuthology.orchestra.run.smithi090.stdout: "op_r": 0, 2024-04-24T09:24:59.147 INFO:teuthology.orchestra.run.smithi090.stdout: "op_w": 40, 2024-04-24T09:24:59.147 INFO:teuthology.orchestra.run.smithi090.stdout: "op_rmw": 0, 2024-04-24T09:24:59.147 INFO:teuthology.orchestra.run.smithi090.stdout: "op_pg": 0, 2024-04-24T09:24:59.147 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_stat": 0, 2024-04-24T09:24:59.148 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_create": 10, 2024-04-24T09:24:59.148 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_read": 0, 2024-04-24T09:24:59.148 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_write": 4, 2024-04-24T09:24:59.148 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_writefull": 7, 2024-04-24T09:24:59.148 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_writesame": 0, 2024-04-24T09:24:59.148 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_append": 0, 2024-04-24T09:24:59.148 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_zero": 0, 2024-04-24T09:24:59.148 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_truncate": 0, 2024-04-24T09:24:59.149 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_delete": 6, 2024-04-24T09:24:59.149 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_mapext": 0, 2024-04-24T09:24:59.149 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_sparse_read": 0, 2024-04-24T09:24:59.149 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_clonerange": 0, 2024-04-24T09:24:59.149 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_getxattr": 0, 2024-04-24T09:24:59.149 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_setxattr": 20, 2024-04-24T09:24:59.149 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_cmpxattr": 0, 2024-04-24T09:24:59.149 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_rmxattr": 0, 2024-04-24T09:24:59.150 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_resetxattrs": 0, 2024-04-24T09:24:59.150 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_call": 0, 2024-04-24T09:24:59.150 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_watch": 0, 2024-04-24T09:24:59.150 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_notify": 0, 2024-04-24T09:24:59.150 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T09:24:59.150 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_pgls": 0, 2024-04-24T09:24:59.150 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_pgls_filter": 0, 2024-04-24T09:24:59.150 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_other": 0, 2024-04-24T09:24:59.151 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_active": 0, 2024-04-24T09:24:59.151 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_send": 0, 2024-04-24T09:24:59.151 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_resend": 0, 2024-04-24T09:24:59.151 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_ping": 0, 2024-04-24T09:24:59.151 INFO:teuthology.orchestra.run.smithi090.stdout: "poolop_active": 0, 2024-04-24T09:24:59.151 INFO:teuthology.orchestra.run.smithi090.stdout: "poolop_send": 0, 2024-04-24T09:24:59.151 INFO:teuthology.orchestra.run.smithi090.stdout: "poolop_resend": 0, 2024-04-24T09:24:59.151 INFO:teuthology.orchestra.run.smithi090.stdout: "poolstat_active": 0, 2024-04-24T09:24:59.152 INFO:teuthology.orchestra.run.smithi090.stdout: "poolstat_send": 0, 2024-04-24T09:24:59.152 INFO:teuthology.orchestra.run.smithi090.stdout: "poolstat_resend": 0, 2024-04-24T09:24:59.152 INFO:teuthology.orchestra.run.smithi090.stdout: "statfs_active": 0, 2024-04-24T09:24:59.152 INFO:teuthology.orchestra.run.smithi090.stdout: "statfs_send": 0, 2024-04-24T09:24:59.152 INFO:teuthology.orchestra.run.smithi090.stdout: "statfs_resend": 0, 2024-04-24T09:24:59.152 INFO:teuthology.orchestra.run.smithi090.stdout: "command_active": 0, 2024-04-24T09:24:59.152 INFO:teuthology.orchestra.run.smithi090.stdout: "command_send": 0, 2024-04-24T09:24:59.152 INFO:teuthology.orchestra.run.smithi090.stdout: "command_resend": 0, 2024-04-24T09:24:59.152 INFO:teuthology.orchestra.run.smithi090.stdout: "map_epoch": 88, 2024-04-24T09:24:59.153 INFO:teuthology.orchestra.run.smithi090.stdout: "map_full": 0, 2024-04-24T09:24:59.153 INFO:teuthology.orchestra.run.smithi090.stdout: "map_inc": 2, 2024-04-24T09:24:59.153 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_sessions": 9, 2024-04-24T09:24:59.153 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_session_open": 9, 2024-04-24T09:24:59.153 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_session_close": 0, 2024-04-24T09:24:59.153 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_laggy": 0, 2024-04-24T09:24:59.153 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_wr": 14, 2024-04-24T09:24:59.153 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_rd": 0, 2024-04-24T09:24:59.153 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_del": 1 2024-04-24T09:24:59.154 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.154 INFO:teuthology.orchestra.run.smithi090.stdout: "oft": { 2024-04-24T09:24:59.154 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_objs": 1, 2024-04-24T09:24:59.154 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_kv_pairs": 0, 2024-04-24T09:24:59.154 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_updates": 0, 2024-04-24T09:24:59.154 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_removes": 0 2024-04-24T09:24:59.154 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.154 INFO:teuthology.orchestra.run.smithi090.stdout: "osd-slow-ops": { 2024-04-24T09:24:59.155 INFO:teuthology.orchestra.run.smithi090.stdout: "slow_ops_count": 0 2024-04-24T09:24:59.155 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.155 INFO:teuthology.orchestra.run.smithi090.stdout: "purge_queue": { 2024-04-24T09:24:59.155 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing_ops": 0, 2024-04-24T09:24:59.155 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing_ops_high_water": 0, 2024-04-24T09:24:59.155 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing": 0, 2024-04-24T09:24:59.155 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing_high_water": 0, 2024-04-24T09:24:59.155 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executed_ops": 0, 2024-04-24T09:24:59.155 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executed": 0, 2024-04-24T09:24:59.156 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_item_in_journal": 0 2024-04-24T09:24:59.156 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.156 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T09:24:59.156 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-04-24T09:24:59.156 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 104857600, 2024-04-24T09:24:59.156 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-04-24T09:24:59.156 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 961, 2024-04-24T09:24:59.156 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 1007367, 2024-04-24T09:24:59.156 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-04-24T09:24:59.157 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 961, 2024-04-24T09:24:59.157 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 0, 2024-04-24T09:24:59.157 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 0, 2024-04-24T09:24:59.157 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 961, 2024-04-24T09:24:59.157 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 1007367, 2024-04-24T09:24:59.157 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-04-24T09:24:59.157 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:59.157 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:59.157 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:59.158 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:59.158 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.158 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-objecter_bytes": { 2024-04-24T09:24:59.158 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-04-24T09:24:59.158 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 104857600, 2024-04-24T09:24:59.158 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-04-24T09:24:59.158 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 0, 2024-04-24T09:24:59.158 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 0, 2024-04-24T09:24:59.158 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-04-24T09:24:59.159 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 0, 2024-04-24T09:24:59.159 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 40, 2024-04-24T09:24:59.159 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 11732, 2024-04-24T09:24:59.159 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 40, 2024-04-24T09:24:59.159 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 11732, 2024-04-24T09:24:59.159 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-04-24T09:24:59.159 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:59.159 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:59.160 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:59.160 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:59.160 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.160 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-objecter_ops": { 2024-04-24T09:24:59.160 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-04-24T09:24:59.160 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 1024, 2024-04-24T09:24:59.160 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-04-24T09:24:59.160 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 0, 2024-04-24T09:24:59.160 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 0, 2024-04-24T09:24:59.161 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-04-24T09:24:59.161 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 0, 2024-04-24T09:24:59.161 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 40, 2024-04-24T09:24:59.161 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 40, 2024-04-24T09:24:59.161 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 40, 2024-04-24T09:24:59.161 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 40, 2024-04-24T09:24:59.161 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-04-24T09:24:59.161 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:59.161 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:59.162 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:59.162 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:59.162 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.162 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-write_buf_throttle": { 2024-04-24T09:24:59.162 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-04-24T09:24:59.162 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 3758096384, 2024-04-24T09:24:59.162 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-04-24T09:24:59.162 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 0, 2024-04-24T09:24:59.162 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 0, 2024-04-24T09:24:59.163 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-04-24T09:24:59.163 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 0, 2024-04-24T09:24:59.163 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 0, 2024-04-24T09:24:59.163 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 0, 2024-04-24T09:24:59.163 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 0, 2024-04-24T09:24:59.163 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 0, 2024-04-24T09:24:59.163 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-04-24T09:24:59.163 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:59.163 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:59.164 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:59.164 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:59.164 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-24T09:24:59.164 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-write_buf_throttle-0x5647bab9a500": { 2024-04-24T09:24:59.164 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-04-24T09:24:59.164 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 3758096384, 2024-04-24T09:24:59.164 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-04-24T09:24:59.164 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 5, 2024-04-24T09:24:59.164 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 1430, 2024-04-24T09:24:59.165 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-04-24T09:24:59.165 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 5, 2024-04-24T09:24:59.165 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 0, 2024-04-24T09:24:59.165 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 0, 2024-04-24T09:24:59.165 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 4, 2024-04-24T09:24:59.165 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 1430, 2024-04-24T09:24:59.165 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-04-24T09:24:59.165 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-04-24T09:24:59.165 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-04-24T09:24:59.166 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-04-24T09:24:59.166 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:59.166 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-24T09:24:59.166 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-24T09:24:59.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:24:58 smithi114 ceph-mon[32501]: pgmap v498: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 281 KiB/s wr, 46 op/s 2024-04-24T09:25:00.363 INFO:tasks.check_counter:Daemon mds.j mds.exported=0 2024-04-24T09:25:00.363 INFO:tasks.check_counter:Daemon mds.j mds.imported=0 2024-04-24T09:25:00.363 INFO:tasks.check_counter:Daemon mds.j mds_cache.dir_update=0 2024-04-24T09:25:00.364 INFO:tasks.check_counter:Daemon mds.j mds_cache.dir_update_receipt=74 2024-04-24T09:25:00.364 INFO:tasks.check_counter:Daemon mds.j mds.dir_split=0 2024-04-24T09:25:00.364 DEBUG:tasks.check_counter:Getting stats from b 2024-04-24T09:25:00.364 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-24T09:25:00.611 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T09:25:00.991 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:00 smithi114 ceph-mon[32501]: pgmap v499: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 284 KiB/s wr, 48 op/s 2024-04-24T09:25:01.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:00 smithi090 ceph-mon[27825]: pgmap v499: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 284 KiB/s wr, 48 op/s 2024-04-24T09:25:01.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:00 smithi149 ceph-mon[32592]: pgmap v499: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 284 KiB/s wr, 48 op/s 2024-04-24T09:25:02.418 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-04-24T09:25:02.418 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T09:25:02.418 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 2520, 2024-04-24T09:25:02.418 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 2758, 2024-04-24T09:25:02.418 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 661631, 2024-04-24T09:25:02.419 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 15127215, 2024-04-24T09:25:02.419 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 11, 2024-04-24T09:25:02.419 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 5, 2024-04-24T09:25:02.419 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.213792537, 2024-04-24T09:25:02.419 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.088924314, 2024-04-24T09:25:02.419 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.056282608, 2024-04-24T09:25:02.419 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.056305749, 2024-04-24T09:25:02.419 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:25:02.419 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 2758, 2024-04-24T09:25:02.420 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.873455388, 2024-04-24T09:25:02.420 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000316698 2024-04-24T09:25:02.420 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.420 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:25:02.420 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 1294, 2024-04-24T09:25:02.420 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000256084, 2024-04-24T09:25:02.420 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000197 2024-04-24T09:25:02.420 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.420 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-04-24T09:25:02.421 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 448 2024-04-24T09:25:02.421 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.421 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T09:25:02.421 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 2653, 2024-04-24T09:25:02.421 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 3250, 2024-04-24T09:25:02.421 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 1070657, 2024-04-24T09:25:02.421 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 15124185, 2024-04-24T09:25:02.421 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 10, 2024-04-24T09:25:02.421 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 5, 2024-04-24T09:25:02.421 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.235371511, 2024-04-24T09:25:02.422 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.108324942, 2024-04-24T09:25:02.422 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.063431246, 2024-04-24T09:25:02.422 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.054988894, 2024-04-24T09:25:02.422 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:25:02.422 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 3253, 2024-04-24T09:25:02.422 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.337070528, 2024-04-24T09:25:02.422 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000103618 2024-04-24T09:25:02.422 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.423 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:25:02.423 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 2320, 2024-04-24T09:25:02.423 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000391458, 2024-04-24T09:25:02.423 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000168 2024-04-24T09:25:02.423 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.423 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-24T09:25:02.423 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 295088 2024-04-24T09:25:02.423 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.424 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T09:25:02.424 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 21208, 2024-04-24T09:25:02.424 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 40090, 2024-04-24T09:25:02.424 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 8990181, 2024-04-24T09:25:02.424 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 33643888, 2024-04-24T09:25:02.424 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 16, 2024-04-24T09:25:02.424 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 9, 2024-04-24T09:25:02.424 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 1.545280429, 2024-04-24T09:25:02.424 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.735596274, 2024-04-24T09:25:02.425 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 1.203396708, 2024-04-24T09:25:02.425 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.064304684, 2024-04-24T09:25:02.425 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:25:02.426 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 40082, 2024-04-24T09:25:02.426 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.862109076, 2024-04-24T09:25:02.426 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000021508 2024-04-24T09:25:02.426 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.426 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:25:02.426 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 47914, 2024-04-24T09:25:02.426 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.006903529, 2024-04-24T09:25:02.426 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000144 2024-04-24T09:25:02.426 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.427 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 8562896, 2024-04-24T09:25:02.427 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 19528944 2024-04-24T09:25:02.427 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.427 INFO:teuthology.orchestra.run.smithi114.stdout: "cct": { 2024-04-24T09:25:02.427 INFO:teuthology.orchestra.run.smithi114.stdout: "total_workers": 1, 2024-04-24T09:25:02.427 INFO:teuthology.orchestra.run.smithi114.stdout: "unhealthy_workers": 0 2024-04-24T09:25:02.427 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.427 INFO:teuthology.orchestra.run.smithi114.stdout: "finisher-MDSRank": { 2024-04-24T09:25:02.428 INFO:teuthology.orchestra.run.smithi114.stdout: "queue_len": 0, 2024-04-24T09:25:02.428 INFO:teuthology.orchestra.run.smithi114.stdout: "complete_latency": { 2024-04-24T09:25:02.428 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 955, 2024-04-24T09:25:02.428 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 11.511093331, 2024-04-24T09:25:02.428 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.012053500 2024-04-24T09:25:02.428 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-04-24T09:25:02.428 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.428 INFO:teuthology.orchestra.run.smithi114.stdout: "finisher-PurgeQueue": { 2024-04-24T09:25:02.428 INFO:teuthology.orchestra.run.smithi114.stdout: "queue_len": 0, 2024-04-24T09:25:02.429 INFO:teuthology.orchestra.run.smithi114.stdout: "complete_latency": { 2024-04-24T09:25:02.429 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 6016, 2024-04-24T09:25:02.429 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 1.450610400, 2024-04-24T09:25:02.429 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000241125 2024-04-24T09:25:02.429 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-04-24T09:25:02.429 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.429 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": { 2024-04-24T09:25:02.429 INFO:teuthology.orchestra.run.smithi114.stdout: "request": 10644, 2024-04-24T09:25:02.429 INFO:teuthology.orchestra.run.smithi114.stdout: "reply": 10631, 2024-04-24T09:25:02.430 INFO:teuthology.orchestra.run.smithi114.stdout: "reply_latency": { 2024-04-24T09:25:02.430 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 10631, 2024-04-24T09:25:02.430 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 32.698484706, 2024-04-24T09:25:02.430 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.003075767 2024-04-24T09:25:02.430 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.430 INFO:teuthology.orchestra.run.smithi114.stdout: "slow_reply": 0, 2024-04-24T09:25:02.430 INFO:teuthology.orchestra.run.smithi114.stdout: "forward": 13, 2024-04-24T09:25:02.430 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_fetch_complete": 0, 2024-04-24T09:25:02.431 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_fetch_keys": 0, 2024-04-24T09:25:02.431 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_commit": 211, 2024-04-24T09:25:02.431 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_split": 24, 2024-04-24T09:25:02.431 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_merge": 1, 2024-04-24T09:25:02.431 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes": 5278, 2024-04-24T09:25:02.431 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_top": 356, 2024-04-24T09:25:02.431 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_bottom": 4712, 2024-04-24T09:25:02.431 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_pin_tail": 210, 2024-04-24T09:25:02.431 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_pinned": 4769, 2024-04-24T09:25:02.432 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_expired": 90, 2024-04-24T09:25:02.432 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_with_caps": 3438, 2024-04-24T09:25:02.432 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": 3438, 2024-04-24T09:25:02.432 INFO:teuthology.orchestra.run.smithi114.stdout: "subtrees": 7, 2024-04-24T09:25:02.432 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse": 10866, 2024-04-24T09:25:02.432 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_hit": 10324, 2024-04-24T09:25:02.432 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_forward": 13, 2024-04-24T09:25:02.432 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_discover": 0, 2024-04-24T09:25:02.432 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_dir_fetch": 0, 2024-04-24T09:25:02.432 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_remote_ino": 0, 2024-04-24T09:25:02.433 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_lock": 3, 2024-04-24T09:25:02.433 INFO:teuthology.orchestra.run.smithi114.stdout: "load_cent": 0, 2024-04-24T09:25:02.433 INFO:teuthology.orchestra.run.smithi114.stdout: "q": 0, 2024-04-24T09:25:02.433 INFO:teuthology.orchestra.run.smithi114.stdout: "exported": 5, 2024-04-24T09:25:02.433 INFO:teuthology.orchestra.run.smithi114.stdout: "exported_inodes": 85, 2024-04-24T09:25:02.433 INFO:teuthology.orchestra.run.smithi114.stdout: "imported": 0, 2024-04-24T09:25:02.433 INFO:teuthology.orchestra.run.smithi114.stdout: "imported_inodes": 0, 2024-04-24T09:25:02.433 INFO:teuthology.orchestra.run.smithi114.stdout: "openino_dir_fetch": 0, 2024-04-24T09:25:02.433 INFO:teuthology.orchestra.run.smithi114.stdout: "openino_backtrace_fetch": 0, 2024-04-24T09:25:02.434 INFO:teuthology.orchestra.run.smithi114.stdout: "openino_peer_discover": 0, 2024-04-24T09:25:02.434 INFO:teuthology.orchestra.run.smithi114.stdout: "root_rfiles": 3454, 2024-04-24T09:25:02.434 INFO:teuthology.orchestra.run.smithi114.stdout: "root_rbytes": 26625079, 2024-04-24T09:25:02.434 INFO:teuthology.orchestra.run.smithi114.stdout: "root_rsnaps": 0, 2024-04-24T09:25:02.434 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_backtrace_fetch": 0, 2024-04-24T09:25:02.434 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_set_tag": 0, 2024-04-24T09:25:02.434 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T09:25:02.437 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_inotable_repaired": 0, 2024-04-24T09:25:02.437 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_dir_inodes": 0, 2024-04-24T09:25:02.438 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T09:25:02.438 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_dirfrag_rstats": 0, 2024-04-24T09:25:02.438 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_file_inodes": 0, 2024-04-24T09:25:02.438 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_inode_file_caps": 3, 2024-04-24T09:25:02.438 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_revoke": 57, 2024-04-24T09:25:02.438 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_grant": 10810, 2024-04-24T09:25:02.438 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_trunc": 8, 2024-04-24T09:25:02.438 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T09:25:02.439 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T09:25:02.439 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_caps": 7736, 2024-04-24T09:25:02.439 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_caps_dirty": 6502, 2024-04-24T09:25:02.439 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_cap_release": 772, 2024-04-24T09:25:02.439 INFO:teuthology.orchestra.run.smithi114.stdout: "process_request_cap_release": 3527 2024-04-24T09:25:02.439 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.439 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_cache": { 2024-04-24T09:25:02.439 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_update": 91, 2024-04-24T09:25:02.439 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_update_receipt": 0, 2024-04-24T09:25:02.440 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_try_discover": 0, 2024-04-24T09:25:02.440 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_send_discover": 0, 2024-04-24T09:25:02.440 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_handle_discover": 191, 2024-04-24T09:25:02.440 INFO:teuthology.orchestra.run.smithi114.stdout: "num_strays": 1, 2024-04-24T09:25:02.440 INFO:teuthology.orchestra.run.smithi114.stdout: "num_strays_delayed": 0, 2024-04-24T09:25:02.440 INFO:teuthology.orchestra.run.smithi114.stdout: "num_strays_enqueuing": 0, 2024-04-24T09:25:02.440 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_created": 3235, 2024-04-24T09:25:02.440 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_enqueued": 3231, 2024-04-24T09:25:02.441 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_reintegrated": 3, 2024-04-24T09:25:02.441 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_migrated": 0, 2024-04-24T09:25:02.441 INFO:teuthology.orchestra.run.smithi114.stdout: "num_recovering_processing": 0, 2024-04-24T09:25:02.441 INFO:teuthology.orchestra.run.smithi114.stdout: "num_recovering_enqueued": 0, 2024-04-24T09:25:02.441 INFO:teuthology.orchestra.run.smithi114.stdout: "num_recovering_prioritized": 0, 2024-04-24T09:25:02.441 INFO:teuthology.orchestra.run.smithi114.stdout: "recovery_started": 0, 2024-04-24T09:25:02.441 INFO:teuthology.orchestra.run.smithi114.stdout: "recovery_completed": 0, 2024-04-24T09:25:02.441 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_quiesce_path": 0, 2024-04-24T09:25:02.441 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_quiesce_inode": 0, 2024-04-24T09:25:02.442 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T09:25:02.442 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_exportdir": 5, 2024-04-24T09:25:02.442 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_flush": 0, 2024-04-24T09:25:02.442 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_fragmentdir": 25, 2024-04-24T09:25:02.442 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_fragstats": 0, 2024-04-24T09:25:02.442 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_inodestats": 0 2024-04-24T09:25:02.442 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.442 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_log": { 2024-04-24T09:25:02.442 INFO:teuthology.orchestra.run.smithi114.stdout: "evlrg": 0, 2024-04-24T09:25:02.443 INFO:teuthology.orchestra.run.smithi114.stdout: "evadd": 23149, 2024-04-24T09:25:02.443 INFO:teuthology.orchestra.run.smithi114.stdout: "evex": 0, 2024-04-24T09:25:02.443 INFO:teuthology.orchestra.run.smithi114.stdout: "evtrm": 0, 2024-04-24T09:25:02.443 INFO:teuthology.orchestra.run.smithi114.stdout: "ev": 23149, 2024-04-24T09:25:02.443 INFO:teuthology.orchestra.run.smithi114.stdout: "evexg": 0, 2024-04-24T09:25:02.443 INFO:teuthology.orchestra.run.smithi114.stdout: "evexd": 0, 2024-04-24T09:25:02.443 INFO:teuthology.orchestra.run.smithi114.stdout: "segadd": 29, 2024-04-24T09:25:02.444 INFO:teuthology.orchestra.run.smithi114.stdout: "segex": 0, 2024-04-24T09:25:02.444 INFO:teuthology.orchestra.run.smithi114.stdout: "segtrm": 0, 2024-04-24T09:25:02.444 INFO:teuthology.orchestra.run.smithi114.stdout: "seg": 29, 2024-04-24T09:25:02.444 INFO:teuthology.orchestra.run.smithi114.stdout: "segmjr": 3, 2024-04-24T09:25:02.444 INFO:teuthology.orchestra.run.smithi114.stdout: "segexg": 0, 2024-04-24T09:25:02.444 INFO:teuthology.orchestra.run.smithi114.stdout: "segexd": 0, 2024-04-24T09:25:02.444 INFO:teuthology.orchestra.run.smithi114.stdout: "expos": 4194304, 2024-04-24T09:25:02.444 INFO:teuthology.orchestra.run.smithi114.stdout: "wrpos": 50378629, 2024-04-24T09:25:02.445 INFO:teuthology.orchestra.run.smithi114.stdout: "rdpos": 0, 2024-04-24T09:25:02.445 INFO:teuthology.orchestra.run.smithi114.stdout: "jlat": { 2024-04-24T09:25:02.445 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 377, 2024-04-24T09:25:02.445 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 14.909132313, 2024-04-24T09:25:02.445 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.039546770 2024-04-24T09:25:02.445 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.445 INFO:teuthology.orchestra.run.smithi114.stdout: "replayed": 0 2024-04-24T09:25:02.445 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.446 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_mem": { 2024-04-24T09:25:02.446 INFO:teuthology.orchestra.run.smithi114.stdout: "ino": 3461, 2024-04-24T09:25:02.446 INFO:teuthology.orchestra.run.smithi114.stdout: "ino+": 6772, 2024-04-24T09:25:02.446 INFO:teuthology.orchestra.run.smithi114.stdout: "ino-": 3311, 2024-04-24T09:25:02.446 INFO:teuthology.orchestra.run.smithi114.stdout: "dir": 248, 2024-04-24T09:25:02.446 INFO:teuthology.orchestra.run.smithi114.stdout: "dir+": 289, 2024-04-24T09:25:02.446 INFO:teuthology.orchestra.run.smithi114.stdout: "dir-": 41, 2024-04-24T09:25:02.446 INFO:teuthology.orchestra.run.smithi114.stdout: "dn": 5278, 2024-04-24T09:25:02.447 INFO:teuthology.orchestra.run.smithi114.stdout: "dn+": 8397, 2024-04-24T09:25:02.447 INFO:teuthology.orchestra.run.smithi114.stdout: "dn-": 3119, 2024-04-24T09:25:02.447 INFO:teuthology.orchestra.run.smithi114.stdout: "cap": 3438, 2024-04-24T09:25:02.447 INFO:teuthology.orchestra.run.smithi114.stdout: "cap+": 6763, 2024-04-24T09:25:02.447 INFO:teuthology.orchestra.run.smithi114.stdout: "cap-": 3325, 2024-04-24T09:25:02.447 INFO:teuthology.orchestra.run.smithi114.stdout: "rss": 99196, 2024-04-24T09:25:02.447 INFO:teuthology.orchestra.run.smithi114.stdout: "heap": 231800 2024-04-24T09:25:02.447 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.447 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_server": { 2024-04-24T09:25:02.448 INFO:teuthology.orchestra.run.smithi114.stdout: "dispatch_client_request": 10934, 2024-04-24T09:25:02.448 INFO:teuthology.orchestra.run.smithi114.stdout: "dispatch_server_request": 27, 2024-04-24T09:25:02.448 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_request": 10644, 2024-04-24T09:25:02.448 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_session": 20, 2024-04-24T09:25:02.448 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_peer_request": 54, 2024-04-24T09:25:02.448 INFO:teuthology.orchestra.run.smithi114.stdout: "req_create_latency": { 2024-04-24T09:25:02.448 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 6669, 2024-04-24T09:25:02.448 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 23.245566745, 2024-04-24T09:25:02.448 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.003485615 2024-04-24T09:25:02.449 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.449 INFO:teuthology.orchestra.run.smithi114.stdout: "req_getattr_latency": { 2024-04-24T09:25:02.449 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 27, 2024-04-24T09:25:02.449 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.407735138, 2024-04-24T09:25:02.449 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.015101301 2024-04-24T09:25:02.449 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.449 INFO:teuthology.orchestra.run.smithi114.stdout: "req_getfilelock_latency": { 2024-04-24T09:25:02.449 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:02.450 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:02.450 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:02.450 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.450 INFO:teuthology.orchestra.run.smithi114.stdout: "req_link_latency": { 2024-04-24T09:25:02.450 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 3, 2024-04-24T09:25:02.450 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.003186294, 2024-04-24T09:25:02.450 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.001062098 2024-04-24T09:25:02.450 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.451 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookup_latency": { 2024-04-24T09:25:02.451 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 5, 2024-04-24T09:25:02.451 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.001372269, 2024-04-24T09:25:02.451 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000274453 2024-04-24T09:25:02.451 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.451 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookuphash_latency": { 2024-04-24T09:25:02.451 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:02.451 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:02.451 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:02.452 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.452 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupino_latency": { 2024-04-24T09:25:02.452 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:02.452 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:02.452 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:02.452 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.452 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupname_latency": { 2024-04-24T09:25:02.452 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:02.453 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:02.453 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:02.453 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.453 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupparent_latency": { 2024-04-24T09:25:02.453 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:02.453 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:02.453 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:02.453 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.453 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupsnap_latency": { 2024-04-24T09:25:02.454 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:02.454 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:02.454 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:02.454 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.454 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lssnap_latency": { 2024-04-24T09:25:02.455 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:02.455 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:02.455 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:02.455 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.455 INFO:teuthology.orchestra.run.smithi114.stdout: "req_mkdir_latency": { 2024-04-24T09:25:02.456 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 84, 2024-04-24T09:25:02.456 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.079322483, 2024-04-24T09:25:02.456 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000944315 2024-04-24T09:25:02.456 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.456 INFO:teuthology.orchestra.run.smithi114.stdout: "req_mknod_latency": { 2024-04-24T09:25:02.456 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:02.456 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:02.457 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:02.457 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.457 INFO:teuthology.orchestra.run.smithi114.stdout: "req_mksnap_latency": { 2024-04-24T09:25:02.457 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:02.457 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:02.457 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:02.457 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.457 INFO:teuthology.orchestra.run.smithi114.stdout: "req_open_latency": { 2024-04-24T09:25:02.457 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 428, 2024-04-24T09:25:02.458 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.109585694, 2024-04-24T09:25:02.458 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000256041 2024-04-24T09:25:02.458 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.458 INFO:teuthology.orchestra.run.smithi114.stdout: "req_readdir_latency": { 2024-04-24T09:25:02.458 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 104, 2024-04-24T09:25:02.458 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.910544311, 2024-04-24T09:25:02.458 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.008755233 2024-04-24T09:25:02.458 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.458 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rename_latency": { 2024-04-24T09:25:02.459 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 82, 2024-04-24T09:25:02.459 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.173193663, 2024-04-24T09:25:02.459 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.002112117 2024-04-24T09:25:02.459 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.459 INFO:teuthology.orchestra.run.smithi114.stdout: "req_renamesnap_latency": { 2024-04-24T09:25:02.459 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:02.459 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:02.459 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:02.459 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.460 INFO:teuthology.orchestra.run.smithi114.stdout: "req_snapdiff_latency": { 2024-04-24T09:25:02.460 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:02.460 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:02.460 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:02.460 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.460 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rmdir_latency": { 2024-04-24T09:25:02.460 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 22, 2024-04-24T09:25:02.460 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.709140278, 2024-04-24T09:25:02.460 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.032233649 2024-04-24T09:25:02.461 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.461 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rmsnap_latency": { 2024-04-24T09:25:02.461 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:02.461 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:02.461 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:02.461 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.461 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rmxattr_latency": { 2024-04-24T09:25:02.461 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:02.461 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:02.462 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:02.462 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.462 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setattr_latency": { 2024-04-24T09:25:02.462 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 9, 2024-04-24T09:25:02.462 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.075473774, 2024-04-24T09:25:02.462 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.008385974 2024-04-24T09:25:02.462 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.462 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setdirlayout_latency": { 2024-04-24T09:25:02.463 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:02.463 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:02.463 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:02.463 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.463 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setfilelock_latency": { 2024-04-24T09:25:02.463 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 8, 2024-04-24T09:25:02.463 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.003245013, 2024-04-24T09:25:02.463 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000405626 2024-04-24T09:25:02.463 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.464 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setlayout_latency": { 2024-04-24T09:25:02.464 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:02.464 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:02.464 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:02.464 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.464 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setxattr_latency": { 2024-04-24T09:25:02.464 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 8, 2024-04-24T09:25:02.464 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.022256147, 2024-04-24T09:25:02.465 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.002782018 2024-04-24T09:25:02.465 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.465 INFO:teuthology.orchestra.run.smithi114.stdout: "req_symlink_latency": { 2024-04-24T09:25:02.465 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 6, 2024-04-24T09:25:02.465 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.004673208, 2024-04-24T09:25:02.465 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000778868 2024-04-24T09:25:02.465 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.465 INFO:teuthology.orchestra.run.smithi114.stdout: "req_unlink_latency": { 2024-04-24T09:25:02.465 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 3176, 2024-04-24T09:25:02.466 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 6.953189689, 2024-04-24T09:25:02.466 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.002189291 2024-04-24T09:25:02.466 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.466 INFO:teuthology.orchestra.run.smithi114.stdout: "cap_revoke_eviction": 0, 2024-04-24T09:25:02.466 INFO:teuthology.orchestra.run.smithi114.stdout: "cap_acquisition_throttle": 0, 2024-04-24T09:25:02.466 INFO:teuthology.orchestra.run.smithi114.stdout: "req_getvxattr_latency": { 2024-04-24T09:25:02.466 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:02.466 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:02.467 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:02.467 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-04-24T09:25:02.467 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.467 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_sessions": { 2024-04-24T09:25:02.467 INFO:teuthology.orchestra.run.smithi114.stdout: "session_count": 1, 2024-04-24T09:25:02.467 INFO:teuthology.orchestra.run.smithi114.stdout: "session_add": 3, 2024-04-24T09:25:02.467 INFO:teuthology.orchestra.run.smithi114.stdout: "session_remove": 2, 2024-04-24T09:25:02.467 INFO:teuthology.orchestra.run.smithi114.stdout: "sessions_open": 1, 2024-04-24T09:25:02.467 INFO:teuthology.orchestra.run.smithi114.stdout: "sessions_stale": 0, 2024-04-24T09:25:02.468 INFO:teuthology.orchestra.run.smithi114.stdout: "total_load": 6759, 2024-04-24T09:25:02.468 INFO:teuthology.orchestra.run.smithi114.stdout: "average_load": 6759, 2024-04-24T09:25:02.468 INFO:teuthology.orchestra.run.smithi114.stdout: "avg_session_uptime": 188, 2024-04-24T09:25:02.468 INFO:teuthology.orchestra.run.smithi114.stdout: "mdthresh_evicted": 0 2024-04-24T09:25:02.468 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.468 INFO:teuthology.orchestra.run.smithi114.stdout: "mempool": { 2024-04-24T09:25:02.468 INFO:teuthology.orchestra.run.smithi114.stdout: "bloom_filter_bytes": 0, 2024-04-24T09:25:02.468 INFO:teuthology.orchestra.run.smithi114.stdout: "bloom_filter_items": 0, 2024-04-24T09:25:02.469 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T09:25:02.469 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_alloc_items": 0, 2024-04-24T09:25:02.469 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T09:25:02.469 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_data_items": 0, 2024-04-24T09:25:02.469 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T09:25:02.469 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T09:25:02.469 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T09:25:02.469 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T09:25:02.470 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T09:25:02.470 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_other_items": 0, 2024-04-24T09:25:02.470 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T09:25:02.470 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T09:25:02.470 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_extent_bytes": 0, 2024-04-24T09:25:02.470 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_extent_items": 0, 2024-04-24T09:25:02.470 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_blob_bytes": 0, 2024-04-24T09:25:02.470 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_blob_items": 0, 2024-04-24T09:25:02.471 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T09:25:02.471 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T09:25:02.471 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T09:25:02.471 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T09:25:02.471 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T09:25:02.471 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_fsck_items": 0, 2024-04-24T09:25:02.471 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_txc_bytes": 0, 2024-04-24T09:25:02.471 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_txc_items": 0, 2024-04-24T09:25:02.471 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T09:25:02.472 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T09:25:02.472 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_bytes": 0, 2024-04-24T09:25:02.472 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_items": 0, 2024-04-24T09:25:02.472 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_bytes": 0, 2024-04-24T09:25:02.472 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_items": 0, 2024-04-24T09:25:02.472 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T09:25:02.472 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_reader_items": 0, 2024-04-24T09:25:02.472 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T09:25:02.473 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_writer_items": 0, 2024-04-24T09:25:02.473 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_anon_bytes": 426397, 2024-04-24T09:25:02.473 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_anon_items": 3569, 2024-04-24T09:25:02.473 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_meta_bytes": 1408, 2024-04-24T09:25:02.473 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_meta_items": 16, 2024-04-24T09:25:02.473 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_bytes": 0, 2024-04-24T09:25:02.473 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_items": 0, 2024-04-24T09:25:02.473 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_mapbl_bytes": 0, 2024-04-24T09:25:02.473 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_mapbl_items": 0, 2024-04-24T09:25:02.474 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_pglog_bytes": 0, 2024-04-24T09:25:02.474 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_pglog_items": 0, 2024-04-24T09:25:02.474 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_bytes": 7624, 2024-04-24T09:25:02.474 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_items": 160, 2024-04-24T09:25:02.474 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T09:25:02.474 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_mapping_items": 0, 2024-04-24T09:25:02.474 INFO:teuthology.orchestra.run.smithi114.stdout: "pgmap_bytes": 0, 2024-04-24T09:25:02.475 INFO:teuthology.orchestra.run.smithi114.stdout: "pgmap_items": 0, 2024-04-24T09:25:02.475 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_co_bytes": 13579103, 2024-04-24T09:25:02.475 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_co_items": 154700, 2024-04-24T09:25:02.475 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_1_bytes": 0, 2024-04-24T09:25:02.475 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_1_items": 0, 2024-04-24T09:25:02.475 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_2_bytes": 0, 2024-04-24T09:25:02.475 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_2_items": 0 2024-04-24T09:25:02.475 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.475 INFO:teuthology.orchestra.run.smithi114.stdout: "objecter": { 2024-04-24T09:25:02.476 INFO:teuthology.orchestra.run.smithi114.stdout: "op_active": 0, 2024-04-24T09:25:02.476 INFO:teuthology.orchestra.run.smithi114.stdout: "op_laggy": 0, 2024-04-24T09:25:02.476 INFO:teuthology.orchestra.run.smithi114.stdout: "op_send": 4620, 2024-04-24T09:25:02.476 INFO:teuthology.orchestra.run.smithi114.stdout: "op_send_bytes": 49749688, 2024-04-24T09:25:02.476 INFO:teuthology.orchestra.run.smithi114.stdout: "op_resend": 32, 2024-04-24T09:25:02.476 INFO:teuthology.orchestra.run.smithi114.stdout: "op_reply": 4620, 2024-04-24T09:25:02.476 INFO:teuthology.orchestra.run.smithi114.stdout: "op_latency": { 2024-04-24T09:25:02.476 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 4620, 2024-04-24T09:25:02.477 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 33.557302426, 2024-04-24T09:25:02.477 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.007263485 2024-04-24T09:25:02.477 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.477 INFO:teuthology.orchestra.run.smithi114.stdout: "op_inflight": 0, 2024-04-24T09:25:02.477 INFO:teuthology.orchestra.run.smithi114.stdout: "oplen_avg": { 2024-04-24T09:25:02.477 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 4620, 2024-04-24T09:25:02.477 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 4938 2024-04-24T09:25:02.477 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.477 INFO:teuthology.orchestra.run.smithi114.stdout: "op": 4620, 2024-04-24T09:25:02.478 INFO:teuthology.orchestra.run.smithi114.stdout: "op_r": 55, 2024-04-24T09:25:02.478 INFO:teuthology.orchestra.run.smithi114.stdout: "op_w": 4565, 2024-04-24T09:25:02.478 INFO:teuthology.orchestra.run.smithi114.stdout: "op_rmw": 0, 2024-04-24T09:25:02.478 INFO:teuthology.orchestra.run.smithi114.stdout: "op_pg": 0, 2024-04-24T09:25:02.478 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_stat": 0, 2024-04-24T09:25:02.478 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_create": 11, 2024-04-24T09:25:02.478 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_read": 55, 2024-04-24T09:25:02.478 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_write": 432, 2024-04-24T09:25:02.478 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_writefull": 44, 2024-04-24T09:25:02.479 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_writesame": 0, 2024-04-24T09:25:02.479 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_append": 0, 2024-04-24T09:25:02.479 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_zero": 0, 2024-04-24T09:25:02.479 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_truncate": 8, 2024-04-24T09:25:02.479 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_delete": 3778, 2024-04-24T09:25:02.479 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_mapext": 0, 2024-04-24T09:25:02.479 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_sparse_read": 0, 2024-04-24T09:25:02.479 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_clonerange": 0, 2024-04-24T09:25:02.479 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_getxattr": 0, 2024-04-24T09:25:02.480 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_setxattr": 22, 2024-04-24T09:25:02.480 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_cmpxattr": 0, 2024-04-24T09:25:02.480 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_rmxattr": 0, 2024-04-24T09:25:02.480 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_resetxattrs": 0, 2024-04-24T09:25:02.480 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_call": 0, 2024-04-24T09:25:02.480 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_watch": 0, 2024-04-24T09:25:02.480 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_notify": 0, 2024-04-24T09:25:02.480 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T09:25:02.481 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_pgls": 0, 2024-04-24T09:25:02.481 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_pgls_filter": 0, 2024-04-24T09:25:02.481 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_other": 8, 2024-04-24T09:25:02.481 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_active": 0, 2024-04-24T09:25:02.481 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_send": 0, 2024-04-24T09:25:02.481 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_resend": 0, 2024-04-24T09:25:02.481 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_ping": 0, 2024-04-24T09:25:02.481 INFO:teuthology.orchestra.run.smithi114.stdout: "poolop_active": 0, 2024-04-24T09:25:02.482 INFO:teuthology.orchestra.run.smithi114.stdout: "poolop_send": 0, 2024-04-24T09:25:02.482 INFO:teuthology.orchestra.run.smithi114.stdout: "poolop_resend": 0, 2024-04-24T09:25:02.482 INFO:teuthology.orchestra.run.smithi114.stdout: "poolstat_active": 0, 2024-04-24T09:25:02.482 INFO:teuthology.orchestra.run.smithi114.stdout: "poolstat_send": 0, 2024-04-24T09:25:02.482 INFO:teuthology.orchestra.run.smithi114.stdout: "poolstat_resend": 0, 2024-04-24T09:25:02.482 INFO:teuthology.orchestra.run.smithi114.stdout: "statfs_active": 0, 2024-04-24T09:25:02.482 INFO:teuthology.orchestra.run.smithi114.stdout: "statfs_send": 0, 2024-04-24T09:25:02.482 INFO:teuthology.orchestra.run.smithi114.stdout: "statfs_resend": 0, 2024-04-24T09:25:02.483 INFO:teuthology.orchestra.run.smithi114.stdout: "command_active": 0, 2024-04-24T09:25:02.483 INFO:teuthology.orchestra.run.smithi114.stdout: "command_send": 0, 2024-04-24T09:25:02.483 INFO:teuthology.orchestra.run.smithi114.stdout: "command_resend": 0, 2024-04-24T09:25:02.483 INFO:teuthology.orchestra.run.smithi114.stdout: "map_epoch": 88, 2024-04-24T09:25:02.483 INFO:teuthology.orchestra.run.smithi114.stdout: "map_full": 0, 2024-04-24T09:25:02.483 INFO:teuthology.orchestra.run.smithi114.stdout: "map_inc": 3, 2024-04-24T09:25:02.483 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_sessions": 12, 2024-04-24T09:25:02.483 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_session_open": 12, 2024-04-24T09:25:02.483 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_session_close": 0, 2024-04-24T09:25:02.484 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_laggy": 0, 2024-04-24T09:25:02.484 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_wr": 544, 2024-04-24T09:25:02.484 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_rd": 0, 2024-04-24T09:25:02.484 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_del": 36 2024-04-24T09:25:02.484 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.484 INFO:teuthology.orchestra.run.smithi114.stdout: "oft": { 2024-04-24T09:25:02.484 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_objs": 1, 2024-04-24T09:25:02.484 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_kv_pairs": 0, 2024-04-24T09:25:02.484 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_updates": 0, 2024-04-24T09:25:02.485 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_removes": 0 2024-04-24T09:25:02.485 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.485 INFO:teuthology.orchestra.run.smithi114.stdout: "osd-slow-ops": { 2024-04-24T09:25:02.485 INFO:teuthology.orchestra.run.smithi114.stdout: "slow_ops_count": 0 2024-04-24T09:25:02.485 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.485 INFO:teuthology.orchestra.run.smithi114.stdout: "purge_queue": { 2024-04-24T09:25:02.485 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing_ops": 0, 2024-04-24T09:25:02.485 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing_ops_high_water": 6, 2024-04-24T09:25:02.485 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing": 0, 2024-04-24T09:25:02.486 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing_high_water": 6, 2024-04-24T09:25:02.486 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executed_ops": 3231, 2024-04-24T09:25:02.486 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executed": 3231, 2024-04-24T09:25:02.486 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_item_in_journal": 0 2024-04-24T09:25:02.486 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.486 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T09:25:02.486 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-04-24T09:25:02.486 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 104857600, 2024-04-24T09:25:02.487 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-04-24T09:25:02.487 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 26377, 2024-04-24T09:25:02.487 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 7402865, 2024-04-24T09:25:02.487 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-04-24T09:25:02.487 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 26377, 2024-04-24T09:25:02.487 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-04-24T09:25:02.487 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-04-24T09:25:02.487 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 26377, 2024-04-24T09:25:02.487 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 7402865, 2024-04-24T09:25:02.488 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-04-24T09:25:02.488 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:02.488 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:02.488 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:02.488 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-04-24T09:25:02.488 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.488 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-objecter_bytes": { 2024-04-24T09:25:02.488 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-04-24T09:25:02.488 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 104857600, 2024-04-24T09:25:02.489 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-04-24T09:25:02.489 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 0, 2024-04-24T09:25:02.489 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 0, 2024-04-24T09:25:02.489 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-04-24T09:25:02.489 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 0, 2024-04-24T09:25:02.489 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 4620, 2024-04-24T09:25:02.489 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 50076019, 2024-04-24T09:25:02.489 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 4620, 2024-04-24T09:25:02.489 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 50076019, 2024-04-24T09:25:02.490 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-04-24T09:25:02.490 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:02.490 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:02.490 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:02.490 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-04-24T09:25:02.490 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.490 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-objecter_ops": { 2024-04-24T09:25:02.490 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-04-24T09:25:02.491 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 1024, 2024-04-24T09:25:02.491 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-04-24T09:25:02.491 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 0, 2024-04-24T09:25:02.491 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 0, 2024-04-24T09:25:02.491 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-04-24T09:25:02.491 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 0, 2024-04-24T09:25:02.491 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 4620, 2024-04-24T09:25:02.491 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 4620, 2024-04-24T09:25:02.491 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 4620, 2024-04-24T09:25:02.491 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 4620, 2024-04-24T09:25:02.492 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-04-24T09:25:02.492 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:02.492 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:02.492 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:02.492 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-04-24T09:25:02.492 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.492 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-write_buf_throttle": { 2024-04-24T09:25:02.492 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-04-24T09:25:02.492 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 3758096384, 2024-04-24T09:25:02.493 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-04-24T09:25:02.493 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 3231, 2024-04-24T09:25:02.493 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 326331, 2024-04-24T09:25:02.493 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-04-24T09:25:02.493 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 3231, 2024-04-24T09:25:02.493 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-04-24T09:25:02.493 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-04-24T09:25:02.493 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 55, 2024-04-24T09:25:02.493 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 326331, 2024-04-24T09:25:02.493 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-04-24T09:25:02.494 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:02.494 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:02.494 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:02.494 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-04-24T09:25:02.494 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:02.494 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-write_buf_throttle-0x55ee0f148500": { 2024-04-24T09:25:02.494 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-04-24T09:25:02.494 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 3758096384, 2024-04-24T09:25:02.495 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-04-24T09:25:02.495 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 23149, 2024-04-24T09:25:02.495 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 46184325, 2024-04-24T09:25:02.495 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-04-24T09:25:02.495 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 23149, 2024-04-24T09:25:02.495 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-04-24T09:25:02.495 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-04-24T09:25:02.495 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 377, 2024-04-24T09:25:02.495 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 46184325, 2024-04-24T09:25:02.496 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-04-24T09:25:02.496 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:02.496 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:02.496 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:02.496 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-04-24T09:25:02.496 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-04-24T09:25:02.496 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-04-24T09:25:03.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:02 smithi090 ceph-mon[27825]: pgmap v500: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 166 KiB/s wr, 39 op/s 2024-04-24T09:25:03.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:02 smithi149 ceph-mon[32592]: pgmap v500: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 166 KiB/s wr, 39 op/s 2024-04-24T09:25:03.175 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:02 smithi114 ceph-mon[32501]: pgmap v500: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 166 KiB/s wr, 39 op/s 2024-04-24T09:25:03.293 INFO:tasks.check_counter:Daemon mds.b mds.exported=5 2024-04-24T09:25:03.293 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-04-24T09:25:03.293 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=91 2024-04-24T09:25:03.293 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-04-24T09:25:03.294 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=24 2024-04-24T09:25:03.294 DEBUG:tasks.check_counter:Getting stats from e 2024-04-24T09:25:03.294 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-24T09:25:03.538 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T09:25:04.992 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:04 smithi114 ceph-mon[32501]: pgmap v501: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 48 KiB/s wr, 24 op/s 2024-04-24T09:25:05.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:04 smithi090 ceph-mon[27825]: pgmap v501: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 48 KiB/s wr, 24 op/s 2024-04-24T09:25:05.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:04 smithi149 ceph-mon[32592]: pgmap v501: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 48 KiB/s wr, 24 op/s 2024-04-24T09:25:05.241 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-04-24T09:25:05.241 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T09:25:05.242 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 361, 2024-04-24T09:25:05.242 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 363, 2024-04-24T09:25:05.242 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 196963, 2024-04-24T09:25:05.242 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 114862, 2024-04-24T09:25:05.242 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 6, 2024-04-24T09:25:05.242 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 2, 2024-04-24T09:25:05.242 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.032042038, 2024-04-24T09:25:05.242 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.012659849, 2024-04-24T09:25:05.243 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.007526030, 2024-04-24T09:25:05.243 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.007990741, 2024-04-24T09:25:05.243 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:25:05.243 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 361, 2024-04-24T09:25:05.243 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.014018645, 2024-04-24T09:25:05.243 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000038832 2024-04-24T09:25:05.244 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.244 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:25:05.244 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:05.244 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:05.244 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:05.244 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.244 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T09:25:05.245 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T09:25:05.245 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.245 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T09:25:05.245 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 408, 2024-04-24T09:25:05.245 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 394, 2024-04-24T09:25:05.245 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 859920, 2024-04-24T09:25:05.245 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 154887, 2024-04-24T09:25:05.246 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 4, 2024-04-24T09:25:05.246 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 3, 2024-04-24T09:25:05.246 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.045218441, 2024-04-24T09:25:05.246 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.013691248, 2024-04-24T09:25:05.246 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.021914185, 2024-04-24T09:25:05.246 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.011329740, 2024-04-24T09:25:05.247 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:25:05.247 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 394, 2024-04-24T09:25:05.247 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.016247002, 2024-04-24T09:25:05.247 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000041236 2024-04-24T09:25:05.247 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.247 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:25:05.247 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:05.248 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:05.248 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:05.248 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.248 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 791264, 2024-04-24T09:25:05.248 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 66160 2024-04-24T09:25:05.249 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.249 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T09:25:05.249 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 350, 2024-04-24T09:25:05.249 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 429, 2024-04-24T09:25:05.249 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 117424, 2024-04-24T09:25:05.249 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 325574, 2024-04-24T09:25:05.250 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 4, 2024-04-24T09:25:05.250 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 3, 2024-04-24T09:25:05.250 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.044611676, 2024-04-24T09:25:05.250 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.015518811, 2024-04-24T09:25:05.250 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.011353978, 2024-04-24T09:25:05.250 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.012976255, 2024-04-24T09:25:05.250 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:25:05.250 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 427, 2024-04-24T09:25:05.251 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.026677632, 2024-04-24T09:25:05.251 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000062476 2024-04-24T09:25:05.251 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.251 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:25:05.251 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:05.251 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:05.251 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:05.251 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.251 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-24T09:25:05.252 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 214592 2024-04-24T09:25:05.252 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.252 INFO:teuthology.orchestra.run.smithi114.stdout: "cct": { 2024-04-24T09:25:05.252 INFO:teuthology.orchestra.run.smithi114.stdout: "total_workers": 1, 2024-04-24T09:25:05.252 INFO:teuthology.orchestra.run.smithi114.stdout: "unhealthy_workers": 0 2024-04-24T09:25:05.252 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.252 INFO:teuthology.orchestra.run.smithi114.stdout: "finisher-MDSRank": { 2024-04-24T09:25:05.252 INFO:teuthology.orchestra.run.smithi114.stdout: "queue_len": 0, 2024-04-24T09:25:05.252 INFO:teuthology.orchestra.run.smithi114.stdout: "complete_latency": { 2024-04-24T09:25:05.253 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 1691, 2024-04-24T09:25:05.253 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.044277596, 2024-04-24T09:25:05.253 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000026184 2024-04-24T09:25:05.253 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-04-24T09:25:05.253 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.253 INFO:teuthology.orchestra.run.smithi114.stdout: "finisher-PurgeQueue": { 2024-04-24T09:25:05.253 INFO:teuthology.orchestra.run.smithi114.stdout: "queue_len": 0, 2024-04-24T09:25:05.253 INFO:teuthology.orchestra.run.smithi114.stdout: "complete_latency": { 2024-04-24T09:25:05.253 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:05.254 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:05.254 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:05.254 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-04-24T09:25:05.254 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.254 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": { 2024-04-24T09:25:05.254 INFO:teuthology.orchestra.run.smithi114.stdout: "request": 0, 2024-04-24T09:25:05.254 INFO:teuthology.orchestra.run.smithi114.stdout: "reply": 0, 2024-04-24T09:25:05.254 INFO:teuthology.orchestra.run.smithi114.stdout: "reply_latency": { 2024-04-24T09:25:05.255 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:05.255 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:05.255 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:05.255 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.255 INFO:teuthology.orchestra.run.smithi114.stdout: "slow_reply": 0, 2024-04-24T09:25:05.255 INFO:teuthology.orchestra.run.smithi114.stdout: "forward": 0, 2024-04-24T09:25:05.255 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_fetch_complete": 0, 2024-04-24T09:25:05.255 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_fetch_keys": 0, 2024-04-24T09:25:05.255 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_commit": 0, 2024-04-24T09:25:05.256 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_split": 0, 2024-04-24T09:25:05.256 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_merge": 0, 2024-04-24T09:25:05.256 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes": 62, 2024-04-24T09:25:05.256 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_top": 0, 2024-04-24T09:25:05.256 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_bottom": 62, 2024-04-24T09:25:05.256 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_pin_tail": 0, 2024-04-24T09:25:05.256 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_pinned": 61, 2024-04-24T09:25:05.256 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_expired": 0, 2024-04-24T09:25:05.256 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_with_caps": 0, 2024-04-24T09:25:05.257 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": 0, 2024-04-24T09:25:05.257 INFO:teuthology.orchestra.run.smithi114.stdout: "subtrees": 3, 2024-04-24T09:25:05.257 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse": 0, 2024-04-24T09:25:05.257 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_hit": 0, 2024-04-24T09:25:05.257 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_forward": 0, 2024-04-24T09:25:05.257 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_discover": 0, 2024-04-24T09:25:05.257 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_dir_fetch": 0, 2024-04-24T09:25:05.257 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_remote_ino": 0, 2024-04-24T09:25:05.258 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_lock": 0, 2024-04-24T09:25:05.258 INFO:teuthology.orchestra.run.smithi114.stdout: "load_cent": 0, 2024-04-24T09:25:05.258 INFO:teuthology.orchestra.run.smithi114.stdout: "q": 0, 2024-04-24T09:25:05.258 INFO:teuthology.orchestra.run.smithi114.stdout: "exported": 0, 2024-04-24T09:25:05.258 INFO:teuthology.orchestra.run.smithi114.stdout: "exported_inodes": 0, 2024-04-24T09:25:05.258 INFO:teuthology.orchestra.run.smithi114.stdout: "imported": 0, 2024-04-24T09:25:05.258 INFO:teuthology.orchestra.run.smithi114.stdout: "imported_inodes": 0, 2024-04-24T09:25:05.258 INFO:teuthology.orchestra.run.smithi114.stdout: "openino_dir_fetch": 0, 2024-04-24T09:25:05.258 INFO:teuthology.orchestra.run.smithi114.stdout: "openino_backtrace_fetch": 0, 2024-04-24T09:25:05.259 INFO:teuthology.orchestra.run.smithi114.stdout: "openino_peer_discover": 0, 2024-04-24T09:25:05.259 INFO:teuthology.orchestra.run.smithi114.stdout: "root_rfiles": 0, 2024-04-24T09:25:05.259 INFO:teuthology.orchestra.run.smithi114.stdout: "root_rbytes": 0, 2024-04-24T09:25:05.259 INFO:teuthology.orchestra.run.smithi114.stdout: "root_rsnaps": 0, 2024-04-24T09:25:05.259 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_backtrace_fetch": 0, 2024-04-24T09:25:05.259 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_set_tag": 0, 2024-04-24T09:25:05.259 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T09:25:05.260 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_inotable_repaired": 0, 2024-04-24T09:25:05.260 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_dir_inodes": 0, 2024-04-24T09:25:05.260 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T09:25:05.260 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_dirfrag_rstats": 0, 2024-04-24T09:25:05.260 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_file_inodes": 0, 2024-04-24T09:25:05.260 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_inode_file_caps": 0, 2024-04-24T09:25:05.260 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_revoke": 0, 2024-04-24T09:25:05.260 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_grant": 0, 2024-04-24T09:25:05.260 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T09:25:05.261 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T09:25:05.261 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T09:25:05.261 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_caps": 0, 2024-04-24T09:25:05.261 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_caps_dirty": 0, 2024-04-24T09:25:05.261 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_cap_release": 0, 2024-04-24T09:25:05.261 INFO:teuthology.orchestra.run.smithi114.stdout: "process_request_cap_release": 0 2024-04-24T09:25:05.261 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.261 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_cache": { 2024-04-24T09:25:05.262 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_update": 0, 2024-04-24T09:25:05.262 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_update_receipt": 0, 2024-04-24T09:25:05.262 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_try_discover": 0, 2024-04-24T09:25:05.262 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_send_discover": 0, 2024-04-24T09:25:05.262 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_handle_discover": 0, 2024-04-24T09:25:05.262 INFO:teuthology.orchestra.run.smithi114.stdout: "num_strays": 0, 2024-04-24T09:25:05.262 INFO:teuthology.orchestra.run.smithi114.stdout: "num_strays_delayed": 0, 2024-04-24T09:25:05.262 INFO:teuthology.orchestra.run.smithi114.stdout: "num_strays_enqueuing": 0, 2024-04-24T09:25:05.262 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_created": 0, 2024-04-24T09:25:05.263 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_enqueued": 0, 2024-04-24T09:25:05.263 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_reintegrated": 0, 2024-04-24T09:25:05.263 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_migrated": 0, 2024-04-24T09:25:05.263 INFO:teuthology.orchestra.run.smithi114.stdout: "num_recovering_processing": 0, 2024-04-24T09:25:05.263 INFO:teuthology.orchestra.run.smithi114.stdout: "num_recovering_enqueued": 0, 2024-04-24T09:25:05.263 INFO:teuthology.orchestra.run.smithi114.stdout: "num_recovering_prioritized": 0, 2024-04-24T09:25:05.263 INFO:teuthology.orchestra.run.smithi114.stdout: "recovery_started": 0, 2024-04-24T09:25:05.263 INFO:teuthology.orchestra.run.smithi114.stdout: "recovery_completed": 0, 2024-04-24T09:25:05.264 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_quiesce_path": 0, 2024-04-24T09:25:05.264 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_quiesce_inode": 0, 2024-04-24T09:25:05.264 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T09:25:05.264 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_exportdir": 0, 2024-04-24T09:25:05.264 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_flush": 0, 2024-04-24T09:25:05.264 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_fragmentdir": 0, 2024-04-24T09:25:05.264 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_fragstats": 0, 2024-04-24T09:25:05.264 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_inodestats": 0 2024-04-24T09:25:05.265 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.265 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_log": { 2024-04-24T09:25:05.265 INFO:teuthology.orchestra.run.smithi114.stdout: "evlrg": 0, 2024-04-24T09:25:05.265 INFO:teuthology.orchestra.run.smithi114.stdout: "evadd": 0, 2024-04-24T09:25:05.265 INFO:teuthology.orchestra.run.smithi114.stdout: "evex": 0, 2024-04-24T09:25:05.265 INFO:teuthology.orchestra.run.smithi114.stdout: "evtrm": 0, 2024-04-24T09:25:05.265 INFO:teuthology.orchestra.run.smithi114.stdout: "ev": 60, 2024-04-24T09:25:05.265 INFO:teuthology.orchestra.run.smithi114.stdout: "evexg": 0, 2024-04-24T09:25:05.265 INFO:teuthology.orchestra.run.smithi114.stdout: "evexd": 0, 2024-04-24T09:25:05.266 INFO:teuthology.orchestra.run.smithi114.stdout: "segadd": 0, 2024-04-24T09:25:05.266 INFO:teuthology.orchestra.run.smithi114.stdout: "segex": 0, 2024-04-24T09:25:05.266 INFO:teuthology.orchestra.run.smithi114.stdout: "segtrm": 0, 2024-04-24T09:25:05.266 INFO:teuthology.orchestra.run.smithi114.stdout: "seg": 2, 2024-04-24T09:25:05.266 INFO:teuthology.orchestra.run.smithi114.stdout: "segmjr": 2, 2024-04-24T09:25:05.266 INFO:teuthology.orchestra.run.smithi114.stdout: "segexg": 0, 2024-04-24T09:25:05.266 INFO:teuthology.orchestra.run.smithi114.stdout: "segexd": 0, 2024-04-24T09:25:05.266 INFO:teuthology.orchestra.run.smithi114.stdout: "expos": 4194304, 2024-04-24T09:25:05.267 INFO:teuthology.orchestra.run.smithi114.stdout: "wrpos": 4296190, 2024-04-24T09:25:05.267 INFO:teuthology.orchestra.run.smithi114.stdout: "rdpos": 4295616, 2024-04-24T09:25:05.267 INFO:teuthology.orchestra.run.smithi114.stdout: "jlat": { 2024-04-24T09:25:05.267 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:05.267 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:05.267 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:05.267 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.267 INFO:teuthology.orchestra.run.smithi114.stdout: "replayed": 60 2024-04-24T09:25:05.267 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.268 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_mem": { 2024-04-24T09:25:05.268 INFO:teuthology.orchestra.run.smithi114.stdout: "ino": 64, 2024-04-24T09:25:05.268 INFO:teuthology.orchestra.run.smithi114.stdout: "ino+": 64, 2024-04-24T09:25:05.268 INFO:teuthology.orchestra.run.smithi114.stdout: "ino-": 0, 2024-04-24T09:25:05.268 INFO:teuthology.orchestra.run.smithi114.stdout: "dir": 13, 2024-04-24T09:25:05.268 INFO:teuthology.orchestra.run.smithi114.stdout: "dir+": 13, 2024-04-24T09:25:05.268 INFO:teuthology.orchestra.run.smithi114.stdout: "dir-": 0, 2024-04-24T09:25:05.268 INFO:teuthology.orchestra.run.smithi114.stdout: "dn": 62, 2024-04-24T09:25:05.268 INFO:teuthology.orchestra.run.smithi114.stdout: "dn+": 62, 2024-04-24T09:25:05.269 INFO:teuthology.orchestra.run.smithi114.stdout: "dn-": 0, 2024-04-24T09:25:05.269 INFO:teuthology.orchestra.run.smithi114.stdout: "cap": 0, 2024-04-24T09:25:05.269 INFO:teuthology.orchestra.run.smithi114.stdout: "cap+": 0, 2024-04-24T09:25:05.269 INFO:teuthology.orchestra.run.smithi114.stdout: "cap-": 0, 2024-04-24T09:25:05.269 INFO:teuthology.orchestra.run.smithi114.stdout: "rss": 51816, 2024-04-24T09:25:05.269 INFO:teuthology.orchestra.run.smithi114.stdout: "heap": 215416 2024-04-24T09:25:05.269 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.269 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_server": { 2024-04-24T09:25:05.270 INFO:teuthology.orchestra.run.smithi114.stdout: "dispatch_client_request": 0, 2024-04-24T09:25:05.270 INFO:teuthology.orchestra.run.smithi114.stdout: "dispatch_server_request": 0, 2024-04-24T09:25:05.270 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_request": 0, 2024-04-24T09:25:05.270 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_session": 0, 2024-04-24T09:25:05.270 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_peer_request": 0, 2024-04-24T09:25:05.270 INFO:teuthology.orchestra.run.smithi114.stdout: "req_create_latency": { 2024-04-24T09:25:05.270 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:05.270 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:05.270 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:05.271 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.271 INFO:teuthology.orchestra.run.smithi114.stdout: "req_getattr_latency": { 2024-04-24T09:25:05.271 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:05.271 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:05.271 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:05.271 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.271 INFO:teuthology.orchestra.run.smithi114.stdout: "req_getfilelock_latency": { 2024-04-24T09:25:05.271 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:05.271 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:05.272 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:05.272 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.272 INFO:teuthology.orchestra.run.smithi114.stdout: "req_link_latency": { 2024-04-24T09:25:05.272 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:05.272 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:05.272 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:05.272 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.272 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookup_latency": { 2024-04-24T09:25:05.272 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:05.273 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:05.273 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:05.273 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.273 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookuphash_latency": { 2024-04-24T09:25:05.273 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:05.273 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:05.274 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:05.274 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.274 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupino_latency": { 2024-04-24T09:25:05.274 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:05.274 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:05.274 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:05.274 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.274 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupname_latency": { 2024-04-24T09:25:05.275 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:05.275 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:05.275 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:05.275 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.275 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupparent_latency": { 2024-04-24T09:25:05.275 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:05.275 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:05.276 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:05.276 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.276 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupsnap_latency": { 2024-04-24T09:25:05.276 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:05.276 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:05.276 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:05.276 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.277 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lssnap_latency": { 2024-04-24T09:25:05.277 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:05.277 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:05.277 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:05.277 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.277 INFO:teuthology.orchestra.run.smithi114.stdout: "req_mkdir_latency": { 2024-04-24T09:25:05.277 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:05.278 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:05.278 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:05.278 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.279 INFO:teuthology.orchestra.run.smithi114.stdout: "req_mknod_latency": { 2024-04-24T09:25:05.279 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:05.279 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:05.279 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:05.279 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.279 INFO:teuthology.orchestra.run.smithi114.stdout: "req_mksnap_latency": { 2024-04-24T09:25:05.280 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:05.280 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:05.280 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:05.280 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.280 INFO:teuthology.orchestra.run.smithi114.stdout: "req_open_latency": { 2024-04-24T09:25:05.280 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:05.280 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:05.281 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:05.281 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.281 INFO:teuthology.orchestra.run.smithi114.stdout: "req_readdir_latency": { 2024-04-24T09:25:05.281 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:05.281 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:05.281 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:05.282 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.282 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rename_latency": { 2024-04-24T09:25:05.282 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:05.282 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:05.282 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:05.282 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.282 INFO:teuthology.orchestra.run.smithi114.stdout: "req_renamesnap_latency": { 2024-04-24T09:25:05.283 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:05.283 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:05.283 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:05.283 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.283 INFO:teuthology.orchestra.run.smithi114.stdout: "req_snapdiff_latency": { 2024-04-24T09:25:05.283 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:05.284 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:05.284 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:05.284 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.284 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rmdir_latency": { 2024-04-24T09:25:05.284 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:05.284 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:05.285 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:05.285 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.285 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rmsnap_latency": { 2024-04-24T09:25:05.285 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:05.285 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:05.285 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:05.285 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.286 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rmxattr_latency": { 2024-04-24T09:25:05.286 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:05.286 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:05.286 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:05.286 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.286 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setattr_latency": { 2024-04-24T09:25:05.286 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:05.286 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:05.287 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:05.287 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.287 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setdirlayout_latency": { 2024-04-24T09:25:05.287 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:05.287 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:05.287 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:05.287 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.287 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setfilelock_latency": { 2024-04-24T09:25:05.287 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:05.288 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:05.288 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:05.288 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.288 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setlayout_latency": { 2024-04-24T09:25:05.288 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:05.288 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:05.288 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:05.288 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.288 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setxattr_latency": { 2024-04-24T09:25:05.289 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:05.289 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:05.289 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:05.289 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.289 INFO:teuthology.orchestra.run.smithi114.stdout: "req_symlink_latency": { 2024-04-24T09:25:05.289 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:05.289 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:05.289 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:05.289 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.290 INFO:teuthology.orchestra.run.smithi114.stdout: "req_unlink_latency": { 2024-04-24T09:25:05.290 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:05.290 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:05.290 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:05.290 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.290 INFO:teuthology.orchestra.run.smithi114.stdout: "cap_revoke_eviction": 0, 2024-04-24T09:25:05.290 INFO:teuthology.orchestra.run.smithi114.stdout: "cap_acquisition_throttle": 0, 2024-04-24T09:25:05.290 INFO:teuthology.orchestra.run.smithi114.stdout: "req_getvxattr_latency": { 2024-04-24T09:25:05.291 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:05.291 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:05.293 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:05.294 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-04-24T09:25:05.294 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.294 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_sessions": { 2024-04-24T09:25:05.294 INFO:teuthology.orchestra.run.smithi114.stdout: "session_count": 1, 2024-04-24T09:25:05.294 INFO:teuthology.orchestra.run.smithi114.stdout: "session_add": 3, 2024-04-24T09:25:05.294 INFO:teuthology.orchestra.run.smithi114.stdout: "session_remove": 2, 2024-04-24T09:25:05.294 INFO:teuthology.orchestra.run.smithi114.stdout: "sessions_open": 1, 2024-04-24T09:25:05.294 INFO:teuthology.orchestra.run.smithi114.stdout: "sessions_stale": 0, 2024-04-24T09:25:05.294 INFO:teuthology.orchestra.run.smithi114.stdout: "total_load": 0, 2024-04-24T09:25:05.295 INFO:teuthology.orchestra.run.smithi114.stdout: "average_load": 0, 2024-04-24T09:25:05.295 INFO:teuthology.orchestra.run.smithi114.stdout: "avg_session_uptime": 1758, 2024-04-24T09:25:05.295 INFO:teuthology.orchestra.run.smithi114.stdout: "mdthresh_evicted": 0 2024-04-24T09:25:05.295 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.295 INFO:teuthology.orchestra.run.smithi114.stdout: "mempool": { 2024-04-24T09:25:05.295 INFO:teuthology.orchestra.run.smithi114.stdout: "bloom_filter_bytes": 0, 2024-04-24T09:25:05.295 INFO:teuthology.orchestra.run.smithi114.stdout: "bloom_filter_items": 0, 2024-04-24T09:25:05.295 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T09:25:05.295 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_alloc_items": 0, 2024-04-24T09:25:05.295 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T09:25:05.296 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_data_items": 0, 2024-04-24T09:25:05.296 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T09:25:05.296 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T09:25:05.296 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T09:25:05.296 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T09:25:05.296 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T09:25:05.296 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_other_items": 0, 2024-04-24T09:25:05.296 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T09:25:05.296 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T09:25:05.297 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_extent_bytes": 0, 2024-04-24T09:25:05.297 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_extent_items": 0, 2024-04-24T09:25:05.297 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_blob_bytes": 0, 2024-04-24T09:25:05.297 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_blob_items": 0, 2024-04-24T09:25:05.297 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T09:25:05.297 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T09:25:05.297 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T09:25:05.297 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T09:25:05.297 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T09:25:05.298 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_fsck_items": 0, 2024-04-24T09:25:05.298 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_txc_bytes": 0, 2024-04-24T09:25:05.298 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_txc_items": 0, 2024-04-24T09:25:05.298 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T09:25:05.298 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T09:25:05.298 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_bytes": 0, 2024-04-24T09:25:05.298 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_items": 0, 2024-04-24T09:25:05.298 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_bytes": 0, 2024-04-24T09:25:05.299 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_items": 0, 2024-04-24T09:25:05.299 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T09:25:05.299 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_reader_items": 0, 2024-04-24T09:25:05.299 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T09:25:05.299 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_writer_items": 0, 2024-04-24T09:25:05.299 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_anon_bytes": 191326, 2024-04-24T09:25:05.299 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_anon_items": 105, 2024-04-24T09:25:05.299 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_meta_bytes": 0, 2024-04-24T09:25:05.299 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_meta_items": 0, 2024-04-24T09:25:05.299 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_bytes": 0, 2024-04-24T09:25:05.300 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_items": 0, 2024-04-24T09:25:05.300 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_mapbl_bytes": 0, 2024-04-24T09:25:05.300 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_mapbl_items": 0, 2024-04-24T09:25:05.300 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_pglog_bytes": 0, 2024-04-24T09:25:05.300 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_pglog_items": 0, 2024-04-24T09:25:05.300 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_bytes": 7624, 2024-04-24T09:25:05.300 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_items": 160, 2024-04-24T09:25:05.300 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T09:25:05.300 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_mapping_items": 0, 2024-04-24T09:25:05.301 INFO:teuthology.orchestra.run.smithi114.stdout: "pgmap_bytes": 0, 2024-04-24T09:25:05.301 INFO:teuthology.orchestra.run.smithi114.stdout: "pgmap_items": 0, 2024-04-24T09:25:05.301 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_co_bytes": 207506, 2024-04-24T09:25:05.301 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_co_items": 1914, 2024-04-24T09:25:05.301 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_1_bytes": 0, 2024-04-24T09:25:05.301 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_1_items": 0, 2024-04-24T09:25:05.301 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_2_bytes": 0, 2024-04-24T09:25:05.301 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_2_items": 0 2024-04-24T09:25:05.302 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.302 INFO:teuthology.orchestra.run.smithi114.stdout: "objecter": { 2024-04-24T09:25:05.302 INFO:teuthology.orchestra.run.smithi114.stdout: "op_active": 0, 2024-04-24T09:25:05.302 INFO:teuthology.orchestra.run.smithi114.stdout: "op_laggy": 0, 2024-04-24T09:25:05.302 INFO:teuthology.orchestra.run.smithi114.stdout: "op_send": 989, 2024-04-24T09:25:05.302 INFO:teuthology.orchestra.run.smithi114.stdout: "op_send_bytes": 21, 2024-04-24T09:25:05.302 INFO:teuthology.orchestra.run.smithi114.stdout: "op_resend": 3, 2024-04-24T09:25:05.302 INFO:teuthology.orchestra.run.smithi114.stdout: "op_reply": 989, 2024-04-24T09:25:05.303 INFO:teuthology.orchestra.run.smithi114.stdout: "op_latency": { 2024-04-24T09:25:05.303 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 989, 2024-04-24T09:25:05.303 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.727984844, 2024-04-24T09:25:05.303 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000736081 2024-04-24T09:25:05.303 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.303 INFO:teuthology.orchestra.run.smithi114.stdout: "op_inflight": 0, 2024-04-24T09:25:05.303 INFO:teuthology.orchestra.run.smithi114.stdout: "oplen_avg": { 2024-04-24T09:25:05.303 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 989, 2024-04-24T09:25:05.303 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 990 2024-04-24T09:25:05.303 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.304 INFO:teuthology.orchestra.run.smithi114.stdout: "op": 989, 2024-04-24T09:25:05.304 INFO:teuthology.orchestra.run.smithi114.stdout: "op_r": 989, 2024-04-24T09:25:05.304 INFO:teuthology.orchestra.run.smithi114.stdout: "op_w": 0, 2024-04-24T09:25:05.304 INFO:teuthology.orchestra.run.smithi114.stdout: "op_rmw": 0, 2024-04-24T09:25:05.304 INFO:teuthology.orchestra.run.smithi114.stdout: "op_pg": 0, 2024-04-24T09:25:05.304 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_stat": 626, 2024-04-24T09:25:05.304 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_create": 0, 2024-04-24T09:25:05.304 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_read": 361, 2024-04-24T09:25:05.304 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_write": 0, 2024-04-24T09:25:05.304 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_writefull": 0, 2024-04-24T09:25:05.305 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_writesame": 0, 2024-04-24T09:25:05.305 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_append": 0, 2024-04-24T09:25:05.305 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_zero": 0, 2024-04-24T09:25:05.305 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_truncate": 0, 2024-04-24T09:25:05.305 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_delete": 0, 2024-04-24T09:25:05.305 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_mapext": 0, 2024-04-24T09:25:05.305 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_sparse_read": 0, 2024-04-24T09:25:05.305 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_clonerange": 0, 2024-04-24T09:25:05.305 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_getxattr": 1, 2024-04-24T09:25:05.306 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_setxattr": 0, 2024-04-24T09:25:05.306 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_cmpxattr": 0, 2024-04-24T09:25:05.306 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_rmxattr": 0, 2024-04-24T09:25:05.306 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_resetxattrs": 0, 2024-04-24T09:25:05.306 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_call": 0, 2024-04-24T09:25:05.306 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_watch": 0, 2024-04-24T09:25:05.306 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_notify": 0, 2024-04-24T09:25:05.306 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T09:25:05.306 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_pgls": 0, 2024-04-24T09:25:05.307 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_pgls_filter": 0, 2024-04-24T09:25:05.307 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_other": 0, 2024-04-24T09:25:05.307 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_active": 0, 2024-04-24T09:25:05.307 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_send": 0, 2024-04-24T09:25:05.307 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_resend": 0, 2024-04-24T09:25:05.307 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_ping": 0, 2024-04-24T09:25:05.307 INFO:teuthology.orchestra.run.smithi114.stdout: "poolop_active": 0, 2024-04-24T09:25:05.307 INFO:teuthology.orchestra.run.smithi114.stdout: "poolop_send": 0, 2024-04-24T09:25:05.307 INFO:teuthology.orchestra.run.smithi114.stdout: "poolop_resend": 0, 2024-04-24T09:25:05.308 INFO:teuthology.orchestra.run.smithi114.stdout: "poolstat_active": 0, 2024-04-24T09:25:05.308 INFO:teuthology.orchestra.run.smithi114.stdout: "poolstat_send": 0, 2024-04-24T09:25:05.308 INFO:teuthology.orchestra.run.smithi114.stdout: "poolstat_resend": 0, 2024-04-24T09:25:05.308 INFO:teuthology.orchestra.run.smithi114.stdout: "statfs_active": 0, 2024-04-24T09:25:05.308 INFO:teuthology.orchestra.run.smithi114.stdout: "statfs_send": 0, 2024-04-24T09:25:05.308 INFO:teuthology.orchestra.run.smithi114.stdout: "statfs_resend": 0, 2024-04-24T09:25:05.308 INFO:teuthology.orchestra.run.smithi114.stdout: "command_active": 0, 2024-04-24T09:25:05.308 INFO:teuthology.orchestra.run.smithi114.stdout: "command_send": 0, 2024-04-24T09:25:05.308 INFO:teuthology.orchestra.run.smithi114.stdout: "command_resend": 0, 2024-04-24T09:25:05.308 INFO:teuthology.orchestra.run.smithi114.stdout: "map_epoch": 88, 2024-04-24T09:25:05.309 INFO:teuthology.orchestra.run.smithi114.stdout: "map_full": 0, 2024-04-24T09:25:05.309 INFO:teuthology.orchestra.run.smithi114.stdout: "map_inc": 2, 2024-04-24T09:25:05.309 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_sessions": 5, 2024-04-24T09:25:05.309 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_session_open": 5, 2024-04-24T09:25:05.309 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_session_close": 0, 2024-04-24T09:25:05.309 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_laggy": 0, 2024-04-24T09:25:05.309 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_wr": 0, 2024-04-24T09:25:05.310 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_rd": 2, 2024-04-24T09:25:05.310 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_del": 0 2024-04-24T09:25:05.310 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.310 INFO:teuthology.orchestra.run.smithi114.stdout: "oft": { 2024-04-24T09:25:05.310 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_objs": 0, 2024-04-24T09:25:05.310 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_kv_pairs": 0, 2024-04-24T09:25:05.310 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_updates": 0, 2024-04-24T09:25:05.311 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_removes": 0 2024-04-24T09:25:05.311 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.311 INFO:teuthology.orchestra.run.smithi114.stdout: "osd-slow-ops": { 2024-04-24T09:25:05.311 INFO:teuthology.orchestra.run.smithi114.stdout: "slow_ops_count": 0 2024-04-24T09:25:05.311 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.311 INFO:teuthology.orchestra.run.smithi114.stdout: "purge_queue": { 2024-04-24T09:25:05.311 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing_ops": 0, 2024-04-24T09:25:05.311 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing_ops_high_water": 0, 2024-04-24T09:25:05.312 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing": 0, 2024-04-24T09:25:05.312 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing_high_water": 0, 2024-04-24T09:25:05.312 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executed_ops": 0, 2024-04-24T09:25:05.312 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executed": 0, 2024-04-24T09:25:05.312 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_item_in_journal": 0 2024-04-24T09:25:05.312 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.313 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T09:25:05.313 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-04-24T09:25:05.313 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 104857600, 2024-04-24T09:25:05.313 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-04-24T09:25:05.313 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 1115, 2024-04-24T09:25:05.313 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 1064528, 2024-04-24T09:25:05.313 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-04-24T09:25:05.313 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 1115, 2024-04-24T09:25:05.314 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-04-24T09:25:05.314 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-04-24T09:25:05.314 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 1115, 2024-04-24T09:25:05.314 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 1064528, 2024-04-24T09:25:05.314 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-04-24T09:25:05.314 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:05.314 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:05.315 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:05.315 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-04-24T09:25:05.315 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.315 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-objecter_bytes": { 2024-04-24T09:25:05.315 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-04-24T09:25:05.315 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 104857600, 2024-04-24T09:25:05.315 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-04-24T09:25:05.315 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 0, 2024-04-24T09:25:05.315 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 0, 2024-04-24T09:25:05.316 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-04-24T09:25:05.316 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 0, 2024-04-24T09:25:05.316 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 989, 2024-04-24T09:25:05.316 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 101891, 2024-04-24T09:25:05.316 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 989, 2024-04-24T09:25:05.316 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 101891, 2024-04-24T09:25:05.316 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-04-24T09:25:05.317 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:05.317 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:05.317 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:05.317 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-04-24T09:25:05.317 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.317 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-objecter_ops": { 2024-04-24T09:25:05.318 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-04-24T09:25:05.318 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 1024, 2024-04-24T09:25:05.318 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-04-24T09:25:05.318 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 0, 2024-04-24T09:25:05.318 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 0, 2024-04-24T09:25:05.318 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-04-24T09:25:05.318 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 0, 2024-04-24T09:25:05.318 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 989, 2024-04-24T09:25:05.319 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 989, 2024-04-24T09:25:05.319 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 989, 2024-04-24T09:25:05.319 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 989, 2024-04-24T09:25:05.319 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-04-24T09:25:05.319 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:05.319 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:05.320 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:05.320 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-04-24T09:25:05.320 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.320 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-write_buf_throttle": { 2024-04-24T09:25:05.320 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-04-24T09:25:05.320 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 3758096384, 2024-04-24T09:25:05.320 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-04-24T09:25:05.321 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 0, 2024-04-24T09:25:05.321 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 0, 2024-04-24T09:25:05.321 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-04-24T09:25:05.321 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 0, 2024-04-24T09:25:05.321 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-04-24T09:25:05.322 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-04-24T09:25:05.322 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 0, 2024-04-24T09:25:05.322 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 0, 2024-04-24T09:25:05.322 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-04-24T09:25:05.322 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:05.322 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:05.322 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:05.322 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-04-24T09:25:05.322 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:05.323 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-write_buf_throttle-0x555980a2c6e0": { 2024-04-24T09:25:05.323 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-04-24T09:25:05.323 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 3758096384, 2024-04-24T09:25:05.323 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-04-24T09:25:05.323 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 0, 2024-04-24T09:25:05.323 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 0, 2024-04-24T09:25:05.323 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-04-24T09:25:05.323 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 0, 2024-04-24T09:25:05.323 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-04-24T09:25:05.324 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-04-24T09:25:05.324 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 0, 2024-04-24T09:25:05.324 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 0, 2024-04-24T09:25:05.324 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-04-24T09:25:05.324 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:05.324 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:05.324 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:05.324 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-04-24T09:25:05.324 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-04-24T09:25:05.324 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-04-24T09:25:06.039 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-24T09:25:06.040 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-24T09:25:06.040 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-04-24T09:25:06.040 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-04-24T09:25:06.040 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-04-24T09:25:06.040 DEBUG:tasks.check_counter:Getting stats from h 2024-04-24T09:25:06.041 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-24T09:25:06.282 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T09:25:07.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:06 smithi090 ceph-mon[27825]: pgmap v502: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.3 KiB/s wr, 14 op/s 2024-04-24T09:25:07.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:06 smithi149 ceph-mon[32592]: pgmap v502: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.3 KiB/s wr, 14 op/s 2024-04-24T09:25:07.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:06 smithi114 ceph-mon[32501]: pgmap v502: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.3 KiB/s wr, 14 op/s 2024-04-24T09:25:08.092 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-04-24T09:25:08.093 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T09:25:08.093 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 929, 2024-04-24T09:25:08.093 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 929, 2024-04-24T09:25:08.094 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 391971, 2024-04-24T09:25:08.094 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 294605, 2024-04-24T09:25:08.094 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 6, 2024-04-24T09:25:08.094 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 2, 2024-04-24T09:25:08.094 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.075039796, 2024-04-24T09:25:08.095 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.024952165, 2024-04-24T09:25:08.095 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.018927911, 2024-04-24T09:25:08.095 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.024319761, 2024-04-24T09:25:08.095 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:25:08.095 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 928, 2024-04-24T09:25:08.095 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.030911038, 2024-04-24T09:25:08.095 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000033309 2024-04-24T09:25:08.095 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.095 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:25:08.096 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:08.096 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:08.096 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:08.096 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.096 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-04-24T09:25:08.096 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 448 2024-04-24T09:25:08.096 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.096 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T09:25:08.096 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 4, 2024-04-24T09:25:08.097 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 83, 2024-04-24T09:25:08.097 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 876, 2024-04-24T09:25:08.097 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 293754, 2024-04-24T09:25:08.097 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 5, 2024-04-24T09:25:08.097 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 3, 2024-04-24T09:25:08.097 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.005452707, 2024-04-24T09:25:08.097 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.003469294, 2024-04-24T09:25:08.097 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.001061781, 2024-04-24T09:25:08.097 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.000047960, 2024-04-24T09:25:08.098 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:25:08.098 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 80, 2024-04-24T09:25:08.098 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.007708222, 2024-04-24T09:25:08.098 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000096352 2024-04-24T09:25:08.098 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.098 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:25:08.098 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:08.098 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:08.099 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:08.099 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.099 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-24T09:25:08.099 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 293056 2024-04-24T09:25:08.099 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.099 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T09:25:08.099 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 124, 2024-04-24T09:25:08.099 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 112, 2024-04-24T09:25:08.099 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 787916, 2024-04-24T09:25:08.100 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 64750, 2024-04-24T09:25:08.100 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 3, 2024-04-24T09:25:08.100 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 2, 2024-04-24T09:25:08.100 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.020802865, 2024-04-24T09:25:08.100 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.006586166, 2024-04-24T09:25:08.100 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.015439344, 2024-04-24T09:25:08.100 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.002390745, 2024-04-24T09:25:08.100 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:25:08.100 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 111, 2024-04-24T09:25:08.101 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.006384512, 2024-04-24T09:25:08.101 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000057518 2024-04-24T09:25:08.101 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.101 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:25:08.101 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:08.101 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:08.101 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:08.101 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.101 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 787648, 2024-04-24T09:25:08.102 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 64432 2024-04-24T09:25:08.102 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.102 INFO:teuthology.orchestra.run.smithi114.stdout: "cct": { 2024-04-24T09:25:08.102 INFO:teuthology.orchestra.run.smithi114.stdout: "total_workers": 1, 2024-04-24T09:25:08.102 INFO:teuthology.orchestra.run.smithi114.stdout: "unhealthy_workers": 0 2024-04-24T09:25:08.102 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.102 INFO:teuthology.orchestra.run.smithi114.stdout: "finisher-MDSRank": { 2024-04-24T09:25:08.102 INFO:teuthology.orchestra.run.smithi114.stdout: "queue_len": 0, 2024-04-24T09:25:08.102 INFO:teuthology.orchestra.run.smithi114.stdout: "complete_latency": { 2024-04-24T09:25:08.103 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 1642, 2024-04-24T09:25:08.103 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.039165318, 2024-04-24T09:25:08.103 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000023852 2024-04-24T09:25:08.103 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-04-24T09:25:08.103 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.103 INFO:teuthology.orchestra.run.smithi114.stdout: "finisher-PurgeQueue": { 2024-04-24T09:25:08.103 INFO:teuthology.orchestra.run.smithi114.stdout: "queue_len": 0, 2024-04-24T09:25:08.103 INFO:teuthology.orchestra.run.smithi114.stdout: "complete_latency": { 2024-04-24T09:25:08.103 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:08.103 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:08.104 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:08.104 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-04-24T09:25:08.104 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.104 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": { 2024-04-24T09:25:08.104 INFO:teuthology.orchestra.run.smithi114.stdout: "request": 0, 2024-04-24T09:25:08.104 INFO:teuthology.orchestra.run.smithi114.stdout: "reply": 0, 2024-04-24T09:25:08.104 INFO:teuthology.orchestra.run.smithi114.stdout: "reply_latency": { 2024-04-24T09:25:08.104 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:08.104 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:08.105 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:08.105 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.105 INFO:teuthology.orchestra.run.smithi114.stdout: "slow_reply": 0, 2024-04-24T09:25:08.105 INFO:teuthology.orchestra.run.smithi114.stdout: "forward": 0, 2024-04-24T09:25:08.105 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_fetch_complete": 0, 2024-04-24T09:25:08.105 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_fetch_keys": 0, 2024-04-24T09:25:08.105 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_commit": 0, 2024-04-24T09:25:08.105 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_split": 0, 2024-04-24T09:25:08.105 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_merge": 0, 2024-04-24T09:25:08.105 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes": 49, 2024-04-24T09:25:08.106 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_top": 2, 2024-04-24T09:25:08.106 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_bottom": 47, 2024-04-24T09:25:08.106 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_pin_tail": 0, 2024-04-24T09:25:08.106 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_pinned": 46, 2024-04-24T09:25:08.106 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_expired": 0, 2024-04-24T09:25:08.106 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_with_caps": 0, 2024-04-24T09:25:08.106 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": 0, 2024-04-24T09:25:08.106 INFO:teuthology.orchestra.run.smithi114.stdout: "subtrees": 5, 2024-04-24T09:25:08.106 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse": 0, 2024-04-24T09:25:08.107 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_hit": 0, 2024-04-24T09:25:08.107 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_forward": 0, 2024-04-24T09:25:08.107 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_discover": 0, 2024-04-24T09:25:08.107 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_dir_fetch": 0, 2024-04-24T09:25:08.107 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_remote_ino": 0, 2024-04-24T09:25:08.107 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_lock": 0, 2024-04-24T09:25:08.107 INFO:teuthology.orchestra.run.smithi114.stdout: "load_cent": 0, 2024-04-24T09:25:08.107 INFO:teuthology.orchestra.run.smithi114.stdout: "q": 0, 2024-04-24T09:25:08.107 INFO:teuthology.orchestra.run.smithi114.stdout: "exported": 0, 2024-04-24T09:25:08.108 INFO:teuthology.orchestra.run.smithi114.stdout: "exported_inodes": 0, 2024-04-24T09:25:08.108 INFO:teuthology.orchestra.run.smithi114.stdout: "imported": 0, 2024-04-24T09:25:08.108 INFO:teuthology.orchestra.run.smithi114.stdout: "imported_inodes": 0, 2024-04-24T09:25:08.108 INFO:teuthology.orchestra.run.smithi114.stdout: "openino_dir_fetch": 0, 2024-04-24T09:25:08.108 INFO:teuthology.orchestra.run.smithi114.stdout: "openino_backtrace_fetch": 0, 2024-04-24T09:25:08.108 INFO:teuthology.orchestra.run.smithi114.stdout: "openino_peer_discover": 0, 2024-04-24T09:25:08.108 INFO:teuthology.orchestra.run.smithi114.stdout: "root_rfiles": 0, 2024-04-24T09:25:08.108 INFO:teuthology.orchestra.run.smithi114.stdout: "root_rbytes": 0, 2024-04-24T09:25:08.108 INFO:teuthology.orchestra.run.smithi114.stdout: "root_rsnaps": 0, 2024-04-24T09:25:08.109 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_backtrace_fetch": 0, 2024-04-24T09:25:08.109 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_set_tag": 0, 2024-04-24T09:25:08.109 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T09:25:08.109 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_inotable_repaired": 0, 2024-04-24T09:25:08.109 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_dir_inodes": 0, 2024-04-24T09:25:08.109 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T09:25:08.109 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_dirfrag_rstats": 0, 2024-04-24T09:25:08.109 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_file_inodes": 0, 2024-04-24T09:25:08.109 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_inode_file_caps": 0, 2024-04-24T09:25:08.110 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_revoke": 0, 2024-04-24T09:25:08.110 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_grant": 0, 2024-04-24T09:25:08.110 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T09:25:08.110 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T09:25:08.110 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T09:25:08.110 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_caps": 0, 2024-04-24T09:25:08.110 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_caps_dirty": 0, 2024-04-24T09:25:08.110 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_cap_release": 0, 2024-04-24T09:25:08.111 INFO:teuthology.orchestra.run.smithi114.stdout: "process_request_cap_release": 0 2024-04-24T09:25:08.111 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.111 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_cache": { 2024-04-24T09:25:08.111 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_update": 0, 2024-04-24T09:25:08.111 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_update_receipt": 0, 2024-04-24T09:25:08.111 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_try_discover": 0, 2024-04-24T09:25:08.111 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_send_discover": 0, 2024-04-24T09:25:08.111 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_handle_discover": 0, 2024-04-24T09:25:08.111 INFO:teuthology.orchestra.run.smithi114.stdout: "num_strays": 0, 2024-04-24T09:25:08.112 INFO:teuthology.orchestra.run.smithi114.stdout: "num_strays_delayed": 0, 2024-04-24T09:25:08.112 INFO:teuthology.orchestra.run.smithi114.stdout: "num_strays_enqueuing": 0, 2024-04-24T09:25:08.112 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_created": 0, 2024-04-24T09:25:08.112 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_enqueued": 0, 2024-04-24T09:25:08.112 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_reintegrated": 0, 2024-04-24T09:25:08.112 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_migrated": 0, 2024-04-24T09:25:08.112 INFO:teuthology.orchestra.run.smithi114.stdout: "num_recovering_processing": 0, 2024-04-24T09:25:08.112 INFO:teuthology.orchestra.run.smithi114.stdout: "num_recovering_enqueued": 0, 2024-04-24T09:25:08.112 INFO:teuthology.orchestra.run.smithi114.stdout: "num_recovering_prioritized": 0, 2024-04-24T09:25:08.113 INFO:teuthology.orchestra.run.smithi114.stdout: "recovery_started": 0, 2024-04-24T09:25:08.113 INFO:teuthology.orchestra.run.smithi114.stdout: "recovery_completed": 0, 2024-04-24T09:25:08.113 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_quiesce_path": 0, 2024-04-24T09:25:08.113 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_quiesce_inode": 0, 2024-04-24T09:25:08.113 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T09:25:08.113 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_exportdir": 0, 2024-04-24T09:25:08.113 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_flush": 0, 2024-04-24T09:25:08.113 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_fragmentdir": 0, 2024-04-24T09:25:08.114 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_fragstats": 0, 2024-04-24T09:25:08.114 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_inodestats": 0 2024-04-24T09:25:08.114 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.114 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_log": { 2024-04-24T09:25:08.114 INFO:teuthology.orchestra.run.smithi114.stdout: "evlrg": 0, 2024-04-24T09:25:08.114 INFO:teuthology.orchestra.run.smithi114.stdout: "evadd": 0, 2024-04-24T09:25:08.114 INFO:teuthology.orchestra.run.smithi114.stdout: "evex": 0, 2024-04-24T09:25:08.114 INFO:teuthology.orchestra.run.smithi114.stdout: "evtrm": 0, 2024-04-24T09:25:08.114 INFO:teuthology.orchestra.run.smithi114.stdout: "ev": 73, 2024-04-24T09:25:08.115 INFO:teuthology.orchestra.run.smithi114.stdout: "evexg": 0, 2024-04-24T09:25:08.115 INFO:teuthology.orchestra.run.smithi114.stdout: "evexd": 0, 2024-04-24T09:25:08.115 INFO:teuthology.orchestra.run.smithi114.stdout: "segadd": 0, 2024-04-24T09:25:08.115 INFO:teuthology.orchestra.run.smithi114.stdout: "segex": 0, 2024-04-24T09:25:08.115 INFO:teuthology.orchestra.run.smithi114.stdout: "segtrm": 0, 2024-04-24T09:25:08.115 INFO:teuthology.orchestra.run.smithi114.stdout: "seg": 2, 2024-04-24T09:25:08.115 INFO:teuthology.orchestra.run.smithi114.stdout: "segmjr": 2, 2024-04-24T09:25:08.115 INFO:teuthology.orchestra.run.smithi114.stdout: "segexg": 0, 2024-04-24T09:25:08.115 INFO:teuthology.orchestra.run.smithi114.stdout: "segexd": 0, 2024-04-24T09:25:08.115 INFO:teuthology.orchestra.run.smithi114.stdout: "expos": 4194304, 2024-04-24T09:25:08.116 INFO:teuthology.orchestra.run.smithi114.stdout: "wrpos": 4317718, 2024-04-24T09:25:08.116 INFO:teuthology.orchestra.run.smithi114.stdout: "rdpos": 4317144, 2024-04-24T09:25:08.116 INFO:teuthology.orchestra.run.smithi114.stdout: "jlat": { 2024-04-24T09:25:08.116 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:08.116 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:08.116 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:08.117 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.117 INFO:teuthology.orchestra.run.smithi114.stdout: "replayed": 73 2024-04-24T09:25:08.117 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.117 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_mem": { 2024-04-24T09:25:08.117 INFO:teuthology.orchestra.run.smithi114.stdout: "ino": 38, 2024-04-24T09:25:08.117 INFO:teuthology.orchestra.run.smithi114.stdout: "ino+": 45, 2024-04-24T09:25:08.117 INFO:teuthology.orchestra.run.smithi114.stdout: "ino-": 7, 2024-04-24T09:25:08.117 INFO:teuthology.orchestra.run.smithi114.stdout: "dir": 18, 2024-04-24T09:25:08.118 INFO:teuthology.orchestra.run.smithi114.stdout: "dir+": 18, 2024-04-24T09:25:08.118 INFO:teuthology.orchestra.run.smithi114.stdout: "dir-": 0, 2024-04-24T09:25:08.118 INFO:teuthology.orchestra.run.smithi114.stdout: "dn": 49, 2024-04-24T09:25:08.118 INFO:teuthology.orchestra.run.smithi114.stdout: "dn+": 49, 2024-04-24T09:25:08.118 INFO:teuthology.orchestra.run.smithi114.stdout: "dn-": 0, 2024-04-24T09:25:08.118 INFO:teuthology.orchestra.run.smithi114.stdout: "cap": 0, 2024-04-24T09:25:08.118 INFO:teuthology.orchestra.run.smithi114.stdout: "cap+": 0, 2024-04-24T09:25:08.119 INFO:teuthology.orchestra.run.smithi114.stdout: "cap-": 0, 2024-04-24T09:25:08.119 INFO:teuthology.orchestra.run.smithi114.stdout: "rss": 49612, 2024-04-24T09:25:08.119 INFO:teuthology.orchestra.run.smithi114.stdout: "heap": 215416 2024-04-24T09:25:08.119 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.119 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_server": { 2024-04-24T09:25:08.119 INFO:teuthology.orchestra.run.smithi114.stdout: "dispatch_client_request": 0, 2024-04-24T09:25:08.119 INFO:teuthology.orchestra.run.smithi114.stdout: "dispatch_server_request": 0, 2024-04-24T09:25:08.119 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_request": 0, 2024-04-24T09:25:08.120 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_session": 0, 2024-04-24T09:25:08.120 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_peer_request": 0, 2024-04-24T09:25:08.120 INFO:teuthology.orchestra.run.smithi114.stdout: "req_create_latency": { 2024-04-24T09:25:08.120 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:08.120 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:08.120 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:08.121 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.121 INFO:teuthology.orchestra.run.smithi114.stdout: "req_getattr_latency": { 2024-04-24T09:25:08.121 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:08.121 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:08.121 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:08.121 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.121 INFO:teuthology.orchestra.run.smithi114.stdout: "req_getfilelock_latency": { 2024-04-24T09:25:08.121 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:08.122 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:08.122 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:08.122 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.122 INFO:teuthology.orchestra.run.smithi114.stdout: "req_link_latency": { 2024-04-24T09:25:08.122 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:08.122 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:08.122 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:08.123 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.123 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookup_latency": { 2024-04-24T09:25:08.123 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:08.123 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:08.123 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:08.123 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.123 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookuphash_latency": { 2024-04-24T09:25:08.123 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:08.123 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:08.124 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:08.124 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.124 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupino_latency": { 2024-04-24T09:25:08.124 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:08.124 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:08.124 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:08.125 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.125 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupname_latency": { 2024-04-24T09:25:08.125 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:08.125 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:08.125 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:08.125 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.125 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupparent_latency": { 2024-04-24T09:25:08.126 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:08.126 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:08.126 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:08.126 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.126 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupsnap_latency": { 2024-04-24T09:25:08.126 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:08.126 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:08.127 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:08.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.127 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lssnap_latency": { 2024-04-24T09:25:08.127 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:08.127 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:08.127 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:08.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.128 INFO:teuthology.orchestra.run.smithi114.stdout: "req_mkdir_latency": { 2024-04-24T09:25:08.128 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:08.128 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:08.129 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:08.129 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.129 INFO:teuthology.orchestra.run.smithi114.stdout: "req_mknod_latency": { 2024-04-24T09:25:08.129 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:08.129 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:08.129 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:08.129 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.129 INFO:teuthology.orchestra.run.smithi114.stdout: "req_mksnap_latency": { 2024-04-24T09:25:08.130 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:08.130 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:08.130 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:08.130 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.130 INFO:teuthology.orchestra.run.smithi114.stdout: "req_open_latency": { 2024-04-24T09:25:08.131 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:08.131 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:08.131 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:08.131 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.131 INFO:teuthology.orchestra.run.smithi114.stdout: "req_readdir_latency": { 2024-04-24T09:25:08.132 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:08.132 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:08.132 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:08.132 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.132 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rename_latency": { 2024-04-24T09:25:08.132 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:08.133 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:08.133 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:08.133 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.133 INFO:teuthology.orchestra.run.smithi114.stdout: "req_renamesnap_latency": { 2024-04-24T09:25:08.133 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:08.133 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:08.133 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:08.133 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.133 INFO:teuthology.orchestra.run.smithi114.stdout: "req_snapdiff_latency": { 2024-04-24T09:25:08.133 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:08.134 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:08.134 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:08.134 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.134 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rmdir_latency": { 2024-04-24T09:25:08.134 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:08.134 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:08.134 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:08.134 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.135 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rmsnap_latency": { 2024-04-24T09:25:08.135 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:08.135 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:08.135 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:08.135 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.135 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rmxattr_latency": { 2024-04-24T09:25:08.135 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:08.135 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:08.135 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:08.136 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.136 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setattr_latency": { 2024-04-24T09:25:08.136 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:08.136 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:08.136 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:08.136 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.136 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setdirlayout_latency": { 2024-04-24T09:25:08.136 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:08.136 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:08.136 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:08.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.137 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setfilelock_latency": { 2024-04-24T09:25:08.137 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:08.137 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:08.137 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:08.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.137 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setlayout_latency": { 2024-04-24T09:25:08.137 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:08.137 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:08.137 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:08.138 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.138 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setxattr_latency": { 2024-04-24T09:25:08.138 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:08.138 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:08.138 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:08.138 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.138 INFO:teuthology.orchestra.run.smithi114.stdout: "req_symlink_latency": { 2024-04-24T09:25:08.138 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:08.138 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:08.139 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:08.139 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.139 INFO:teuthology.orchestra.run.smithi114.stdout: "req_unlink_latency": { 2024-04-24T09:25:08.139 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:08.139 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:08.139 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:08.139 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.139 INFO:teuthology.orchestra.run.smithi114.stdout: "cap_revoke_eviction": 0, 2024-04-24T09:25:08.140 INFO:teuthology.orchestra.run.smithi114.stdout: "cap_acquisition_throttle": 0, 2024-04-24T09:25:08.140 INFO:teuthology.orchestra.run.smithi114.stdout: "req_getvxattr_latency": { 2024-04-24T09:25:08.140 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:08.140 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:08.140 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:08.140 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-04-24T09:25:08.140 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.140 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_sessions": { 2024-04-24T09:25:08.140 INFO:teuthology.orchestra.run.smithi114.stdout: "session_count": 1, 2024-04-24T09:25:08.141 INFO:teuthology.orchestra.run.smithi114.stdout: "session_add": 2, 2024-04-24T09:25:08.141 INFO:teuthology.orchestra.run.smithi114.stdout: "session_remove": 1, 2024-04-24T09:25:08.141 INFO:teuthology.orchestra.run.smithi114.stdout: "sessions_open": 1, 2024-04-24T09:25:08.141 INFO:teuthology.orchestra.run.smithi114.stdout: "sessions_stale": 0, 2024-04-24T09:25:08.141 INFO:teuthology.orchestra.run.smithi114.stdout: "total_load": 0, 2024-04-24T09:25:08.141 INFO:teuthology.orchestra.run.smithi114.stdout: "average_load": 0, 2024-04-24T09:25:08.141 INFO:teuthology.orchestra.run.smithi114.stdout: "avg_session_uptime": 1761, 2024-04-24T09:25:08.141 INFO:teuthology.orchestra.run.smithi114.stdout: "mdthresh_evicted": 0 2024-04-24T09:25:08.141 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.142 INFO:teuthology.orchestra.run.smithi114.stdout: "mempool": { 2024-04-24T09:25:08.142 INFO:teuthology.orchestra.run.smithi114.stdout: "bloom_filter_bytes": 0, 2024-04-24T09:25:08.142 INFO:teuthology.orchestra.run.smithi114.stdout: "bloom_filter_items": 0, 2024-04-24T09:25:08.142 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T09:25:08.142 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_alloc_items": 0, 2024-04-24T09:25:08.142 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T09:25:08.142 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_data_items": 0, 2024-04-24T09:25:08.142 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T09:25:08.143 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T09:25:08.143 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T09:25:08.143 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T09:25:08.143 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T09:25:08.143 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_other_items": 0, 2024-04-24T09:25:08.143 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T09:25:08.143 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T09:25:08.143 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_extent_bytes": 0, 2024-04-24T09:25:08.143 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_extent_items": 0, 2024-04-24T09:25:08.144 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_blob_bytes": 0, 2024-04-24T09:25:08.144 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_blob_items": 0, 2024-04-24T09:25:08.144 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T09:25:08.144 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T09:25:08.144 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T09:25:08.144 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T09:25:08.144 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T09:25:08.144 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_fsck_items": 0, 2024-04-24T09:25:08.144 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_txc_bytes": 0, 2024-04-24T09:25:08.145 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_txc_items": 0, 2024-04-24T09:25:08.145 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T09:25:08.145 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T09:25:08.145 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_bytes": 0, 2024-04-24T09:25:08.145 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_items": 0, 2024-04-24T09:25:08.145 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_bytes": 0, 2024-04-24T09:25:08.145 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_items": 0, 2024-04-24T09:25:08.145 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T09:25:08.146 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_reader_items": 0, 2024-04-24T09:25:08.146 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T09:25:08.146 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_writer_items": 0, 2024-04-24T09:25:08.146 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_anon_bytes": 182247, 2024-04-24T09:25:08.146 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_anon_items": 74, 2024-04-24T09:25:08.146 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_meta_bytes": 0, 2024-04-24T09:25:08.146 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_meta_items": 0, 2024-04-24T09:25:08.146 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_bytes": 0, 2024-04-24T09:25:08.147 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_items": 0, 2024-04-24T09:25:08.147 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_mapbl_bytes": 0, 2024-04-24T09:25:08.147 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_mapbl_items": 0, 2024-04-24T09:25:08.147 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_pglog_bytes": 0, 2024-04-24T09:25:08.147 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_pglog_items": 0, 2024-04-24T09:25:08.147 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_bytes": 7624, 2024-04-24T09:25:08.147 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_items": 160, 2024-04-24T09:25:08.147 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T09:25:08.147 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_mapping_items": 0, 2024-04-24T09:25:08.147 INFO:teuthology.orchestra.run.smithi114.stdout: "pgmap_bytes": 0, 2024-04-24T09:25:08.148 INFO:teuthology.orchestra.run.smithi114.stdout: "pgmap_items": 0, 2024-04-24T09:25:08.148 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_co_bytes": 145810, 2024-04-24T09:25:08.148 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_co_items": 1126, 2024-04-24T09:25:08.148 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_1_bytes": 0, 2024-04-24T09:25:08.148 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_1_items": 0, 2024-04-24T09:25:08.148 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_2_bytes": 0, 2024-04-24T09:25:08.148 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_2_items": 0 2024-04-24T09:25:08.148 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.148 INFO:teuthology.orchestra.run.smithi114.stdout: "objecter": { 2024-04-24T09:25:08.149 INFO:teuthology.orchestra.run.smithi114.stdout: "op_active": 0, 2024-04-24T09:25:08.149 INFO:teuthology.orchestra.run.smithi114.stdout: "op_laggy": 0, 2024-04-24T09:25:08.149 INFO:teuthology.orchestra.run.smithi114.stdout: "op_send": 928, 2024-04-24T09:25:08.149 INFO:teuthology.orchestra.run.smithi114.stdout: "op_send_bytes": 21, 2024-04-24T09:25:08.149 INFO:teuthology.orchestra.run.smithi114.stdout: "op_resend": 3, 2024-04-24T09:25:08.149 INFO:teuthology.orchestra.run.smithi114.stdout: "op_reply": 928, 2024-04-24T09:25:08.149 INFO:teuthology.orchestra.run.smithi114.stdout: "op_latency": { 2024-04-24T09:25:08.149 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 928, 2024-04-24T09:25:08.149 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.504989476, 2024-04-24T09:25:08.150 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000544169 2024-04-24T09:25:08.150 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.150 INFO:teuthology.orchestra.run.smithi114.stdout: "op_inflight": 0, 2024-04-24T09:25:08.150 INFO:teuthology.orchestra.run.smithi114.stdout: "oplen_avg": { 2024-04-24T09:25:08.150 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 928, 2024-04-24T09:25:08.150 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 929 2024-04-24T09:25:08.150 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.150 INFO:teuthology.orchestra.run.smithi114.stdout: "op": 928, 2024-04-24T09:25:08.150 INFO:teuthology.orchestra.run.smithi114.stdout: "op_r": 928, 2024-04-24T09:25:08.151 INFO:teuthology.orchestra.run.smithi114.stdout: "op_w": 0, 2024-04-24T09:25:08.151 INFO:teuthology.orchestra.run.smithi114.stdout: "op_rmw": 0, 2024-04-24T09:25:08.151 INFO:teuthology.orchestra.run.smithi114.stdout: "op_pg": 0, 2024-04-24T09:25:08.151 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_stat": 548, 2024-04-24T09:25:08.151 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_create": 0, 2024-04-24T09:25:08.151 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_read": 378, 2024-04-24T09:25:08.151 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_write": 0, 2024-04-24T09:25:08.151 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_writefull": 0, 2024-04-24T09:25:08.151 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_writesame": 0, 2024-04-24T09:25:08.152 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_append": 0, 2024-04-24T09:25:08.152 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_zero": 0, 2024-04-24T09:25:08.152 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_truncate": 0, 2024-04-24T09:25:08.152 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_delete": 0, 2024-04-24T09:25:08.152 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_mapext": 0, 2024-04-24T09:25:08.152 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_sparse_read": 0, 2024-04-24T09:25:08.152 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_clonerange": 0, 2024-04-24T09:25:08.152 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_getxattr": 1, 2024-04-24T09:25:08.152 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_setxattr": 0, 2024-04-24T09:25:08.152 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_cmpxattr": 0, 2024-04-24T09:25:08.153 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_rmxattr": 0, 2024-04-24T09:25:08.153 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_resetxattrs": 0, 2024-04-24T09:25:08.153 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_call": 0, 2024-04-24T09:25:08.153 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_watch": 0, 2024-04-24T09:25:08.153 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_notify": 0, 2024-04-24T09:25:08.153 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T09:25:08.153 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_pgls": 0, 2024-04-24T09:25:08.153 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_pgls_filter": 0, 2024-04-24T09:25:08.153 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_other": 0, 2024-04-24T09:25:08.154 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_active": 0, 2024-04-24T09:25:08.154 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_send": 0, 2024-04-24T09:25:08.154 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_resend": 0, 2024-04-24T09:25:08.154 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_ping": 0, 2024-04-24T09:25:08.154 INFO:teuthology.orchestra.run.smithi114.stdout: "poolop_active": 0, 2024-04-24T09:25:08.154 INFO:teuthology.orchestra.run.smithi114.stdout: "poolop_send": 0, 2024-04-24T09:25:08.155 INFO:teuthology.orchestra.run.smithi114.stdout: "poolop_resend": 0, 2024-04-24T09:25:08.155 INFO:teuthology.orchestra.run.smithi114.stdout: "poolstat_active": 0, 2024-04-24T09:25:08.155 INFO:teuthology.orchestra.run.smithi114.stdout: "poolstat_send": 0, 2024-04-24T09:25:08.155 INFO:teuthology.orchestra.run.smithi114.stdout: "poolstat_resend": 0, 2024-04-24T09:25:08.155 INFO:teuthology.orchestra.run.smithi114.stdout: "statfs_active": 0, 2024-04-24T09:25:08.155 INFO:teuthology.orchestra.run.smithi114.stdout: "statfs_send": 0, 2024-04-24T09:25:08.155 INFO:teuthology.orchestra.run.smithi114.stdout: "statfs_resend": 0, 2024-04-24T09:25:08.155 INFO:teuthology.orchestra.run.smithi114.stdout: "command_active": 0, 2024-04-24T09:25:08.155 INFO:teuthology.orchestra.run.smithi114.stdout: "command_send": 0, 2024-04-24T09:25:08.156 INFO:teuthology.orchestra.run.smithi114.stdout: "command_resend": 0, 2024-04-24T09:25:08.156 INFO:teuthology.orchestra.run.smithi114.stdout: "map_epoch": 88, 2024-04-24T09:25:08.156 INFO:teuthology.orchestra.run.smithi114.stdout: "map_full": 0, 2024-04-24T09:25:08.156 INFO:teuthology.orchestra.run.smithi114.stdout: "map_inc": 2, 2024-04-24T09:25:08.156 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_sessions": 4, 2024-04-24T09:25:08.156 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_session_open": 4, 2024-04-24T09:25:08.156 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_session_close": 0, 2024-04-24T09:25:08.157 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_laggy": 0, 2024-04-24T09:25:08.157 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_wr": 0, 2024-04-24T09:25:08.157 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_rd": 2, 2024-04-24T09:25:08.157 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_del": 0 2024-04-24T09:25:08.157 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.157 INFO:teuthology.orchestra.run.smithi114.stdout: "oft": { 2024-04-24T09:25:08.157 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_objs": 0, 2024-04-24T09:25:08.157 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_kv_pairs": 0, 2024-04-24T09:25:08.158 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_updates": 0, 2024-04-24T09:25:08.158 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_removes": 0 2024-04-24T09:25:08.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.158 INFO:teuthology.orchestra.run.smithi114.stdout: "osd-slow-ops": { 2024-04-24T09:25:08.158 INFO:teuthology.orchestra.run.smithi114.stdout: "slow_ops_count": 0 2024-04-24T09:25:08.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.158 INFO:teuthology.orchestra.run.smithi114.stdout: "purge_queue": { 2024-04-24T09:25:08.158 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing_ops": 0, 2024-04-24T09:25:08.159 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing_ops_high_water": 0, 2024-04-24T09:25:08.159 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing": 0, 2024-04-24T09:25:08.159 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing_high_water": 0, 2024-04-24T09:25:08.159 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executed_ops": 0, 2024-04-24T09:25:08.159 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executed": 0, 2024-04-24T09:25:08.160 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_item_in_journal": 0 2024-04-24T09:25:08.160 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.160 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T09:25:08.160 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-04-24T09:25:08.160 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 104857600, 2024-04-24T09:25:08.160 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-04-24T09:25:08.160 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 1053, 2024-04-24T09:25:08.161 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 1076625, 2024-04-24T09:25:08.161 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-04-24T09:25:08.161 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 1053, 2024-04-24T09:25:08.161 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-04-24T09:25:08.161 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-04-24T09:25:08.161 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 1053, 2024-04-24T09:25:08.161 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 1076625, 2024-04-24T09:25:08.162 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-04-24T09:25:08.162 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:08.162 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:08.162 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:08.162 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-04-24T09:25:08.162 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.163 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-objecter_bytes": { 2024-04-24T09:25:08.163 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-04-24T09:25:08.163 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 104857600, 2024-04-24T09:25:08.163 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-04-24T09:25:08.163 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 0, 2024-04-24T09:25:08.163 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 0, 2024-04-24T09:25:08.164 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-04-24T09:25:08.164 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 0, 2024-04-24T09:25:08.164 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 928, 2024-04-24T09:25:08.164 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 123419, 2024-04-24T09:25:08.164 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 928, 2024-04-24T09:25:08.164 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 123419, 2024-04-24T09:25:08.164 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-04-24T09:25:08.165 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:08.165 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:08.165 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:08.165 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-04-24T09:25:08.165 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.166 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-objecter_ops": { 2024-04-24T09:25:08.166 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-04-24T09:25:08.166 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 1024, 2024-04-24T09:25:08.166 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-04-24T09:25:08.166 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 0, 2024-04-24T09:25:08.167 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 0, 2024-04-24T09:25:08.167 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-04-24T09:25:08.167 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 0, 2024-04-24T09:25:08.167 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 928, 2024-04-24T09:25:08.167 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 928, 2024-04-24T09:25:08.167 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 928, 2024-04-24T09:25:08.167 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 928, 2024-04-24T09:25:08.168 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-04-24T09:25:08.168 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:08.168 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:08.168 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:08.168 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-04-24T09:25:08.168 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.168 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-write_buf_throttle": { 2024-04-24T09:25:08.168 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-04-24T09:25:08.168 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 3758096384, 2024-04-24T09:25:08.169 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-04-24T09:25:08.169 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 0, 2024-04-24T09:25:08.169 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 0, 2024-04-24T09:25:08.169 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-04-24T09:25:08.169 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 0, 2024-04-24T09:25:08.169 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-04-24T09:25:08.169 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-04-24T09:25:08.169 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 0, 2024-04-24T09:25:08.169 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 0, 2024-04-24T09:25:08.170 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-04-24T09:25:08.170 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:08.170 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:08.170 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:08.170 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-04-24T09:25:08.170 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:08.170 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-write_buf_throttle-0x55b67f26a6e0": { 2024-04-24T09:25:08.171 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-04-24T09:25:08.171 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 3758096384, 2024-04-24T09:25:08.171 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-04-24T09:25:08.171 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 0, 2024-04-24T09:25:08.171 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 0, 2024-04-24T09:25:08.171 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-04-24T09:25:08.171 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 0, 2024-04-24T09:25:08.171 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-04-24T09:25:08.171 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-04-24T09:25:08.171 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 0, 2024-04-24T09:25:08.172 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 0, 2024-04-24T09:25:08.172 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-04-24T09:25:08.172 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:08.172 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:08.172 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:08.172 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-04-24T09:25:08.172 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-04-24T09:25:08.172 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-04-24T09:25:08.883 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:08 smithi114 ceph-mon[32501]: pgmap v503: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 4.4 KiB/s wr, 11 op/s 2024-04-24T09:25:08.912 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-24T09:25:08.913 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-04-24T09:25:08.913 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-04-24T09:25:08.913 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2024-04-24T09:25:08.913 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-04-24T09:25:08.913 DEBUG:tasks.check_counter:Getting stats from k 2024-04-24T09:25:08.913 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-24T09:25:09.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:08 smithi090 ceph-mon[27825]: pgmap v503: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 4.4 KiB/s wr, 11 op/s 2024-04-24T09:25:09.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:08 smithi149 ceph-mon[32592]: pgmap v503: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 4.4 KiB/s wr, 11 op/s 2024-04-24T09:25:09.154 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T09:25:10.876 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:10 smithi114 ceph-mon[32501]: pgmap v504: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.5 KiB/s wr, 13 op/s 2024-04-24T09:25:11.001 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-04-24T09:25:11.001 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T09:25:11.001 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 354, 2024-04-24T09:25:11.002 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 356, 2024-04-24T09:25:11.002 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 118889, 2024-04-24T09:25:11.002 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 112573, 2024-04-24T09:25:11.002 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 6, 2024-04-24T09:25:11.002 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 2, 2024-04-24T09:25:11.002 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.043542232, 2024-04-24T09:25:11.002 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.012995100, 2024-04-24T09:25:11.003 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.011356435, 2024-04-24T09:25:11.003 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.013864317, 2024-04-24T09:25:11.003 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:25:11.003 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 354, 2024-04-24T09:25:11.003 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.021205337, 2024-04-24T09:25:11.003 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000059902 2024-04-24T09:25:11.003 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.004 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:25:11.004 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:11.004 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:11.004 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:11.004 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.004 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T09:25:11.005 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T09:25:11.005 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.005 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T09:25:11.005 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 560, 2024-04-24T09:25:11.005 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 641, 2024-04-24T09:25:11.005 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 146904, 2024-04-24T09:25:11.005 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 393907, 2024-04-24T09:25:11.005 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 5, 2024-04-24T09:25:11.006 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 4, 2024-04-24T09:25:11.006 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.044765476, 2024-04-24T09:25:11.006 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.018220975, 2024-04-24T09:25:11.006 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.009865043, 2024-04-24T09:25:11.006 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.012663347, 2024-04-24T09:25:11.006 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:25:11.006 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 640, 2024-04-24T09:25:11.007 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.028057114, 2024-04-24T09:25:11.007 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000043839 2024-04-24T09:25:11.007 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.007 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:25:11.007 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:11.007 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:11.007 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:11.007 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.008 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-04-24T09:25:11.008 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 217360 2024-04-24T09:25:11.008 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.008 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T09:25:11.008 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 129, 2024-04-24T09:25:11.008 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 120, 2024-04-24T09:25:11.008 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 772804, 2024-04-24T09:25:11.009 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 69237, 2024-04-24T09:25:11.009 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 4, 2024-04-24T09:25:11.009 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 3, 2024-04-24T09:25:11.009 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.022754595, 2024-04-24T09:25:11.009 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.006752699, 2024-04-24T09:25:11.009 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.018076658, 2024-04-24T09:25:11.009 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.002576598, 2024-04-24T09:25:11.009 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:25:11.010 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 119, 2024-04-24T09:25:11.010 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.007755245, 2024-04-24T09:25:11.010 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000065170 2024-04-24T09:25:11.010 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.010 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:25:11.010 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:11.010 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:11.011 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:11.011 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.011 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 771456, 2024-04-24T09:25:11.011 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 68272 2024-04-24T09:25:11.011 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.011 INFO:teuthology.orchestra.run.smithi114.stdout: "cct": { 2024-04-24T09:25:11.012 INFO:teuthology.orchestra.run.smithi114.stdout: "total_workers": 1, 2024-04-24T09:25:11.012 INFO:teuthology.orchestra.run.smithi114.stdout: "unhealthy_workers": 0 2024-04-24T09:25:11.012 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.012 INFO:teuthology.orchestra.run.smithi114.stdout: "finisher-MDSRank": { 2024-04-24T09:25:11.012 INFO:teuthology.orchestra.run.smithi114.stdout: "queue_len": 0, 2024-04-24T09:25:11.012 INFO:teuthology.orchestra.run.smithi114.stdout: "complete_latency": { 2024-04-24T09:25:11.012 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 1619, 2024-04-24T09:25:11.012 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.041599282, 2024-04-24T09:25:11.013 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000025694 2024-04-24T09:25:11.013 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-04-24T09:25:11.013 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.013 INFO:teuthology.orchestra.run.smithi114.stdout: "finisher-PurgeQueue": { 2024-04-24T09:25:11.013 INFO:teuthology.orchestra.run.smithi114.stdout: "queue_len": 0, 2024-04-24T09:25:11.013 INFO:teuthology.orchestra.run.smithi114.stdout: "complete_latency": { 2024-04-24T09:25:11.013 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:11.014 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:11.014 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:11.014 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-04-24T09:25:11.014 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.014 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": { 2024-04-24T09:25:11.014 INFO:teuthology.orchestra.run.smithi114.stdout: "request": 0, 2024-04-24T09:25:11.015 INFO:teuthology.orchestra.run.smithi114.stdout: "reply": 0, 2024-04-24T09:25:11.015 INFO:teuthology.orchestra.run.smithi114.stdout: "reply_latency": { 2024-04-24T09:25:11.015 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:11.015 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:11.015 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:11.015 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.015 INFO:teuthology.orchestra.run.smithi114.stdout: "slow_reply": 0, 2024-04-24T09:25:11.015 INFO:teuthology.orchestra.run.smithi114.stdout: "forward": 0, 2024-04-24T09:25:11.016 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_fetch_complete": 0, 2024-04-24T09:25:11.016 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_fetch_keys": 0, 2024-04-24T09:25:11.016 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_commit": 0, 2024-04-24T09:25:11.016 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_split": 0, 2024-04-24T09:25:11.016 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_merge": 0, 2024-04-24T09:25:11.016 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes": 0, 2024-04-24T09:25:11.016 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_top": 0, 2024-04-24T09:25:11.017 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_bottom": 0, 2024-04-24T09:25:11.017 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_pin_tail": 0, 2024-04-24T09:25:11.017 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_pinned": 0, 2024-04-24T09:25:11.017 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_expired": 0, 2024-04-24T09:25:11.017 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_with_caps": 0, 2024-04-24T09:25:11.017 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": 0, 2024-04-24T09:25:11.017 INFO:teuthology.orchestra.run.smithi114.stdout: "subtrees": 1, 2024-04-24T09:25:11.017 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse": 0, 2024-04-24T09:25:11.018 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_hit": 0, 2024-04-24T09:25:11.018 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_forward": 0, 2024-04-24T09:25:11.018 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_discover": 0, 2024-04-24T09:25:11.018 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_dir_fetch": 0, 2024-04-24T09:25:11.018 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_remote_ino": 0, 2024-04-24T09:25:11.018 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_lock": 0, 2024-04-24T09:25:11.018 INFO:teuthology.orchestra.run.smithi114.stdout: "load_cent": 0, 2024-04-24T09:25:11.018 INFO:teuthology.orchestra.run.smithi114.stdout: "q": 0, 2024-04-24T09:25:11.019 INFO:teuthology.orchestra.run.smithi114.stdout: "exported": 0, 2024-04-24T09:25:11.019 INFO:teuthology.orchestra.run.smithi114.stdout: "exported_inodes": 0, 2024-04-24T09:25:11.019 INFO:teuthology.orchestra.run.smithi114.stdout: "imported": 0, 2024-04-24T09:25:11.019 INFO:teuthology.orchestra.run.smithi114.stdout: "imported_inodes": 0, 2024-04-24T09:25:11.019 INFO:teuthology.orchestra.run.smithi114.stdout: "openino_dir_fetch": 0, 2024-04-24T09:25:11.019 INFO:teuthology.orchestra.run.smithi114.stdout: "openino_backtrace_fetch": 0, 2024-04-24T09:25:11.019 INFO:teuthology.orchestra.run.smithi114.stdout: "openino_peer_discover": 0, 2024-04-24T09:25:11.020 INFO:teuthology.orchestra.run.smithi114.stdout: "root_rfiles": 0, 2024-04-24T09:25:11.020 INFO:teuthology.orchestra.run.smithi114.stdout: "root_rbytes": 0, 2024-04-24T09:25:11.020 INFO:teuthology.orchestra.run.smithi114.stdout: "root_rsnaps": 0, 2024-04-24T09:25:11.020 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_backtrace_fetch": 0, 2024-04-24T09:25:11.020 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_set_tag": 0, 2024-04-24T09:25:11.020 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T09:25:11.020 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_inotable_repaired": 0, 2024-04-24T09:25:11.020 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_dir_inodes": 0, 2024-04-24T09:25:11.021 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T09:25:11.021 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_dirfrag_rstats": 0, 2024-04-24T09:25:11.021 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_file_inodes": 0, 2024-04-24T09:25:11.021 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_inode_file_caps": 0, 2024-04-24T09:25:11.021 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_revoke": 0, 2024-04-24T09:25:11.021 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_grant": 0, 2024-04-24T09:25:11.021 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T09:25:11.022 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T09:25:11.022 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T09:25:11.022 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_caps": 0, 2024-04-24T09:25:11.022 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_caps_dirty": 0, 2024-04-24T09:25:11.022 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_cap_release": 0, 2024-04-24T09:25:11.022 INFO:teuthology.orchestra.run.smithi114.stdout: "process_request_cap_release": 0 2024-04-24T09:25:11.022 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.023 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_cache": { 2024-04-24T09:25:11.023 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_update": 0, 2024-04-24T09:25:11.023 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_update_receipt": 0, 2024-04-24T09:25:11.023 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_try_discover": 0, 2024-04-24T09:25:11.023 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_send_discover": 0, 2024-04-24T09:25:11.023 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_handle_discover": 0, 2024-04-24T09:25:11.023 INFO:teuthology.orchestra.run.smithi114.stdout: "num_strays": 0, 2024-04-24T09:25:11.024 INFO:teuthology.orchestra.run.smithi114.stdout: "num_strays_delayed": 0, 2024-04-24T09:25:11.024 INFO:teuthology.orchestra.run.smithi114.stdout: "num_strays_enqueuing": 0, 2024-04-24T09:25:11.024 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_created": 0, 2024-04-24T09:25:11.024 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_enqueued": 0, 2024-04-24T09:25:11.024 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_reintegrated": 0, 2024-04-24T09:25:11.024 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_migrated": 0, 2024-04-24T09:25:11.024 INFO:teuthology.orchestra.run.smithi114.stdout: "num_recovering_processing": 0, 2024-04-24T09:25:11.025 INFO:teuthology.orchestra.run.smithi114.stdout: "num_recovering_enqueued": 0, 2024-04-24T09:25:11.025 INFO:teuthology.orchestra.run.smithi114.stdout: "num_recovering_prioritized": 0, 2024-04-24T09:25:11.025 INFO:teuthology.orchestra.run.smithi114.stdout: "recovery_started": 0, 2024-04-24T09:25:11.025 INFO:teuthology.orchestra.run.smithi114.stdout: "recovery_completed": 0, 2024-04-24T09:25:11.025 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_quiesce_path": 0, 2024-04-24T09:25:11.025 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_quiesce_inode": 0, 2024-04-24T09:25:11.025 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T09:25:11.026 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_exportdir": 0, 2024-04-24T09:25:11.026 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_flush": 0, 2024-04-24T09:25:11.026 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_fragmentdir": 0, 2024-04-24T09:25:11.026 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_fragstats": 0, 2024-04-24T09:25:11.026 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_inodestats": 0 2024-04-24T09:25:11.026 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.027 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_log": { 2024-04-24T09:25:11.027 INFO:teuthology.orchestra.run.smithi114.stdout: "evlrg": 0, 2024-04-24T09:25:11.027 INFO:teuthology.orchestra.run.smithi114.stdout: "evadd": 0, 2024-04-24T09:25:11.027 INFO:teuthology.orchestra.run.smithi114.stdout: "evex": 0, 2024-04-24T09:25:11.027 INFO:teuthology.orchestra.run.smithi114.stdout: "evtrm": 0, 2024-04-24T09:25:11.027 INFO:teuthology.orchestra.run.smithi114.stdout: "ev": 5, 2024-04-24T09:25:11.027 INFO:teuthology.orchestra.run.smithi114.stdout: "evexg": 0, 2024-04-24T09:25:11.027 INFO:teuthology.orchestra.run.smithi114.stdout: "evexd": 0, 2024-04-24T09:25:11.028 INFO:teuthology.orchestra.run.smithi114.stdout: "segadd": 0, 2024-04-24T09:25:11.028 INFO:teuthology.orchestra.run.smithi114.stdout: "segex": 0, 2024-04-24T09:25:11.028 INFO:teuthology.orchestra.run.smithi114.stdout: "segtrm": 0, 2024-04-24T09:25:11.028 INFO:teuthology.orchestra.run.smithi114.stdout: "seg": 2, 2024-04-24T09:25:11.028 INFO:teuthology.orchestra.run.smithi114.stdout: "segmjr": 2, 2024-04-24T09:25:11.028 INFO:teuthology.orchestra.run.smithi114.stdout: "segexg": 0, 2024-04-24T09:25:11.029 INFO:teuthology.orchestra.run.smithi114.stdout: "segexd": 0, 2024-04-24T09:25:11.029 INFO:teuthology.orchestra.run.smithi114.stdout: "expos": 4194304, 2024-04-24T09:25:11.029 INFO:teuthology.orchestra.run.smithi114.stdout: "wrpos": 4195734, 2024-04-24T09:25:11.029 INFO:teuthology.orchestra.run.smithi114.stdout: "rdpos": 4195593, 2024-04-24T09:25:11.029 INFO:teuthology.orchestra.run.smithi114.stdout: "jlat": { 2024-04-24T09:25:11.029 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:11.029 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:11.029 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:11.030 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.030 INFO:teuthology.orchestra.run.smithi114.stdout: "replayed": 5 2024-04-24T09:25:11.030 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.030 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_mem": { 2024-04-24T09:25:11.030 INFO:teuthology.orchestra.run.smithi114.stdout: "ino": 3, 2024-04-24T09:25:11.030 INFO:teuthology.orchestra.run.smithi114.stdout: "ino+": 3, 2024-04-24T09:25:11.030 INFO:teuthology.orchestra.run.smithi114.stdout: "ino-": 0, 2024-04-24T09:25:11.031 INFO:teuthology.orchestra.run.smithi114.stdout: "dir": 1, 2024-04-24T09:25:11.031 INFO:teuthology.orchestra.run.smithi114.stdout: "dir+": 1, 2024-04-24T09:25:11.031 INFO:teuthology.orchestra.run.smithi114.stdout: "dir-": 0, 2024-04-24T09:25:11.031 INFO:teuthology.orchestra.run.smithi114.stdout: "dn": 0, 2024-04-24T09:25:11.031 INFO:teuthology.orchestra.run.smithi114.stdout: "dn+": 0, 2024-04-24T09:25:11.031 INFO:teuthology.orchestra.run.smithi114.stdout: "dn-": 0, 2024-04-24T09:25:11.031 INFO:teuthology.orchestra.run.smithi114.stdout: "cap": 0, 2024-04-24T09:25:11.032 INFO:teuthology.orchestra.run.smithi114.stdout: "cap+": 0, 2024-04-24T09:25:11.032 INFO:teuthology.orchestra.run.smithi114.stdout: "cap-": 0, 2024-04-24T09:25:11.032 INFO:teuthology.orchestra.run.smithi114.stdout: "rss": 49196, 2024-04-24T09:25:11.032 INFO:teuthology.orchestra.run.smithi114.stdout: "heap": 215416 2024-04-24T09:25:11.032 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.032 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_server": { 2024-04-24T09:25:11.032 INFO:teuthology.orchestra.run.smithi114.stdout: "dispatch_client_request": 0, 2024-04-24T09:25:11.032 INFO:teuthology.orchestra.run.smithi114.stdout: "dispatch_server_request": 0, 2024-04-24T09:25:11.033 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_request": 0, 2024-04-24T09:25:11.033 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_session": 0, 2024-04-24T09:25:11.033 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_peer_request": 0, 2024-04-24T09:25:11.033 INFO:teuthology.orchestra.run.smithi114.stdout: "req_create_latency": { 2024-04-24T09:25:11.033 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:11.033 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:11.033 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:11.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.034 INFO:teuthology.orchestra.run.smithi114.stdout: "req_getattr_latency": { 2024-04-24T09:25:11.034 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:11.034 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:11.034 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:11.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.034 INFO:teuthology.orchestra.run.smithi114.stdout: "req_getfilelock_latency": { 2024-04-24T09:25:11.035 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:11.035 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:11.035 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:11.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.035 INFO:teuthology.orchestra.run.smithi114.stdout: "req_link_latency": { 2024-04-24T09:25:11.035 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:11.035 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:11.035 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:11.036 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.036 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookup_latency": { 2024-04-24T09:25:11.036 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:11.036 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:11.036 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:11.036 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.036 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookuphash_latency": { 2024-04-24T09:25:11.037 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:11.037 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:11.037 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:11.037 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.037 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupino_latency": { 2024-04-24T09:25:11.037 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:11.037 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:11.037 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:11.038 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.038 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupname_latency": { 2024-04-24T09:25:11.038 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:11.038 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:11.038 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:11.038 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.038 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupparent_latency": { 2024-04-24T09:25:11.039 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:11.039 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:11.039 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:11.039 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.039 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupsnap_latency": { 2024-04-24T09:25:11.039 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:11.039 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:11.040 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:11.040 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.040 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lssnap_latency": { 2024-04-24T09:25:11.040 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:11.040 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:11.040 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:11.040 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.040 INFO:teuthology.orchestra.run.smithi114.stdout: "req_mkdir_latency": { 2024-04-24T09:25:11.041 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:11.041 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:11.042 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:11.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.042 INFO:teuthology.orchestra.run.smithi114.stdout: "req_mknod_latency": { 2024-04-24T09:25:11.042 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:11.042 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:11.042 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:11.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.042 INFO:teuthology.orchestra.run.smithi114.stdout: "req_mksnap_latency": { 2024-04-24T09:25:11.043 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:11.043 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:11.043 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:11.043 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.043 INFO:teuthology.orchestra.run.smithi114.stdout: "req_open_latency": { 2024-04-24T09:25:11.043 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:11.043 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:11.043 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:11.044 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.044 INFO:teuthology.orchestra.run.smithi114.stdout: "req_readdir_latency": { 2024-04-24T09:25:11.044 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:11.044 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:11.044 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:11.044 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.044 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rename_latency": { 2024-04-24T09:25:11.045 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:11.045 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:11.045 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:11.045 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.045 INFO:teuthology.orchestra.run.smithi114.stdout: "req_renamesnap_latency": { 2024-04-24T09:25:11.045 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:11.045 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:11.045 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:11.045 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.046 INFO:teuthology.orchestra.run.smithi114.stdout: "req_snapdiff_latency": { 2024-04-24T09:25:11.046 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:11.046 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:11.046 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:11.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.046 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rmdir_latency": { 2024-04-24T09:25:11.046 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:11.046 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:11.047 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:11.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.047 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rmsnap_latency": { 2024-04-24T09:25:11.047 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:11.047 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:11.047 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:11.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.047 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rmxattr_latency": { 2024-04-24T09:25:11.048 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:11.048 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:11.048 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:11.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.048 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setattr_latency": { 2024-04-24T09:25:11.048 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:11.048 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:11.048 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:11.049 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.049 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setdirlayout_latency": { 2024-04-24T09:25:11.049 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:11.049 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:11.049 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:11.049 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.049 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setfilelock_latency": { 2024-04-24T09:25:11.049 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:11.050 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:11.050 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:11.050 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.050 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setlayout_latency": { 2024-04-24T09:25:11.050 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:11.050 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:11.050 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:11.051 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.051 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setxattr_latency": { 2024-04-24T09:25:11.051 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:11.051 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:11.051 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:11.051 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.051 INFO:teuthology.orchestra.run.smithi114.stdout: "req_symlink_latency": { 2024-04-24T09:25:11.052 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:11.052 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:11.052 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:11.052 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.052 INFO:teuthology.orchestra.run.smithi114.stdout: "req_unlink_latency": { 2024-04-24T09:25:11.052 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:11.052 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:11.053 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:11.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.053 INFO:teuthology.orchestra.run.smithi114.stdout: "cap_revoke_eviction": 0, 2024-04-24T09:25:11.053 INFO:teuthology.orchestra.run.smithi114.stdout: "cap_acquisition_throttle": 0, 2024-04-24T09:25:11.053 INFO:teuthology.orchestra.run.smithi114.stdout: "req_getvxattr_latency": { 2024-04-24T09:25:11.053 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:11.053 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:11.053 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:11.054 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-04-24T09:25:11.054 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.054 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_sessions": { 2024-04-24T09:25:11.054 INFO:teuthology.orchestra.run.smithi114.stdout: "session_count": 1, 2024-04-24T09:25:11.054 INFO:teuthology.orchestra.run.smithi114.stdout: "session_add": 2, 2024-04-24T09:25:11.054 INFO:teuthology.orchestra.run.smithi114.stdout: "session_remove": 1, 2024-04-24T09:25:11.054 INFO:teuthology.orchestra.run.smithi114.stdout: "sessions_open": 2, 2024-04-24T09:25:11.055 INFO:teuthology.orchestra.run.smithi114.stdout: "sessions_stale": 0, 2024-04-24T09:25:11.055 INFO:teuthology.orchestra.run.smithi114.stdout: "total_load": 0, 2024-04-24T09:25:11.055 INFO:teuthology.orchestra.run.smithi114.stdout: "average_load": 0, 2024-04-24T09:25:11.055 INFO:teuthology.orchestra.run.smithi114.stdout: "avg_session_uptime": 3336, 2024-04-24T09:25:11.055 INFO:teuthology.orchestra.run.smithi114.stdout: "mdthresh_evicted": 0 2024-04-24T09:25:11.055 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.055 INFO:teuthology.orchestra.run.smithi114.stdout: "mempool": { 2024-04-24T09:25:11.055 INFO:teuthology.orchestra.run.smithi114.stdout: "bloom_filter_bytes": 0, 2024-04-24T09:25:11.056 INFO:teuthology.orchestra.run.smithi114.stdout: "bloom_filter_items": 0, 2024-04-24T09:25:11.056 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T09:25:11.056 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_alloc_items": 0, 2024-04-24T09:25:11.056 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T09:25:11.056 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_data_items": 0, 2024-04-24T09:25:11.056 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T09:25:11.056 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T09:25:11.057 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T09:25:11.057 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T09:25:11.057 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T09:25:11.057 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_other_items": 0, 2024-04-24T09:25:11.057 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T09:25:11.057 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T09:25:11.057 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_extent_bytes": 0, 2024-04-24T09:25:11.057 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_extent_items": 0, 2024-04-24T09:25:11.058 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_blob_bytes": 0, 2024-04-24T09:25:11.058 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_blob_items": 0, 2024-04-24T09:25:11.058 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T09:25:11.058 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T09:25:11.058 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T09:25:11.058 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T09:25:11.058 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T09:25:11.058 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_fsck_items": 0, 2024-04-24T09:25:11.059 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_txc_bytes": 0, 2024-04-24T09:25:11.059 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_txc_items": 0, 2024-04-24T09:25:11.059 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T09:25:11.059 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T09:25:11.059 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_bytes": 0, 2024-04-24T09:25:11.059 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_items": 0, 2024-04-24T09:25:11.059 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_bytes": 0, 2024-04-24T09:25:11.060 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_items": 0, 2024-04-24T09:25:11.060 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T09:25:11.060 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_reader_items": 0, 2024-04-24T09:25:11.060 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T09:25:11.060 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_writer_items": 0, 2024-04-24T09:25:11.060 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_anon_bytes": 197275, 2024-04-24T09:25:11.060 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_anon_items": 52, 2024-04-24T09:25:11.060 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_meta_bytes": 0, 2024-04-24T09:25:11.061 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_meta_items": 0, 2024-04-24T09:25:11.061 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_bytes": 0, 2024-04-24T09:25:11.061 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_items": 0, 2024-04-24T09:25:11.061 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_mapbl_bytes": 0, 2024-04-24T09:25:11.061 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_mapbl_items": 0, 2024-04-24T09:25:11.061 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_pglog_bytes": 0, 2024-04-24T09:25:11.061 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_pglog_items": 0, 2024-04-24T09:25:11.062 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_bytes": 7624, 2024-04-24T09:25:11.062 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_items": 160, 2024-04-24T09:25:11.062 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T09:25:11.062 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_mapping_items": 0, 2024-04-24T09:25:11.062 INFO:teuthology.orchestra.run.smithi114.stdout: "pgmap_bytes": 0, 2024-04-24T09:25:11.062 INFO:teuthology.orchestra.run.smithi114.stdout: "pgmap_items": 0, 2024-04-24T09:25:11.062 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_co_bytes": 8512, 2024-04-24T09:25:11.062 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_co_items": 13, 2024-04-24T09:25:11.063 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_1_bytes": 0, 2024-04-24T09:25:11.063 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_1_items": 0, 2024-04-24T09:25:11.063 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_2_bytes": 0, 2024-04-24T09:25:11.063 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_2_items": 0 2024-04-24T09:25:11.063 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.063 INFO:teuthology.orchestra.run.smithi114.stdout: "objecter": { 2024-04-24T09:25:11.063 INFO:teuthology.orchestra.run.smithi114.stdout: "op_active": 0, 2024-04-24T09:25:11.063 INFO:teuthology.orchestra.run.smithi114.stdout: "op_laggy": 0, 2024-04-24T09:25:11.064 INFO:teuthology.orchestra.run.smithi114.stdout: "op_send": 912, 2024-04-24T09:25:11.064 INFO:teuthology.orchestra.run.smithi114.stdout: "op_send_bytes": 21, 2024-04-24T09:25:11.064 INFO:teuthology.orchestra.run.smithi114.stdout: "op_resend": 3, 2024-04-24T09:25:11.064 INFO:teuthology.orchestra.run.smithi114.stdout: "op_reply": 912, 2024-04-24T09:25:11.064 INFO:teuthology.orchestra.run.smithi114.stdout: "op_latency": { 2024-04-24T09:25:11.064 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 912, 2024-04-24T09:25:11.064 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.735984672, 2024-04-24T09:25:11.064 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000807000 2024-04-24T09:25:11.065 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.065 INFO:teuthology.orchestra.run.smithi114.stdout: "op_inflight": 0, 2024-04-24T09:25:11.065 INFO:teuthology.orchestra.run.smithi114.stdout: "oplen_avg": { 2024-04-24T09:25:11.065 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 912, 2024-04-24T09:25:11.065 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 913 2024-04-24T09:25:11.065 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.065 INFO:teuthology.orchestra.run.smithi114.stdout: "op": 912, 2024-04-24T09:25:11.066 INFO:teuthology.orchestra.run.smithi114.stdout: "op_r": 912, 2024-04-24T09:25:11.066 INFO:teuthology.orchestra.run.smithi114.stdout: "op_w": 0, 2024-04-24T09:25:11.066 INFO:teuthology.orchestra.run.smithi114.stdout: "op_rmw": 0, 2024-04-24T09:25:11.066 INFO:teuthology.orchestra.run.smithi114.stdout: "op_pg": 0, 2024-04-24T09:25:11.066 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_stat": 551, 2024-04-24T09:25:11.066 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_create": 0, 2024-04-24T09:25:11.066 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_read": 359, 2024-04-24T09:25:11.067 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_write": 0, 2024-04-24T09:25:11.067 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_writefull": 0, 2024-04-24T09:25:11.067 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_writesame": 0, 2024-04-24T09:25:11.067 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_append": 0, 2024-04-24T09:25:11.067 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_zero": 0, 2024-04-24T09:25:11.067 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_truncate": 0, 2024-04-24T09:25:11.067 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_delete": 0, 2024-04-24T09:25:11.067 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_mapext": 0, 2024-04-24T09:25:11.068 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_sparse_read": 0, 2024-04-24T09:25:11.068 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_clonerange": 0, 2024-04-24T09:25:11.068 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_getxattr": 1, 2024-04-24T09:25:11.068 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_setxattr": 0, 2024-04-24T09:25:11.068 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_cmpxattr": 0, 2024-04-24T09:25:11.068 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_rmxattr": 0, 2024-04-24T09:25:11.068 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_resetxattrs": 0, 2024-04-24T09:25:11.068 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_call": 0, 2024-04-24T09:25:11.069 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_watch": 0, 2024-04-24T09:25:11.069 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_notify": 0, 2024-04-24T09:25:11.069 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T09:25:11.069 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_pgls": 0, 2024-04-24T09:25:11.069 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_pgls_filter": 0, 2024-04-24T09:25:11.069 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_other": 0, 2024-04-24T09:25:11.069 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_active": 0, 2024-04-24T09:25:11.069 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_send": 0, 2024-04-24T09:25:11.070 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_resend": 0, 2024-04-24T09:25:11.070 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_ping": 0, 2024-04-24T09:25:11.070 INFO:teuthology.orchestra.run.smithi114.stdout: "poolop_active": 0, 2024-04-24T09:25:11.070 INFO:teuthology.orchestra.run.smithi114.stdout: "poolop_send": 0, 2024-04-24T09:25:11.070 INFO:teuthology.orchestra.run.smithi114.stdout: "poolop_resend": 0, 2024-04-24T09:25:11.070 INFO:teuthology.orchestra.run.smithi114.stdout: "poolstat_active": 0, 2024-04-24T09:25:11.071 INFO:teuthology.orchestra.run.smithi114.stdout: "poolstat_send": 0, 2024-04-24T09:25:11.071 INFO:teuthology.orchestra.run.smithi114.stdout: "poolstat_resend": 0, 2024-04-24T09:25:11.071 INFO:teuthology.orchestra.run.smithi114.stdout: "statfs_active": 0, 2024-04-24T09:25:11.071 INFO:teuthology.orchestra.run.smithi114.stdout: "statfs_send": 0, 2024-04-24T09:25:11.071 INFO:teuthology.orchestra.run.smithi114.stdout: "statfs_resend": 0, 2024-04-24T09:25:11.071 INFO:teuthology.orchestra.run.smithi114.stdout: "command_active": 0, 2024-04-24T09:25:11.071 INFO:teuthology.orchestra.run.smithi114.stdout: "command_send": 0, 2024-04-24T09:25:11.072 INFO:teuthology.orchestra.run.smithi114.stdout: "command_resend": 0, 2024-04-24T09:25:11.072 INFO:teuthology.orchestra.run.smithi114.stdout: "map_epoch": 88, 2024-04-24T09:25:11.072 INFO:teuthology.orchestra.run.smithi114.stdout: "map_full": 0, 2024-04-24T09:25:11.072 INFO:teuthology.orchestra.run.smithi114.stdout: "map_inc": 2, 2024-04-24T09:25:11.072 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_sessions": 6, 2024-04-24T09:25:11.072 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_session_open": 6, 2024-04-24T09:25:11.072 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_session_close": 0, 2024-04-24T09:25:11.072 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_laggy": 0, 2024-04-24T09:25:11.073 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_wr": 0, 2024-04-24T09:25:11.073 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_rd": 2, 2024-04-24T09:25:11.073 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_del": 0 2024-04-24T09:25:11.073 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.073 INFO:teuthology.orchestra.run.smithi114.stdout: "oft": { 2024-04-24T09:25:11.073 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_objs": 0, 2024-04-24T09:25:11.074 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_kv_pairs": 0, 2024-04-24T09:25:11.074 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_updates": 0, 2024-04-24T09:25:11.074 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_removes": 0 2024-04-24T09:25:11.074 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.074 INFO:teuthology.orchestra.run.smithi114.stdout: "osd-slow-ops": { 2024-04-24T09:25:11.074 INFO:teuthology.orchestra.run.smithi114.stdout: "slow_ops_count": 0 2024-04-24T09:25:11.074 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.074 INFO:teuthology.orchestra.run.smithi114.stdout: "purge_queue": { 2024-04-24T09:25:11.075 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing_ops": 0, 2024-04-24T09:25:11.075 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing_ops_high_water": 0, 2024-04-24T09:25:11.075 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing": 0, 2024-04-24T09:25:11.075 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing_high_water": 0, 2024-04-24T09:25:11.075 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executed_ops": 0, 2024-04-24T09:25:11.075 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executed": 0, 2024-04-24T09:25:11.075 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_item_in_journal": 0 2024-04-24T09:25:11.076 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.076 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T09:25:11.076 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-04-24T09:25:11.076 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 104857600, 2024-04-24T09:25:11.076 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-04-24T09:25:11.076 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 1039, 2024-04-24T09:25:11.076 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 935577, 2024-04-24T09:25:11.076 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-04-24T09:25:11.077 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 1039, 2024-04-24T09:25:11.077 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-04-24T09:25:11.077 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-04-24T09:25:11.077 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 1039, 2024-04-24T09:25:11.077 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 935577, 2024-04-24T09:25:11.077 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-04-24T09:25:11.077 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:11.077 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:11.078 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:11.078 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-04-24T09:25:11.078 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.078 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-objecter_bytes": { 2024-04-24T09:25:11.078 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-04-24T09:25:11.078 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 104857600, 2024-04-24T09:25:11.078 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-04-24T09:25:11.078 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 0, 2024-04-24T09:25:11.079 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 0, 2024-04-24T09:25:11.079 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-04-24T09:25:11.079 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 0, 2024-04-24T09:25:11.079 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 912, 2024-04-24T09:25:11.079 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 1435, 2024-04-24T09:25:11.079 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 912, 2024-04-24T09:25:11.079 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 1435, 2024-04-24T09:25:11.079 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-04-24T09:25:11.080 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:11.080 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:11.080 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:11.080 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-04-24T09:25:11.080 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.080 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-objecter_ops": { 2024-04-24T09:25:11.080 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-04-24T09:25:11.080 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 1024, 2024-04-24T09:25:11.081 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-04-24T09:25:11.081 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 0, 2024-04-24T09:25:11.081 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 0, 2024-04-24T09:25:11.081 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-04-24T09:25:11.081 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 0, 2024-04-24T09:25:11.081 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 912, 2024-04-24T09:25:11.081 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 912, 2024-04-24T09:25:11.082 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 912, 2024-04-24T09:25:11.082 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 912, 2024-04-24T09:25:11.082 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-04-24T09:25:11.082 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:11.082 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:11.082 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:11.082 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-04-24T09:25:11.083 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.083 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-write_buf_throttle": { 2024-04-24T09:25:11.083 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-04-24T09:25:11.083 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 3758096384, 2024-04-24T09:25:11.083 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-04-24T09:25:11.083 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 0, 2024-04-24T09:25:11.083 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 0, 2024-04-24T09:25:11.083 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-04-24T09:25:11.084 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 0, 2024-04-24T09:25:11.084 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-04-24T09:25:11.084 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-04-24T09:25:11.084 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 0, 2024-04-24T09:25:11.084 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 0, 2024-04-24T09:25:11.084 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-04-24T09:25:11.084 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:11.084 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:11.085 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:11.085 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-04-24T09:25:11.085 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-04-24T09:25:11.085 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-write_buf_throttle-0x55aabcff46e0": { 2024-04-24T09:25:11.085 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-04-24T09:25:11.085 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 3758096384, 2024-04-24T09:25:11.085 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-04-24T09:25:11.086 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 0, 2024-04-24T09:25:11.086 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 0, 2024-04-24T09:25:11.086 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-04-24T09:25:11.086 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 0, 2024-04-24T09:25:11.086 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-04-24T09:25:11.086 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-04-24T09:25:11.086 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 0, 2024-04-24T09:25:11.086 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 0, 2024-04-24T09:25:11.087 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-04-24T09:25:11.087 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-04-24T09:25:11.087 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-04-24T09:25:11.087 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-04-24T09:25:11.087 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-04-24T09:25:11.087 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-04-24T09:25:11.087 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-04-24T09:25:11.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:10 smithi090 ceph-mon[27825]: pgmap v504: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.5 KiB/s wr, 13 op/s 2024-04-24T09:25:11.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:10 smithi149 ceph-mon[32592]: pgmap v504: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.5 KiB/s wr, 13 op/s 2024-04-24T09:25:11.817 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-04-24T09:25:11.817 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-04-24T09:25:11.817 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-04-24T09:25:11.817 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-04-24T09:25:11.817 INFO:tasks.check_counter:Daemon mds.k mds.dir_split=0 2024-04-24T09:25:11.818 DEBUG:tasks.check_counter:Getting stats from c 2024-04-24T09:25:11.818 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-24T09:25:12.063 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T09:25:12.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:12 smithi149 ceph-mon[32592]: pgmap v505: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-04-24T09:25:13.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:12 smithi090 ceph-mon[27825]: pgmap v505: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-04-24T09:25:13.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:12 smithi114 ceph-mon[32501]: pgmap v505: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-04-24T09:25:14.073 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-04-24T09:25:14.074 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T09:25:14.074 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 216, 2024-04-24T09:25:14.074 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 253, 2024-04-24T09:25:14.074 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 87043, 2024-04-24T09:25:14.074 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 185058, 2024-04-24T09:25:14.074 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 15, 2024-04-24T09:25:14.074 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 8, 2024-04-24T09:25:14.074 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.027229741, 2024-04-24T09:25:14.075 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.008238588, 2024-04-24T09:25:14.075 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.021030726, 2024-04-24T09:25:14.075 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.000694662, 2024-04-24T09:25:14.075 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:25:14.075 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 248, 2024-04-24T09:25:14.075 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.033989631, 2024-04-24T09:25:14.075 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000137054 2024-04-24T09:25:14.075 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.075 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:25:14.076 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 373, 2024-04-24T09:25:14.076 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000083343, 2024-04-24T09:25:14.076 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000223 2024-04-24T09:25:14.076 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.076 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 63904, 2024-04-24T09:25:14.076 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 108688 2024-04-24T09:25:14.076 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.076 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T09:25:14.076 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 207, 2024-04-24T09:25:14.077 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 194, 2024-04-24T09:25:14.077 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 822145, 2024-04-24T09:25:14.077 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 176654, 2024-04-24T09:25:14.077 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 7, 2024-04-24T09:25:14.077 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 6, 2024-04-24T09:25:14.077 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.030312929, 2024-04-24T09:25:14.077 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.008305908, 2024-04-24T09:25:14.077 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.024054522, 2024-04-24T09:25:14.077 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.004225226, 2024-04-24T09:25:14.078 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:25:14.078 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 193, 2024-04-24T09:25:14.078 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.043849616, 2024-04-24T09:25:14.078 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000227200 2024-04-24T09:25:14.078 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.078 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:25:14.079 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 10, 2024-04-24T09:25:14.079 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000001468, 2024-04-24T09:25:14.079 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000146 2024-04-24T09:25:14.080 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.080 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 796464, 2024-04-24T09:25:14.080 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 71152 2024-04-24T09:25:14.080 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.081 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T09:25:14.081 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 803, 2024-04-24T09:25:14.081 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 649, 2024-04-24T09:25:14.081 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 315031, 2024-04-24T09:25:14.081 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 488562, 2024-04-24T09:25:14.081 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 9, 2024-04-24T09:25:14.081 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 5, 2024-04-24T09:25:14.081 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.064762049, 2024-04-24T09:25:14.082 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.026582478, 2024-04-24T09:25:14.082 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.044874540, 2024-04-24T09:25:14.082 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.002212849, 2024-04-24T09:25:14.082 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:25:14.082 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 651, 2024-04-24T09:25:14.082 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.031514351, 2024-04-24T09:25:14.082 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000048409 2024-04-24T09:25:14.082 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.082 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:25:14.083 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1326, 2024-04-24T09:25:14.083 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000244061, 2024-04-24T09:25:14.083 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000184 2024-04-24T09:25:14.083 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.083 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-24T09:25:14.083 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 296816 2024-04-24T09:25:14.083 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.083 INFO:teuthology.orchestra.run.smithi149.stdout: "cct": { 2024-04-24T09:25:14.083 INFO:teuthology.orchestra.run.smithi149.stdout: "total_workers": 1, 2024-04-24T09:25:14.083 INFO:teuthology.orchestra.run.smithi149.stdout: "unhealthy_workers": 0 2024-04-24T09:25:14.084 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.084 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-MDSRank": { 2024-04-24T09:25:14.084 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-04-24T09:25:14.084 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-04-24T09:25:14.084 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 90, 2024-04-24T09:25:14.084 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.014024783, 2024-04-24T09:25:14.084 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000155830 2024-04-24T09:25:14.084 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-04-24T09:25:14.084 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.085 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-PurgeQueue": { 2024-04-24T09:25:14.085 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-04-24T09:25:14.085 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-04-24T09:25:14.085 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 20, 2024-04-24T09:25:14.085 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000706558, 2024-04-24T09:25:14.085 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000035327 2024-04-24T09:25:14.085 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-04-24T09:25:14.085 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.085 INFO:teuthology.orchestra.run.smithi149.stdout: "mds": { 2024-04-24T09:25:14.085 INFO:teuthology.orchestra.run.smithi149.stdout: "request": 147, 2024-04-24T09:25:14.086 INFO:teuthology.orchestra.run.smithi149.stdout: "reply": 83, 2024-04-24T09:25:14.086 INFO:teuthology.orchestra.run.smithi149.stdout: "reply_latency": { 2024-04-24T09:25:14.086 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 83, 2024-04-24T09:25:14.086 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.050546405, 2024-04-24T09:25:14.086 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000608992 2024-04-24T09:25:14.086 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.086 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_reply": 0, 2024-04-24T09:25:14.087 INFO:teuthology.orchestra.run.smithi149.stdout: "forward": 64, 2024-04-24T09:25:14.087 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_complete": 0, 2024-04-24T09:25:14.087 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_keys": 0, 2024-04-24T09:25:14.087 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_commit": 11, 2024-04-24T09:25:14.087 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_split": 0, 2024-04-24T09:25:14.087 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_merge": 0, 2024-04-24T09:25:14.087 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes": 76, 2024-04-24T09:25:14.088 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_top": 14, 2024-04-24T09:25:14.088 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_bottom": 62, 2024-04-24T09:25:14.088 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pin_tail": 0, 2024-04-24T09:25:14.088 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pinned": 35, 2024-04-24T09:25:14.088 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_expired": 8, 2024-04-24T09:25:14.088 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_with_caps": 4, 2024-04-24T09:25:14.088 INFO:teuthology.orchestra.run.smithi149.stdout: "caps": 4, 2024-04-24T09:25:14.088 INFO:teuthology.orchestra.run.smithi149.stdout: "subtrees": 4, 2024-04-24T09:25:14.088 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse": 265, 2024-04-24T09:25:14.089 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_hit": 91, 2024-04-24T09:25:14.089 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_forward": 64, 2024-04-24T09:25:14.089 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_discover": 19, 2024-04-24T09:25:14.089 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_dir_fetch": 0, 2024-04-24T09:25:14.089 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_remote_ino": 0, 2024-04-24T09:25:14.089 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_lock": 0, 2024-04-24T09:25:14.089 INFO:teuthology.orchestra.run.smithi149.stdout: "load_cent": 0, 2024-04-24T09:25:14.089 INFO:teuthology.orchestra.run.smithi149.stdout: "q": 0, 2024-04-24T09:25:14.089 INFO:teuthology.orchestra.run.smithi149.stdout: "exported": 0, 2024-04-24T09:25:14.090 INFO:teuthology.orchestra.run.smithi149.stdout: "exported_inodes": 0, 2024-04-24T09:25:14.090 INFO:teuthology.orchestra.run.smithi149.stdout: "imported": 1, 2024-04-24T09:25:14.090 INFO:teuthology.orchestra.run.smithi149.stdout: "imported_inodes": 10, 2024-04-24T09:25:14.090 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_dir_fetch": 0, 2024-04-24T09:25:14.090 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_backtrace_fetch": 0, 2024-04-24T09:25:14.090 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_peer_discover": 0, 2024-04-24T09:25:14.090 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rfiles": 4, 2024-04-24T09:25:14.090 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rbytes": 234, 2024-04-24T09:25:14.091 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rsnaps": 0, 2024-04-24T09:25:14.091 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_fetch": 0, 2024-04-24T09:25:14.091 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_set_tag": 0, 2024-04-24T09:25:14.091 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T09:25:14.091 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_inotable_repaired": 0, 2024-04-24T09:25:14.091 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_inodes": 0, 2024-04-24T09:25:14.091 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T09:25:14.092 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dirfrag_rstats": 0, 2024-04-24T09:25:14.092 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_file_inodes": 0, 2024-04-24T09:25:14.092 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_inode_file_caps": 0, 2024-04-24T09:25:14.093 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_revoke": 0, 2024-04-24T09:25:14.093 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_grant": 15, 2024-04-24T09:25:14.093 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T09:25:14.093 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T09:25:14.094 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T09:25:14.094 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps": 17, 2024-04-24T09:25:14.094 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps_dirty": 17, 2024-04-24T09:25:14.095 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_cap_release": 3, 2024-04-24T09:25:14.095 INFO:teuthology.orchestra.run.smithi149.stdout: "process_request_cap_release": 17 2024-04-24T09:25:14.095 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.095 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_cache": { 2024-04-24T09:25:14.095 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update": 6, 2024-04-24T09:25:14.095 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update_receipt": 72, 2024-04-24T09:25:14.096 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_try_discover": 45, 2024-04-24T09:25:14.096 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_send_discover": 51, 2024-04-24T09:25:14.096 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_handle_discover": 7, 2024-04-24T09:25:14.096 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays": 0, 2024-04-24T09:25:14.096 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_delayed": 0, 2024-04-24T09:25:14.096 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_enqueuing": 0, 2024-04-24T09:25:14.097 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_created": 5, 2024-04-24T09:25:14.097 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_enqueued": 5, 2024-04-24T09:25:14.097 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_reintegrated": 0, 2024-04-24T09:25:14.097 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_migrated": 0, 2024-04-24T09:25:14.097 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_processing": 0, 2024-04-24T09:25:14.097 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_enqueued": 0, 2024-04-24T09:25:14.097 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_prioritized": 0, 2024-04-24T09:25:14.098 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_started": 0, 2024-04-24T09:25:14.098 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_completed": 0, 2024-04-24T09:25:14.098 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_quiesce_path": 0, 2024-04-24T09:25:14.098 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_quiesce_inode": 0, 2024-04-24T09:25:14.098 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T09:25:14.098 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_exportdir": 0, 2024-04-24T09:25:14.098 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_flush": 0, 2024-04-24T09:25:14.098 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragmentdir": 0, 2024-04-24T09:25:14.099 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragstats": 0, 2024-04-24T09:25:14.099 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_inodestats": 0 2024-04-24T09:25:14.099 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.099 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_log": { 2024-04-24T09:25:14.099 INFO:teuthology.orchestra.run.smithi149.stdout: "evlrg": 0, 2024-04-24T09:25:14.099 INFO:teuthology.orchestra.run.smithi149.stdout: "evadd": 55, 2024-04-24T09:25:14.099 INFO:teuthology.orchestra.run.smithi149.stdout: "evex": 0, 2024-04-24T09:25:14.099 INFO:teuthology.orchestra.run.smithi149.stdout: "evtrm": 0, 2024-04-24T09:25:14.099 INFO:teuthology.orchestra.run.smithi149.stdout: "ev": 55, 2024-04-24T09:25:14.100 INFO:teuthology.orchestra.run.smithi149.stdout: "evexg": 0, 2024-04-24T09:25:14.100 INFO:teuthology.orchestra.run.smithi149.stdout: "evexd": 0, 2024-04-24T09:25:14.100 INFO:teuthology.orchestra.run.smithi149.stdout: "segadd": 2, 2024-04-24T09:25:14.100 INFO:teuthology.orchestra.run.smithi149.stdout: "segex": 0, 2024-04-24T09:25:14.100 INFO:teuthology.orchestra.run.smithi149.stdout: "segtrm": 0, 2024-04-24T09:25:14.100 INFO:teuthology.orchestra.run.smithi149.stdout: "seg": 2, 2024-04-24T09:25:14.100 INFO:teuthology.orchestra.run.smithi149.stdout: "segmjr": 2, 2024-04-24T09:25:14.100 INFO:teuthology.orchestra.run.smithi149.stdout: "segexg": 0, 2024-04-24T09:25:14.100 INFO:teuthology.orchestra.run.smithi149.stdout: "segexd": 0, 2024-04-24T09:25:14.101 INFO:teuthology.orchestra.run.smithi149.stdout: "expos": 4194304, 2024-04-24T09:25:14.101 INFO:teuthology.orchestra.run.smithi149.stdout: "wrpos": 4266212, 2024-04-24T09:25:14.101 INFO:teuthology.orchestra.run.smithi149.stdout: "rdpos": 0, 2024-04-24T09:25:14.101 INFO:teuthology.orchestra.run.smithi149.stdout: "jlat": { 2024-04-24T09:25:14.101 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 17, 2024-04-24T09:25:14.101 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.249221838, 2024-04-24T09:25:14.101 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.014660108 2024-04-24T09:25:14.101 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.102 INFO:teuthology.orchestra.run.smithi149.stdout: "replayed": 0 2024-04-24T09:25:14.102 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.102 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_mem": { 2024-04-24T09:25:14.102 INFO:teuthology.orchestra.run.smithi149.stdout: "ino": 53, 2024-04-24T09:25:14.102 INFO:teuthology.orchestra.run.smithi149.stdout: "ino+": 63, 2024-04-24T09:25:14.103 INFO:teuthology.orchestra.run.smithi149.stdout: "ino-": 10, 2024-04-24T09:25:14.103 INFO:teuthology.orchestra.run.smithi149.stdout: "dir": 45, 2024-04-24T09:25:14.103 INFO:teuthology.orchestra.run.smithi149.stdout: "dir+": 53, 2024-04-24T09:25:14.103 INFO:teuthology.orchestra.run.smithi149.stdout: "dir-": 8, 2024-04-24T09:25:14.104 INFO:teuthology.orchestra.run.smithi149.stdout: "dn": 76, 2024-04-24T09:25:14.104 INFO:teuthology.orchestra.run.smithi149.stdout: "dn+": 96, 2024-04-24T09:25:14.104 INFO:teuthology.orchestra.run.smithi149.stdout: "dn-": 20, 2024-04-24T09:25:14.104 INFO:teuthology.orchestra.run.smithi149.stdout: "cap": 4, 2024-04-24T09:25:14.105 INFO:teuthology.orchestra.run.smithi149.stdout: "cap+": 20, 2024-04-24T09:25:14.105 INFO:teuthology.orchestra.run.smithi149.stdout: "cap-": 16, 2024-04-24T09:25:14.105 INFO:teuthology.orchestra.run.smithi149.stdout: "rss": 52896, 2024-04-24T09:25:14.105 INFO:teuthology.orchestra.run.smithi149.stdout: "heap": 223608 2024-04-24T09:25:14.105 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.105 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_server": { 2024-04-24T09:25:14.106 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_client_request": 169, 2024-04-24T09:25:14.106 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_server_request": 0, 2024-04-24T09:25:14.106 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_request": 147, 2024-04-24T09:25:14.106 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_session": 3, 2024-04-24T09:25:14.106 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_peer_request": 14, 2024-04-24T09:25:14.106 INFO:teuthology.orchestra.run.smithi149.stdout: "req_create_latency": { 2024-04-24T09:25:14.106 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 8, 2024-04-24T09:25:14.106 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.017847711, 2024-04-24T09:25:14.106 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.002230963 2024-04-24T09:25:14.107 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.107 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getattr_latency": { 2024-04-24T09:25:14.107 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:14.107 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:14.107 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:14.107 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.107 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getfilelock_latency": { 2024-04-24T09:25:14.107 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:14.107 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:14.108 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:14.108 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.108 INFO:teuthology.orchestra.run.smithi149.stdout: "req_link_latency": { 2024-04-24T09:25:14.108 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:14.108 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:14.108 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:14.109 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.109 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookup_latency": { 2024-04-24T09:25:14.109 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 51, 2024-04-24T09:25:14.109 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.015166881, 2024-04-24T09:25:14.109 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000297389 2024-04-24T09:25:14.109 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.109 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookuphash_latency": { 2024-04-24T09:25:14.109 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:14.110 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:14.110 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:14.110 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.110 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupino_latency": { 2024-04-24T09:25:14.110 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1, 2024-04-24T09:25:14.110 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000282712, 2024-04-24T09:25:14.110 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000282712 2024-04-24T09:25:14.111 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.111 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupname_latency": { 2024-04-24T09:25:14.111 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:14.111 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:14.111 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:14.111 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.111 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupparent_latency": { 2024-04-24T09:25:14.111 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:14.111 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:14.112 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:14.112 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.112 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupsnap_latency": { 2024-04-24T09:25:14.112 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:14.112 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:14.112 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:14.112 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.112 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lssnap_latency": { 2024-04-24T09:25:14.113 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:14.113 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:14.113 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:14.113 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.113 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mkdir_latency": { 2024-04-24T09:25:14.114 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1, 2024-04-24T09:25:14.114 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.002776915, 2024-04-24T09:25:14.115 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.002776915 2024-04-24T09:25:14.115 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.115 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mknod_latency": { 2024-04-24T09:25:14.115 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:14.116 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:14.116 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:14.116 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.116 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mksnap_latency": { 2024-04-24T09:25:14.116 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:14.117 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:14.117 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:14.117 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.117 INFO:teuthology.orchestra.run.smithi149.stdout: "req_open_latency": { 2024-04-24T09:25:14.117 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 14, 2024-04-24T09:25:14.117 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.003519062, 2024-04-24T09:25:14.117 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000251361 2024-04-24T09:25:14.117 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.118 INFO:teuthology.orchestra.run.smithi149.stdout: "req_readdir_latency": { 2024-04-24T09:25:14.118 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 3, 2024-04-24T09:25:14.118 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.001707634, 2024-04-24T09:25:14.118 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000569211 2024-04-24T09:25:14.118 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.118 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rename_latency": { 2024-04-24T09:25:14.118 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:14.118 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:14.118 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:14.119 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.119 INFO:teuthology.orchestra.run.smithi149.stdout: "req_renamesnap_latency": { 2024-04-24T09:25:14.119 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:14.119 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:14.119 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:14.119 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.119 INFO:teuthology.orchestra.run.smithi149.stdout: "req_snapdiff_latency": { 2024-04-24T09:25:14.119 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:14.119 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:14.120 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:14.120 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.120 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmdir_latency": { 2024-04-24T09:25:14.120 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:14.120 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:14.120 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:14.120 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.121 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmsnap_latency": { 2024-04-24T09:25:14.121 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:14.121 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:14.121 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:14.121 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.121 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmxattr_latency": { 2024-04-24T09:25:14.121 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:14.121 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:14.121 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:14.122 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.122 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setattr_latency": { 2024-04-24T09:25:14.122 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:14.122 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:14.122 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:14.122 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.122 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setdirlayout_latency": { 2024-04-24T09:25:14.122 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:14.122 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:14.123 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:14.123 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.123 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setfilelock_latency": { 2024-04-24T09:25:14.123 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:14.123 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:14.123 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:14.123 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.123 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setlayout_latency": { 2024-04-24T09:25:14.123 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:14.124 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:14.124 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:14.124 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.124 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setxattr_latency": { 2024-04-24T09:25:14.124 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:14.124 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:14.124 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:14.124 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.124 INFO:teuthology.orchestra.run.smithi149.stdout: "req_symlink_latency": { 2024-04-24T09:25:14.124 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:14.125 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:14.125 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:14.125 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.125 INFO:teuthology.orchestra.run.smithi149.stdout: "req_unlink_latency": { 2024-04-24T09:25:14.125 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 5, 2024-04-24T09:25:14.125 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.009245490, 2024-04-24T09:25:14.126 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.001849098 2024-04-24T09:25:14.126 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.126 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_revoke_eviction": 0, 2024-04-24T09:25:14.126 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_acquisition_throttle": 0, 2024-04-24T09:25:14.127 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getvxattr_latency": { 2024-04-24T09:25:14.127 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:14.127 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:14.127 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:14.128 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-04-24T09:25:14.128 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.128 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_sessions": { 2024-04-24T09:25:14.128 INFO:teuthology.orchestra.run.smithi149.stdout: "session_count": 1, 2024-04-24T09:25:14.128 INFO:teuthology.orchestra.run.smithi149.stdout: "session_add": 1, 2024-04-24T09:25:14.128 INFO:teuthology.orchestra.run.smithi149.stdout: "session_remove": 0, 2024-04-24T09:25:14.129 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_open": 1, 2024-04-24T09:25:14.129 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_stale": 0, 2024-04-24T09:25:14.129 INFO:teuthology.orchestra.run.smithi149.stdout: "total_load": 74, 2024-04-24T09:25:14.129 INFO:teuthology.orchestra.run.smithi149.stdout: "average_load": 74, 2024-04-24T09:25:14.129 INFO:teuthology.orchestra.run.smithi149.stdout: "avg_session_uptime": 201, 2024-04-24T09:25:14.129 INFO:teuthology.orchestra.run.smithi149.stdout: "mdthresh_evicted": 0 2024-04-24T09:25:14.129 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.129 INFO:teuthology.orchestra.run.smithi149.stdout: "mempool": { 2024-04-24T09:25:14.129 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_bytes": 0, 2024-04-24T09:25:14.130 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_items": 0, 2024-04-24T09:25:14.130 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T09:25:14.130 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_items": 0, 2024-04-24T09:25:14.130 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T09:25:14.130 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_items": 0, 2024-04-24T09:25:14.130 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T09:25:14.130 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T09:25:14.130 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T09:25:14.130 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T09:25:14.131 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T09:25:14.131 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_items": 0, 2024-04-24T09:25:14.131 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T09:25:14.131 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T09:25:14.131 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_bytes": 0, 2024-04-24T09:25:14.131 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_items": 0, 2024-04-24T09:25:14.131 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_bytes": 0, 2024-04-24T09:25:14.131 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_items": 0, 2024-04-24T09:25:14.131 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T09:25:14.132 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T09:25:14.132 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T09:25:14.132 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T09:25:14.132 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T09:25:14.132 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_items": 0, 2024-04-24T09:25:14.132 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_bytes": 0, 2024-04-24T09:25:14.132 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_items": 0, 2024-04-24T09:25:14.132 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T09:25:14.133 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T09:25:14.133 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_bytes": 0, 2024-04-24T09:25:14.133 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_items": 0, 2024-04-24T09:25:14.133 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_bytes": 0, 2024-04-24T09:25:14.133 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_items": 0, 2024-04-24T09:25:14.133 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T09:25:14.133 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_items": 0, 2024-04-24T09:25:14.134 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T09:25:14.134 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_items": 0, 2024-04-24T09:25:14.134 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_bytes": 302167, 2024-04-24T09:25:14.134 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_items": 162, 2024-04-24T09:25:14.134 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_bytes": 792, 2024-04-24T09:25:14.134 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_items": 9, 2024-04-24T09:25:14.134 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_bytes": 0, 2024-04-24T09:25:14.134 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_items": 0, 2024-04-24T09:25:14.135 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_bytes": 0, 2024-04-24T09:25:14.135 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_items": 0, 2024-04-24T09:25:14.135 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_bytes": 0, 2024-04-24T09:25:14.135 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_items": 0, 2024-04-24T09:25:14.135 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_bytes": 7624, 2024-04-24T09:25:14.135 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_items": 160, 2024-04-24T09:25:14.135 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T09:25:14.135 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_items": 0, 2024-04-24T09:25:14.135 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_bytes": 0, 2024-04-24T09:25:14.136 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_items": 0, 2024-04-24T09:25:14.136 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_bytes": 237944, 2024-04-24T09:25:14.136 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_items": 1631, 2024-04-24T09:25:14.136 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_bytes": 0, 2024-04-24T09:25:14.136 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_items": 0, 2024-04-24T09:25:14.136 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_bytes": 0, 2024-04-24T09:25:14.136 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_items": 0 2024-04-24T09:25:14.136 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.136 INFO:teuthology.orchestra.run.smithi149.stdout: "objecter": { 2024-04-24T09:25:14.136 INFO:teuthology.orchestra.run.smithi149.stdout: "op_active": 0, 2024-04-24T09:25:14.137 INFO:teuthology.orchestra.run.smithi149.stdout: "op_laggy": 0, 2024-04-24T09:25:14.137 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send": 85, 2024-04-24T09:25:14.137 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send_bytes": 86332, 2024-04-24T09:25:14.137 INFO:teuthology.orchestra.run.smithi149.stdout: "op_resend": 33, 2024-04-24T09:25:14.137 INFO:teuthology.orchestra.run.smithi149.stdout: "op_reply": 85, 2024-04-24T09:25:14.137 INFO:teuthology.orchestra.run.smithi149.stdout: "op_latency": { 2024-04-24T09:25:14.137 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 85, 2024-04-24T09:25:14.138 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.672986217, 2024-04-24T09:25:14.138 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.007917484 2024-04-24T09:25:14.138 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.138 INFO:teuthology.orchestra.run.smithi149.stdout: "op_inflight": 0, 2024-04-24T09:25:14.138 INFO:teuthology.orchestra.run.smithi149.stdout: "oplen_avg": { 2024-04-24T09:25:14.138 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 85, 2024-04-24T09:25:14.138 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 120 2024-04-24T09:25:14.138 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.139 INFO:teuthology.orchestra.run.smithi149.stdout: "op": 85, 2024-04-24T09:25:14.139 INFO:teuthology.orchestra.run.smithi149.stdout: "op_r": 1, 2024-04-24T09:25:14.139 INFO:teuthology.orchestra.run.smithi149.stdout: "op_w": 84, 2024-04-24T09:25:14.139 INFO:teuthology.orchestra.run.smithi149.stdout: "op_rmw": 0, 2024-04-24T09:25:14.139 INFO:teuthology.orchestra.run.smithi149.stdout: "op_pg": 0, 2024-04-24T09:25:14.139 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_stat": 0, 2024-04-24T09:25:14.139 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_create": 10, 2024-04-24T09:25:14.139 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_read": 1, 2024-04-24T09:25:14.139 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_write": 18, 2024-04-24T09:25:14.140 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writefull": 13, 2024-04-24T09:25:14.140 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writesame": 0, 2024-04-24T09:25:14.140 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_append": 0, 2024-04-24T09:25:14.140 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_zero": 0, 2024-04-24T09:25:14.140 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_truncate": 0, 2024-04-24T09:25:14.140 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_delete": 17, 2024-04-24T09:25:14.140 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_mapext": 0, 2024-04-24T09:25:14.140 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_sparse_read": 0, 2024-04-24T09:25:14.140 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_clonerange": 0, 2024-04-24T09:25:14.140 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_getxattr": 0, 2024-04-24T09:25:14.141 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_setxattr": 20, 2024-04-24T09:25:14.141 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_cmpxattr": 0, 2024-04-24T09:25:14.141 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_rmxattr": 0, 2024-04-24T09:25:14.141 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_resetxattrs": 0, 2024-04-24T09:25:14.141 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_call": 0, 2024-04-24T09:25:14.141 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_watch": 0, 2024-04-24T09:25:14.141 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_notify": 0, 2024-04-24T09:25:14.141 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T09:25:14.141 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls": 0, 2024-04-24T09:25:14.142 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls_filter": 0, 2024-04-24T09:25:14.142 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_other": 0, 2024-04-24T09:25:14.142 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_active": 0, 2024-04-24T09:25:14.142 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_send": 0, 2024-04-24T09:25:14.142 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_resend": 0, 2024-04-24T09:25:14.142 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_ping": 0, 2024-04-24T09:25:14.142 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_active": 0, 2024-04-24T09:25:14.142 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_send": 0, 2024-04-24T09:25:14.142 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_resend": 0, 2024-04-24T09:25:14.143 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_active": 0, 2024-04-24T09:25:14.143 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_send": 0, 2024-04-24T09:25:14.143 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_resend": 0, 2024-04-24T09:25:14.143 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_active": 0, 2024-04-24T09:25:14.143 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_send": 0, 2024-04-24T09:25:14.143 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_resend": 0, 2024-04-24T09:25:14.143 INFO:teuthology.orchestra.run.smithi149.stdout: "command_active": 0, 2024-04-24T09:25:14.143 INFO:teuthology.orchestra.run.smithi149.stdout: "command_send": 0, 2024-04-24T09:25:14.143 INFO:teuthology.orchestra.run.smithi149.stdout: "command_resend": 0, 2024-04-24T09:25:14.144 INFO:teuthology.orchestra.run.smithi149.stdout: "map_epoch": 88, 2024-04-24T09:25:14.144 INFO:teuthology.orchestra.run.smithi149.stdout: "map_full": 0, 2024-04-24T09:25:14.144 INFO:teuthology.orchestra.run.smithi149.stdout: "map_inc": 2, 2024-04-24T09:25:14.144 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_sessions": 11, 2024-04-24T09:25:14.144 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_open": 11, 2024-04-24T09:25:14.144 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_close": 0, 2024-04-24T09:25:14.144 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_laggy": 0, 2024-04-24T09:25:14.144 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_wr": 36, 2024-04-24T09:25:14.144 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_rd": 0, 2024-04-24T09:25:14.144 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_del": 5 2024-04-24T09:25:14.145 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.145 INFO:teuthology.orchestra.run.smithi149.stdout: "oft": { 2024-04-24T09:25:14.145 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_objs": 1, 2024-04-24T09:25:14.145 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_kv_pairs": 0, 2024-04-24T09:25:14.145 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_updates": 0, 2024-04-24T09:25:14.145 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_removes": 0 2024-04-24T09:25:14.145 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.145 INFO:teuthology.orchestra.run.smithi149.stdout: "osd-slow-ops": { 2024-04-24T09:25:14.145 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_ops_count": 0 2024-04-24T09:25:14.145 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.146 INFO:teuthology.orchestra.run.smithi149.stdout: "purge_queue": { 2024-04-24T09:25:14.146 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops": 0, 2024-04-24T09:25:14.146 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops_high_water": 5, 2024-04-24T09:25:14.146 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing": 0, 2024-04-24T09:25:14.146 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_high_water": 5, 2024-04-24T09:25:14.146 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed_ops": 5, 2024-04-24T09:25:14.146 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed": 5, 2024-04-24T09:25:14.146 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_item_in_journal": 0 2024-04-24T09:25:14.147 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.147 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T09:25:14.147 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-04-24T09:25:14.147 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-04-24T09:25:14.147 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-04-24T09:25:14.147 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 1222, 2024-04-24T09:25:14.147 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 1095637, 2024-04-24T09:25:14.147 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-04-24T09:25:14.147 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 1222, 2024-04-24T09:25:14.147 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-04-24T09:25:14.148 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-04-24T09:25:14.148 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 1222, 2024-04-24T09:25:14.148 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 1095637, 2024-04-24T09:25:14.148 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-04-24T09:25:14.148 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:14.148 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:14.148 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:14.148 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-04-24T09:25:14.148 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.149 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_bytes": { 2024-04-24T09:25:14.149 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-04-24T09:25:14.149 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-04-24T09:25:14.149 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-04-24T09:25:14.149 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-04-24T09:25:14.149 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-04-24T09:25:14.149 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-04-24T09:25:14.149 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-04-24T09:25:14.149 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 85, 2024-04-24T09:25:14.150 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 86837, 2024-04-24T09:25:14.150 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 85, 2024-04-24T09:25:14.150 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 86837, 2024-04-24T09:25:14.150 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-04-24T09:25:14.150 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:14.150 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:14.150 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:14.150 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-04-24T09:25:14.151 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.151 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_ops": { 2024-04-24T09:25:14.151 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-04-24T09:25:14.151 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 1024, 2024-04-24T09:25:14.151 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-04-24T09:25:14.151 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-04-24T09:25:14.151 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-04-24T09:25:14.151 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-04-24T09:25:14.151 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-04-24T09:25:14.151 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 85, 2024-04-24T09:25:14.152 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 85, 2024-04-24T09:25:14.152 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 85, 2024-04-24T09:25:14.152 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 85, 2024-04-24T09:25:14.152 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-04-24T09:25:14.152 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:14.152 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:14.152 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:14.152 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-04-24T09:25:14.153 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.153 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle": { 2024-04-24T09:25:14.153 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-04-24T09:25:14.153 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-04-24T09:25:14.153 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-04-24T09:25:14.153 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 5, 2024-04-24T09:25:14.153 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 505, 2024-04-24T09:25:14.153 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-04-24T09:25:14.153 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 5, 2024-04-24T09:25:14.153 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-04-24T09:25:14.154 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-04-24T09:25:14.154 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 1, 2024-04-24T09:25:14.154 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 505, 2024-04-24T09:25:14.154 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-04-24T09:25:14.154 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:14.154 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:14.154 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:14.154 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-04-24T09:25:14.154 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:14.155 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle-0x55769b63e500": { 2024-04-24T09:25:14.155 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-04-24T09:25:14.155 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-04-24T09:25:14.155 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-04-24T09:25:14.155 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 55, 2024-04-24T09:25:14.155 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 71908, 2024-04-24T09:25:14.155 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-04-24T09:25:14.155 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 55, 2024-04-24T09:25:14.155 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-04-24T09:25:14.156 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-04-24T09:25:14.156 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 17, 2024-04-24T09:25:14.156 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 71908, 2024-04-24T09:25:14.156 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-04-24T09:25:14.156 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:14.156 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:14.156 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:14.156 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-04-24T09:25:14.156 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-04-24T09:25:14.157 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-04-24T09:25:14.896 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-24T09:25:14.896 INFO:tasks.check_counter:Daemon mds.c mds.imported=1 2024-04-24T09:25:14.896 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=6 2024-04-24T09:25:14.896 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=72 2024-04-24T09:25:14.896 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-04-24T09:25:14.897 DEBUG:tasks.check_counter:Getting stats from f 2024-04-24T09:25:14.897 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-24T09:25:14.925 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:14 smithi114 ceph-mon[32501]: pgmap v506: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-04-24T09:25:15.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:14 smithi090 ceph-mon[27825]: pgmap v506: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-04-24T09:25:15.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:14 smithi149 ceph-mon[32592]: pgmap v506: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-04-24T09:25:15.145 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T09:25:16.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:15 smithi090 ceph-mon[27825]: pgmap v507: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 767 B/s wr, 11 op/s 2024-04-24T09:25:16.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:15 smithi149 ceph-mon[32592]: pgmap v507: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 767 B/s wr, 11 op/s 2024-04-24T09:25:16.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:15 smithi114 ceph-mon[32501]: pgmap v507: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 767 B/s wr, 11 op/s 2024-04-24T09:25:16.824 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-04-24T09:25:16.824 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T09:25:16.824 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 847, 2024-04-24T09:25:16.824 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 849, 2024-04-24T09:25:16.824 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 21272120, 2024-04-24T09:25:16.824 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 269356, 2024-04-24T09:25:16.824 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 7, 2024-04-24T09:25:16.825 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 3, 2024-04-24T09:25:16.825 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.097114024, 2024-04-24T09:25:16.825 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.026247733, 2024-04-24T09:25:16.825 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.036195837, 2024-04-24T09:25:16.825 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.026461916, 2024-04-24T09:25:16.825 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:25:16.825 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 847, 2024-04-24T09:25:16.825 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.035070652, 2024-04-24T09:25:16.826 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000041405 2024-04-24T09:25:16.826 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.826 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:25:16.826 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:16.826 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:16.826 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:16.826 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.827 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T09:25:16.827 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T09:25:16.827 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.827 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T09:25:16.827 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 192, 2024-04-24T09:25:16.827 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 178, 2024-04-24T09:25:16.827 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 17588283, 2024-04-24T09:25:16.827 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 86746, 2024-04-24T09:25:16.828 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 4, 2024-04-24T09:25:16.828 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 3, 2024-04-24T09:25:16.828 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.034526348, 2024-04-24T09:25:16.828 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.008091525, 2024-04-24T09:25:16.828 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.026117820, 2024-04-24T09:25:16.828 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.004101254, 2024-04-24T09:25:16.828 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:25:16.828 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 178, 2024-04-24T09:25:16.829 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.013907856, 2024-04-24T09:25:16.829 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000078134 2024-04-24T09:25:16.829 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.829 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:25:16.829 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:16.829 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:16.829 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:16.829 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.829 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 791968, 2024-04-24T09:25:16.830 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 67600 2024-04-24T09:25:16.830 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.830 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T09:25:16.830 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 291, 2024-04-24T09:25:16.830 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 373, 2024-04-24T09:25:16.830 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 8460659, 2024-04-24T09:25:16.830 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 389037, 2024-04-24T09:25:16.830 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 6, 2024-04-24T09:25:16.831 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 4, 2024-04-24T09:25:16.831 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.039456268, 2024-04-24T09:25:16.831 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.014170796, 2024-04-24T09:25:16.831 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.012560291, 2024-04-24T09:25:16.831 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.009059776, 2024-04-24T09:25:16.831 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:25:16.831 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 371, 2024-04-24T09:25:16.831 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.020582364, 2024-04-24T09:25:16.832 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000055478 2024-04-24T09:25:16.832 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.832 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:25:16.832 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:16.832 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:16.832 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:16.832 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.833 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-24T09:25:16.833 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 296816 2024-04-24T09:25:16.833 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.833 INFO:teuthology.orchestra.run.smithi149.stdout: "cct": { 2024-04-24T09:25:16.833 INFO:teuthology.orchestra.run.smithi149.stdout: "total_workers": 1, 2024-04-24T09:25:16.833 INFO:teuthology.orchestra.run.smithi149.stdout: "unhealthy_workers": 0 2024-04-24T09:25:16.833 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.833 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-MDSRank": { 2024-04-24T09:25:16.833 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-04-24T09:25:16.834 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-04-24T09:25:16.834 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1931, 2024-04-24T09:25:16.834 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.055357100, 2024-04-24T09:25:16.834 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000028667 2024-04-24T09:25:16.834 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-04-24T09:25:16.834 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.834 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-PurgeQueue": { 2024-04-24T09:25:16.835 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-04-24T09:25:16.835 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-04-24T09:25:16.835 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:16.835 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:16.835 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:16.835 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-04-24T09:25:16.835 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.836 INFO:teuthology.orchestra.run.smithi149.stdout: "mds": { 2024-04-24T09:25:16.836 INFO:teuthology.orchestra.run.smithi149.stdout: "request": 0, 2024-04-24T09:25:16.836 INFO:teuthology.orchestra.run.smithi149.stdout: "reply": 0, 2024-04-24T09:25:16.836 INFO:teuthology.orchestra.run.smithi149.stdout: "reply_latency": { 2024-04-24T09:25:16.836 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:16.836 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:16.836 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:16.837 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.837 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_reply": 0, 2024-04-24T09:25:16.837 INFO:teuthology.orchestra.run.smithi149.stdout: "forward": 0, 2024-04-24T09:25:16.837 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_complete": 0, 2024-04-24T09:25:16.837 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_keys": 0, 2024-04-24T09:25:16.837 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_commit": 0, 2024-04-24T09:25:16.837 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_split": 0, 2024-04-24T09:25:16.837 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_merge": 0, 2024-04-24T09:25:16.837 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes": 7843, 2024-04-24T09:25:16.837 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_top": 0, 2024-04-24T09:25:16.838 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_bottom": 7843, 2024-04-24T09:25:16.838 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pin_tail": 0, 2024-04-24T09:25:16.838 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pinned": 7843, 2024-04-24T09:25:16.838 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_expired": 0, 2024-04-24T09:25:16.838 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_with_caps": 0, 2024-04-24T09:25:16.838 INFO:teuthology.orchestra.run.smithi149.stdout: "caps": 0, 2024-04-24T09:25:16.838 INFO:teuthology.orchestra.run.smithi149.stdout: "subtrees": 7, 2024-04-24T09:25:16.838 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse": 0, 2024-04-24T09:25:16.839 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_hit": 0, 2024-04-24T09:25:16.839 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_forward": 0, 2024-04-24T09:25:16.839 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_discover": 0, 2024-04-24T09:25:16.839 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_dir_fetch": 0, 2024-04-24T09:25:16.839 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_remote_ino": 0, 2024-04-24T09:25:16.839 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_lock": 0, 2024-04-24T09:25:16.839 INFO:teuthology.orchestra.run.smithi149.stdout: "load_cent": 0, 2024-04-24T09:25:16.839 INFO:teuthology.orchestra.run.smithi149.stdout: "q": 0, 2024-04-24T09:25:16.839 INFO:teuthology.orchestra.run.smithi149.stdout: "exported": 0, 2024-04-24T09:25:16.840 INFO:teuthology.orchestra.run.smithi149.stdout: "exported_inodes": 0, 2024-04-24T09:25:16.840 INFO:teuthology.orchestra.run.smithi149.stdout: "imported": 0, 2024-04-24T09:25:16.840 INFO:teuthology.orchestra.run.smithi149.stdout: "imported_inodes": 0, 2024-04-24T09:25:16.840 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_dir_fetch": 0, 2024-04-24T09:25:16.840 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_backtrace_fetch": 0, 2024-04-24T09:25:16.840 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_peer_discover": 0, 2024-04-24T09:25:16.840 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rfiles": 3454, 2024-04-24T09:25:16.840 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rbytes": 26625079, 2024-04-24T09:25:16.841 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rsnaps": 0, 2024-04-24T09:25:16.841 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_fetch": 0, 2024-04-24T09:25:16.841 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_set_tag": 0, 2024-04-24T09:25:16.841 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T09:25:16.841 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_inotable_repaired": 0, 2024-04-24T09:25:16.841 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_inodes": 0, 2024-04-24T09:25:16.841 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T09:25:16.841 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dirfrag_rstats": 0, 2024-04-24T09:25:16.842 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_file_inodes": 0, 2024-04-24T09:25:16.842 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_inode_file_caps": 0, 2024-04-24T09:25:16.842 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_revoke": 0, 2024-04-24T09:25:16.842 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_grant": 0, 2024-04-24T09:25:16.842 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T09:25:16.842 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T09:25:16.842 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T09:25:16.843 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps": 0, 2024-04-24T09:25:16.843 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps_dirty": 0, 2024-04-24T09:25:16.843 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_cap_release": 0, 2024-04-24T09:25:16.843 INFO:teuthology.orchestra.run.smithi149.stdout: "process_request_cap_release": 0 2024-04-24T09:25:16.844 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.844 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_cache": { 2024-04-24T09:25:16.844 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update": 0, 2024-04-24T09:25:16.844 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update_receipt": 0, 2024-04-24T09:25:16.844 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_try_discover": 0, 2024-04-24T09:25:16.844 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_send_discover": 0, 2024-04-24T09:25:16.844 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_handle_discover": 0, 2024-04-24T09:25:16.844 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays": 0, 2024-04-24T09:25:16.845 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_delayed": 0, 2024-04-24T09:25:16.845 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_enqueuing": 0, 2024-04-24T09:25:16.845 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_created": 0, 2024-04-24T09:25:16.845 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_enqueued": 0, 2024-04-24T09:25:16.845 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_reintegrated": 0, 2024-04-24T09:25:16.845 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_migrated": 0, 2024-04-24T09:25:16.845 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_processing": 0, 2024-04-24T09:25:16.845 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_enqueued": 0, 2024-04-24T09:25:16.845 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_prioritized": 0, 2024-04-24T09:25:16.846 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_started": 0, 2024-04-24T09:25:16.846 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_completed": 0, 2024-04-24T09:25:16.846 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_quiesce_path": 0, 2024-04-24T09:25:16.846 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_quiesce_inode": 0, 2024-04-24T09:25:16.846 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T09:25:16.846 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_exportdir": 0, 2024-04-24T09:25:16.847 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_flush": 0, 2024-04-24T09:25:16.847 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragmentdir": 0, 2024-04-24T09:25:16.847 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragstats": 0, 2024-04-24T09:25:16.847 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_inodestats": 0 2024-04-24T09:25:16.847 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.847 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_log": { 2024-04-24T09:25:16.847 INFO:teuthology.orchestra.run.smithi149.stdout: "evlrg": 0, 2024-04-24T09:25:16.847 INFO:teuthology.orchestra.run.smithi149.stdout: "evadd": 0, 2024-04-24T09:25:16.847 INFO:teuthology.orchestra.run.smithi149.stdout: "evex": 0, 2024-04-24T09:25:16.848 INFO:teuthology.orchestra.run.smithi149.stdout: "evtrm": 0, 2024-04-24T09:25:16.848 INFO:teuthology.orchestra.run.smithi149.stdout: "ev": 23150, 2024-04-24T09:25:16.848 INFO:teuthology.orchestra.run.smithi149.stdout: "evexg": 0, 2024-04-24T09:25:16.848 INFO:teuthology.orchestra.run.smithi149.stdout: "evexd": 0, 2024-04-24T09:25:16.848 INFO:teuthology.orchestra.run.smithi149.stdout: "segadd": 0, 2024-04-24T09:25:16.848 INFO:teuthology.orchestra.run.smithi149.stdout: "segex": 0, 2024-04-24T09:25:16.848 INFO:teuthology.orchestra.run.smithi149.stdout: "segtrm": 0, 2024-04-24T09:25:16.848 INFO:teuthology.orchestra.run.smithi149.stdout: "seg": 29, 2024-04-24T09:25:16.849 INFO:teuthology.orchestra.run.smithi149.stdout: "segmjr": 3, 2024-04-24T09:25:16.849 INFO:teuthology.orchestra.run.smithi149.stdout: "segexg": 0, 2024-04-24T09:25:16.849 INFO:teuthology.orchestra.run.smithi149.stdout: "segexd": 0, 2024-04-24T09:25:16.849 INFO:teuthology.orchestra.run.smithi149.stdout: "expos": 4194304, 2024-04-24T09:25:16.849 INFO:teuthology.orchestra.run.smithi149.stdout: "wrpos": 50384749, 2024-04-24T09:25:16.849 INFO:teuthology.orchestra.run.smithi149.stdout: "rdpos": 50378629, 2024-04-24T09:25:16.849 INFO:teuthology.orchestra.run.smithi149.stdout: "jlat": { 2024-04-24T09:25:16.850 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:16.850 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:16.850 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:16.850 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.850 INFO:teuthology.orchestra.run.smithi149.stdout: "replayed": 23150 2024-04-24T09:25:16.850 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.850 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_mem": { 2024-04-24T09:25:16.850 INFO:teuthology.orchestra.run.smithi149.stdout: "ino": 3461, 2024-04-24T09:25:16.851 INFO:teuthology.orchestra.run.smithi149.stdout: "ino+": 6772, 2024-04-24T09:25:16.851 INFO:teuthology.orchestra.run.smithi149.stdout: "ino-": 3311, 2024-04-24T09:25:16.851 INFO:teuthology.orchestra.run.smithi149.stdout: "dir": 248, 2024-04-24T09:25:16.851 INFO:teuthology.orchestra.run.smithi149.stdout: "dir+": 289, 2024-04-24T09:25:16.851 INFO:teuthology.orchestra.run.smithi149.stdout: "dir-": 41, 2024-04-24T09:25:16.851 INFO:teuthology.orchestra.run.smithi149.stdout: "dn": 7843, 2024-04-24T09:25:16.851 INFO:teuthology.orchestra.run.smithi149.stdout: "dn+": 7974, 2024-04-24T09:25:16.851 INFO:teuthology.orchestra.run.smithi149.stdout: "dn-": 131, 2024-04-24T09:25:16.851 INFO:teuthology.orchestra.run.smithi149.stdout: "cap": 0, 2024-04-24T09:25:16.852 INFO:teuthology.orchestra.run.smithi149.stdout: "cap+": 0, 2024-04-24T09:25:16.852 INFO:teuthology.orchestra.run.smithi149.stdout: "cap-": 0, 2024-04-24T09:25:16.852 INFO:teuthology.orchestra.run.smithi149.stdout: "rss": 69752, 2024-04-24T09:25:16.852 INFO:teuthology.orchestra.run.smithi149.stdout: "heap": 215416 2024-04-24T09:25:16.852 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.852 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_server": { 2024-04-24T09:25:16.852 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_client_request": 0, 2024-04-24T09:25:16.852 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_server_request": 0, 2024-04-24T09:25:16.852 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_request": 0, 2024-04-24T09:25:16.853 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_session": 0, 2024-04-24T09:25:16.853 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_peer_request": 0, 2024-04-24T09:25:16.853 INFO:teuthology.orchestra.run.smithi149.stdout: "req_create_latency": { 2024-04-24T09:25:16.853 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:16.853 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:16.853 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:16.853 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.853 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getattr_latency": { 2024-04-24T09:25:16.853 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:16.854 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:16.854 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:16.854 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.854 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getfilelock_latency": { 2024-04-24T09:25:16.854 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:16.854 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:16.854 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:16.854 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.855 INFO:teuthology.orchestra.run.smithi149.stdout: "req_link_latency": { 2024-04-24T09:25:16.855 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:16.855 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:16.855 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:16.855 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.855 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookup_latency": { 2024-04-24T09:25:16.855 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:16.855 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:16.856 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:16.856 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.856 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookuphash_latency": { 2024-04-24T09:25:16.856 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:16.856 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:16.856 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:16.856 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.856 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupino_latency": { 2024-04-24T09:25:16.856 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:16.857 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:16.857 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:16.857 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.857 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupname_latency": { 2024-04-24T09:25:16.857 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:16.857 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:16.857 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:16.857 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.857 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupparent_latency": { 2024-04-24T09:25:16.858 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:16.858 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:16.858 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:16.858 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.858 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupsnap_latency": { 2024-04-24T09:25:16.858 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:16.858 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:16.858 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:16.859 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.859 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lssnap_latency": { 2024-04-24T09:25:16.859 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:16.859 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:16.859 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:16.859 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.860 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mkdir_latency": { 2024-04-24T09:25:16.860 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:16.860 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:16.860 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:16.860 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.860 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mknod_latency": { 2024-04-24T09:25:16.860 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:16.861 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:16.861 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:16.861 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.861 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mksnap_latency": { 2024-04-24T09:25:16.861 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:16.861 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:16.861 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:16.861 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.861 INFO:teuthology.orchestra.run.smithi149.stdout: "req_open_latency": { 2024-04-24T09:25:16.862 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:16.862 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:16.862 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:16.862 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.862 INFO:teuthology.orchestra.run.smithi149.stdout: "req_readdir_latency": { 2024-04-24T09:25:16.862 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:16.862 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:16.863 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:16.863 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.863 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rename_latency": { 2024-04-24T09:25:16.863 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:16.863 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:16.863 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:16.863 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.864 INFO:teuthology.orchestra.run.smithi149.stdout: "req_renamesnap_latency": { 2024-04-24T09:25:16.864 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:16.864 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:16.864 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:16.864 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.864 INFO:teuthology.orchestra.run.smithi149.stdout: "req_snapdiff_latency": { 2024-04-24T09:25:16.864 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:16.864 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:16.864 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:16.865 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.865 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmdir_latency": { 2024-04-24T09:25:16.865 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:16.865 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:16.865 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:16.865 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.865 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmsnap_latency": { 2024-04-24T09:25:16.865 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:16.866 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:16.866 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:16.866 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.866 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmxattr_latency": { 2024-04-24T09:25:16.866 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:16.866 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:16.866 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:16.866 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.867 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setattr_latency": { 2024-04-24T09:25:16.867 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:16.867 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:16.867 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:16.867 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.867 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setdirlayout_latency": { 2024-04-24T09:25:16.867 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:16.867 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:16.867 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:16.868 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.868 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setfilelock_latency": { 2024-04-24T09:25:16.868 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:16.868 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:16.868 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:16.868 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.868 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setlayout_latency": { 2024-04-24T09:25:16.868 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:16.869 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:16.869 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:16.869 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.869 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setxattr_latency": { 2024-04-24T09:25:16.869 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:16.869 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:16.869 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:16.869 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.869 INFO:teuthology.orchestra.run.smithi149.stdout: "req_symlink_latency": { 2024-04-24T09:25:16.870 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:16.870 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:16.870 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:16.870 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.870 INFO:teuthology.orchestra.run.smithi149.stdout: "req_unlink_latency": { 2024-04-24T09:25:16.870 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:16.870 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:16.870 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:16.871 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.871 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_revoke_eviction": 0, 2024-04-24T09:25:16.871 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_acquisition_throttle": 0, 2024-04-24T09:25:16.871 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getvxattr_latency": { 2024-04-24T09:25:16.871 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:16.871 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:16.871 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:16.871 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-04-24T09:25:16.872 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.872 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_sessions": { 2024-04-24T09:25:16.872 INFO:teuthology.orchestra.run.smithi149.stdout: "session_count": 1, 2024-04-24T09:25:16.872 INFO:teuthology.orchestra.run.smithi149.stdout: "session_add": 3, 2024-04-24T09:25:16.872 INFO:teuthology.orchestra.run.smithi149.stdout: "session_remove": 2, 2024-04-24T09:25:16.872 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_open": 2, 2024-04-24T09:25:16.872 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_stale": 0, 2024-04-24T09:25:16.872 INFO:teuthology.orchestra.run.smithi149.stdout: "total_load": 0, 2024-04-24T09:25:16.873 INFO:teuthology.orchestra.run.smithi149.stdout: "average_load": 0, 2024-04-24T09:25:16.873 INFO:teuthology.orchestra.run.smithi149.stdout: "avg_session_uptime": 3300, 2024-04-24T09:25:16.873 INFO:teuthology.orchestra.run.smithi149.stdout: "mdthresh_evicted": 0 2024-04-24T09:25:16.873 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.873 INFO:teuthology.orchestra.run.smithi149.stdout: "mempool": { 2024-04-24T09:25:16.873 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_bytes": 0, 2024-04-24T09:25:16.873 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_items": 0, 2024-04-24T09:25:16.873 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T09:25:16.873 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_items": 0, 2024-04-24T09:25:16.874 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T09:25:16.874 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_items": 0, 2024-04-24T09:25:16.874 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T09:25:16.874 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T09:25:16.874 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T09:25:16.874 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T09:25:16.874 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T09:25:16.874 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_items": 0, 2024-04-24T09:25:16.875 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T09:25:16.875 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T09:25:16.875 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_bytes": 0, 2024-04-24T09:25:16.875 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_items": 0, 2024-04-24T09:25:16.875 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_bytes": 0, 2024-04-24T09:25:16.875 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_items": 0, 2024-04-24T09:25:16.875 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T09:25:16.875 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T09:25:16.875 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T09:25:16.876 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T09:25:16.876 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T09:25:16.876 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_items": 0, 2024-04-24T09:25:16.876 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_bytes": 0, 2024-04-24T09:25:16.876 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_items": 0, 2024-04-24T09:25:16.876 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T09:25:16.876 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T09:25:16.876 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_bytes": 0, 2024-04-24T09:25:16.876 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_items": 0, 2024-04-24T09:25:16.877 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_bytes": 0, 2024-04-24T09:25:16.877 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_items": 0, 2024-04-24T09:25:16.877 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T09:25:16.877 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_items": 0, 2024-04-24T09:25:16.877 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T09:25:16.877 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_items": 0, 2024-04-24T09:25:16.877 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_bytes": 333407, 2024-04-24T09:25:16.877 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_items": 3497, 2024-04-24T09:25:16.877 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_bytes": 0, 2024-04-24T09:25:16.878 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_items": 0, 2024-04-24T09:25:16.878 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_bytes": 0, 2024-04-24T09:25:16.878 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_items": 0, 2024-04-24T09:25:16.878 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_bytes": 0, 2024-04-24T09:25:16.878 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_items": 0, 2024-04-24T09:25:16.878 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_bytes": 0, 2024-04-24T09:25:16.878 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_items": 0, 2024-04-24T09:25:16.878 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_bytes": 7624, 2024-04-24T09:25:16.878 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_items": 160, 2024-04-24T09:25:16.879 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T09:25:16.879 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_items": 0, 2024-04-24T09:25:16.879 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_bytes": 0, 2024-04-24T09:25:16.879 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_items": 0, 2024-04-24T09:25:16.879 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_bytes": 13768317, 2024-04-24T09:25:16.879 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_items": 115612, 2024-04-24T09:25:16.879 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_bytes": 0, 2024-04-24T09:25:16.880 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_items": 0, 2024-04-24T09:25:16.880 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_bytes": 0, 2024-04-24T09:25:16.880 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_items": 0 2024-04-24T09:25:16.880 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.880 INFO:teuthology.orchestra.run.smithi149.stdout: "objecter": { 2024-04-24T09:25:16.880 INFO:teuthology.orchestra.run.smithi149.stdout: "op_active": 0, 2024-04-24T09:25:16.880 INFO:teuthology.orchestra.run.smithi149.stdout: "op_laggy": 0, 2024-04-24T09:25:16.880 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send": 1197, 2024-04-24T09:25:16.881 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send_bytes": 26, 2024-04-24T09:25:16.881 INFO:teuthology.orchestra.run.smithi149.stdout: "op_resend": 4, 2024-04-24T09:25:16.881 INFO:teuthology.orchestra.run.smithi149.stdout: "op_reply": 1197, 2024-04-24T09:25:16.881 INFO:teuthology.orchestra.run.smithi149.stdout: "op_latency": { 2024-04-24T09:25:16.881 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1197, 2024-04-24T09:25:16.881 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.659986483, 2024-04-24T09:25:16.881 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000551367 2024-04-24T09:25:16.881 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.881 INFO:teuthology.orchestra.run.smithi149.stdout: "op_inflight": 0, 2024-04-24T09:25:16.882 INFO:teuthology.orchestra.run.smithi149.stdout: "oplen_avg": { 2024-04-24T09:25:16.882 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1197, 2024-04-24T09:25:16.882 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 1198 2024-04-24T09:25:16.882 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.882 INFO:teuthology.orchestra.run.smithi149.stdout: "op": 1197, 2024-04-24T09:25:16.882 INFO:teuthology.orchestra.run.smithi149.stdout: "op_r": 1197, 2024-04-24T09:25:16.882 INFO:teuthology.orchestra.run.smithi149.stdout: "op_w": 0, 2024-04-24T09:25:16.882 INFO:teuthology.orchestra.run.smithi149.stdout: "op_rmw": 0, 2024-04-24T09:25:16.883 INFO:teuthology.orchestra.run.smithi149.stdout: "op_pg": 0, 2024-04-24T09:25:16.883 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_stat": 745, 2024-04-24T09:25:16.883 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_create": 0, 2024-04-24T09:25:16.883 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_read": 449, 2024-04-24T09:25:16.883 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_write": 0, 2024-04-24T09:25:16.883 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writefull": 0, 2024-04-24T09:25:16.883 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writesame": 0, 2024-04-24T09:25:16.883 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_append": 0, 2024-04-24T09:25:16.883 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_zero": 0, 2024-04-24T09:25:16.884 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_truncate": 0, 2024-04-24T09:25:16.884 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_delete": 0, 2024-04-24T09:25:16.884 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_mapext": 0, 2024-04-24T09:25:16.884 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_sparse_read": 0, 2024-04-24T09:25:16.884 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_clonerange": 0, 2024-04-24T09:25:16.884 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_getxattr": 2, 2024-04-24T09:25:16.884 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_setxattr": 0, 2024-04-24T09:25:16.884 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_cmpxattr": 0, 2024-04-24T09:25:16.884 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_rmxattr": 0, 2024-04-24T09:25:16.885 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_resetxattrs": 0, 2024-04-24T09:25:16.885 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_call": 0, 2024-04-24T09:25:16.885 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_watch": 0, 2024-04-24T09:25:16.885 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_notify": 0, 2024-04-24T09:25:16.885 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T09:25:16.885 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls": 0, 2024-04-24T09:25:16.885 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls_filter": 0, 2024-04-24T09:25:16.885 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_other": 0, 2024-04-24T09:25:16.885 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_active": 0, 2024-04-24T09:25:16.886 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_send": 0, 2024-04-24T09:25:16.886 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_resend": 0, 2024-04-24T09:25:16.886 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_ping": 0, 2024-04-24T09:25:16.886 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_active": 0, 2024-04-24T09:25:16.886 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_send": 0, 2024-04-24T09:25:16.886 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_resend": 0, 2024-04-24T09:25:16.886 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_active": 0, 2024-04-24T09:25:16.887 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_send": 0, 2024-04-24T09:25:16.887 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_resend": 0, 2024-04-24T09:25:16.887 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_active": 0, 2024-04-24T09:25:16.887 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_send": 0, 2024-04-24T09:25:16.887 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_resend": 0, 2024-04-24T09:25:16.887 INFO:teuthology.orchestra.run.smithi149.stdout: "command_active": 0, 2024-04-24T09:25:16.887 INFO:teuthology.orchestra.run.smithi149.stdout: "command_send": 0, 2024-04-24T09:25:16.887 INFO:teuthology.orchestra.run.smithi149.stdout: "command_resend": 0, 2024-04-24T09:25:16.888 INFO:teuthology.orchestra.run.smithi149.stdout: "map_epoch": 88, 2024-04-24T09:25:16.888 INFO:teuthology.orchestra.run.smithi149.stdout: "map_full": 0, 2024-04-24T09:25:16.888 INFO:teuthology.orchestra.run.smithi149.stdout: "map_inc": 2, 2024-04-24T09:25:16.888 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_sessions": 7, 2024-04-24T09:25:16.888 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_open": 7, 2024-04-24T09:25:16.888 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_close": 0, 2024-04-24T09:25:16.888 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_laggy": 0, 2024-04-24T09:25:16.888 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_wr": 0, 2024-04-24T09:25:16.889 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_rd": 2, 2024-04-24T09:25:16.889 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_del": 0 2024-04-24T09:25:16.889 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.889 INFO:teuthology.orchestra.run.smithi149.stdout: "oft": { 2024-04-24T09:25:16.889 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_objs": 0, 2024-04-24T09:25:16.889 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_kv_pairs": 0, 2024-04-24T09:25:16.889 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_updates": 0, 2024-04-24T09:25:16.889 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_removes": 0 2024-04-24T09:25:16.890 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.890 INFO:teuthology.orchestra.run.smithi149.stdout: "osd-slow-ops": { 2024-04-24T09:25:16.890 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_ops_count": 0 2024-04-24T09:25:16.890 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.890 INFO:teuthology.orchestra.run.smithi149.stdout: "purge_queue": { 2024-04-24T09:25:16.890 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops": 0, 2024-04-24T09:25:16.890 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops_high_water": 0, 2024-04-24T09:25:16.890 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing": 0, 2024-04-24T09:25:16.891 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_high_water": 0, 2024-04-24T09:25:16.891 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed_ops": 0, 2024-04-24T09:25:16.891 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed": 0, 2024-04-24T09:25:16.891 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_item_in_journal": 0 2024-04-24T09:25:16.891 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.891 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T09:25:16.891 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-04-24T09:25:16.891 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-04-24T09:25:16.892 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-04-24T09:25:16.892 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 1326, 2024-04-24T09:25:16.892 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 47192143, 2024-04-24T09:25:16.892 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-04-24T09:25:16.892 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 1326, 2024-04-24T09:25:16.892 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-04-24T09:25:16.892 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-04-24T09:25:16.892 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 1326, 2024-04-24T09:25:16.892 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 47192143, 2024-04-24T09:25:16.892 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-04-24T09:25:16.893 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:16.893 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:16.893 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:16.893 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-04-24T09:25:16.893 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.893 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_bytes": { 2024-04-24T09:25:16.893 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-04-24T09:25:16.893 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-04-24T09:25:16.894 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-04-24T09:25:16.894 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-04-24T09:25:16.894 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-04-24T09:25:16.894 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-04-24T09:25:16.894 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-04-24T09:25:16.894 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 1197, 2024-04-24T09:25:16.894 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 46193634, 2024-04-24T09:25:16.894 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 1197, 2024-04-24T09:25:16.895 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 46193634, 2024-04-24T09:25:16.895 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-04-24T09:25:16.895 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:16.895 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:16.895 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:16.895 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-04-24T09:25:16.895 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.895 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_ops": { 2024-04-24T09:25:16.895 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-04-24T09:25:16.896 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 1024, 2024-04-24T09:25:16.896 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-04-24T09:25:16.896 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-04-24T09:25:16.896 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-04-24T09:25:16.896 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-04-24T09:25:16.896 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-04-24T09:25:16.896 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 1197, 2024-04-24T09:25:16.897 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 1197, 2024-04-24T09:25:16.897 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 1197, 2024-04-24T09:25:16.897 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 1197, 2024-04-24T09:25:16.897 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-04-24T09:25:16.897 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:16.897 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:16.897 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:16.897 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-04-24T09:25:16.898 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.898 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle": { 2024-04-24T09:25:16.898 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-04-24T09:25:16.898 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-04-24T09:25:16.898 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-04-24T09:25:16.898 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-04-24T09:25:16.898 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-04-24T09:25:16.898 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-04-24T09:25:16.899 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-04-24T09:25:16.899 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-04-24T09:25:16.899 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-04-24T09:25:16.899 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 0, 2024-04-24T09:25:16.899 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 0, 2024-04-24T09:25:16.899 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-04-24T09:25:16.899 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:16.900 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:16.900 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:16.900 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-04-24T09:25:16.900 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:16.900 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle-0x55a90fbf66e0": { 2024-04-24T09:25:16.900 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-04-24T09:25:16.900 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-04-24T09:25:16.900 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-04-24T09:25:16.901 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-04-24T09:25:16.901 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-04-24T09:25:16.901 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-04-24T09:25:16.901 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-04-24T09:25:16.901 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-04-24T09:25:16.901 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-04-24T09:25:16.901 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 0, 2024-04-24T09:25:16.901 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 0, 2024-04-24T09:25:16.902 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-04-24T09:25:16.902 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:16.902 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:16.902 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:16.902 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-04-24T09:25:16.902 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-04-24T09:25:16.902 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-04-24T09:25:17.651 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-24T09:25:17.651 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-24T09:25:17.651 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-04-24T09:25:17.651 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-04-24T09:25:17.651 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-04-24T09:25:17.651 DEBUG:tasks.check_counter:Getting stats from i 2024-04-24T09:25:17.652 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-24T09:25:17.899 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T09:25:19.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:18 smithi090 ceph-mon[27825]: pgmap v508: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-24T09:25:19.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:18 smithi149 ceph-mon[32592]: pgmap v508: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-24T09:25:19.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:18 smithi114 ceph-mon[32501]: pgmap v508: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-24T09:25:19.867 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-04-24T09:25:19.867 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T09:25:19.867 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 873, 2024-04-24T09:25:19.867 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 639, 2024-04-24T09:25:19.867 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 347449, 2024-04-24T09:25:19.867 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 325014, 2024-04-24T09:25:19.868 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 10, 2024-04-24T09:25:19.868 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 5, 2024-04-24T09:25:19.868 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.065552735, 2024-04-24T09:25:19.868 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.024492586, 2024-04-24T09:25:19.868 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.045616044, 2024-04-24T09:25:19.868 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.003592854, 2024-04-24T09:25:19.868 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:25:19.868 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 637, 2024-04-24T09:25:19.869 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.040594136, 2024-04-24T09:25:19.869 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000063727 2024-04-24T09:25:19.869 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.869 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:25:19.869 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1366, 2024-04-24T09:25:19.869 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000281288, 2024-04-24T09:25:19.869 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000205 2024-04-24T09:25:19.869 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.869 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T09:25:19.870 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T09:25:19.870 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.870 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T09:25:19.870 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 192, 2024-04-24T09:25:19.870 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 306, 2024-04-24T09:25:19.870 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 76700, 2024-04-24T09:25:19.870 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 377024, 2024-04-24T09:25:19.870 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 15, 2024-04-24T09:25:19.871 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 9, 2024-04-24T09:25:19.871 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.026377470, 2024-04-24T09:25:19.871 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.010316491, 2024-04-24T09:25:19.871 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.018991236, 2024-04-24T09:25:19.871 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.000299599, 2024-04-24T09:25:19.871 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:25:19.871 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 304, 2024-04-24T09:25:19.871 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.021673539, 2024-04-24T09:25:19.871 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000071294 2024-04-24T09:25:19.872 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.872 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:25:19.872 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 348, 2024-04-24T09:25:19.872 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000062136, 2024-04-24T09:25:19.872 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000178 2024-04-24T09:25:19.872 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.872 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 57632, 2024-04-24T09:25:19.873 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 306800 2024-04-24T09:25:19.873 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.873 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T09:25:19.873 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 201, 2024-04-24T09:25:19.873 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 182, 2024-04-24T09:25:19.873 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 831336, 2024-04-24T09:25:19.873 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 108684, 2024-04-24T09:25:19.873 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 7, 2024-04-24T09:25:19.873 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 6, 2024-04-24T09:25:19.874 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.027189692, 2024-04-24T09:25:19.874 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.008418007, 2024-04-24T09:25:19.874 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.021141557, 2024-04-24T09:25:19.874 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.003432145, 2024-04-24T09:25:19.874 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:25:19.874 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 182, 2024-04-24T09:25:19.874 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.019304458, 2024-04-24T09:25:19.874 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000106068 2024-04-24T09:25:19.875 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.875 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:25:19.875 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 18, 2024-04-24T09:25:19.875 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000003506, 2024-04-24T09:25:19.875 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000194 2024-04-24T09:25:19.875 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.875 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 808384, 2024-04-24T09:25:19.875 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 71280 2024-04-24T09:25:19.875 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.876 INFO:teuthology.orchestra.run.smithi149.stdout: "cct": { 2024-04-24T09:25:19.876 INFO:teuthology.orchestra.run.smithi149.stdout: "total_workers": 1, 2024-04-24T09:25:19.876 INFO:teuthology.orchestra.run.smithi149.stdout: "unhealthy_workers": 0 2024-04-24T09:25:19.876 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.876 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-MDSRank": { 2024-04-24T09:25:19.876 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-04-24T09:25:19.876 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-04-24T09:25:19.876 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 117, 2024-04-24T09:25:19.876 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.015937839, 2024-04-24T09:25:19.877 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000136220 2024-04-24T09:25:19.877 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-04-24T09:25:19.877 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.877 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-PurgeQueue": { 2024-04-24T09:25:19.877 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-04-24T09:25:19.877 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-04-24T09:25:19.877 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 22, 2024-04-24T09:25:19.877 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000765376, 2024-04-24T09:25:19.877 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000034789 2024-04-24T09:25:19.878 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-04-24T09:25:19.878 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.878 INFO:teuthology.orchestra.run.smithi149.stdout: "mds": { 2024-04-24T09:25:19.878 INFO:teuthology.orchestra.run.smithi149.stdout: "request": 116, 2024-04-24T09:25:19.878 INFO:teuthology.orchestra.run.smithi149.stdout: "reply": 41, 2024-04-24T09:25:19.878 INFO:teuthology.orchestra.run.smithi149.stdout: "reply_latency": { 2024-04-24T09:25:19.878 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 41, 2024-04-24T09:25:19.878 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.037215641, 2024-04-24T09:25:19.878 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000907698 2024-04-24T09:25:19.878 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.879 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_reply": 0, 2024-04-24T09:25:19.879 INFO:teuthology.orchestra.run.smithi149.stdout: "forward": 75, 2024-04-24T09:25:19.879 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_complete": 0, 2024-04-24T09:25:19.879 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_keys": 0, 2024-04-24T09:25:19.879 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_commit": 11, 2024-04-24T09:25:19.879 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_split": 0, 2024-04-24T09:25:19.879 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_merge": 0, 2024-04-24T09:25:19.879 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes": 65, 2024-04-24T09:25:19.880 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_top": 9, 2024-04-24T09:25:19.880 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_bottom": 56, 2024-04-24T09:25:19.880 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pin_tail": 0, 2024-04-24T09:25:19.880 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pinned": 50, 2024-04-24T09:25:19.880 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_expired": 6, 2024-04-24T09:25:19.880 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_with_caps": 4, 2024-04-24T09:25:19.880 INFO:teuthology.orchestra.run.smithi149.stdout: "caps": 4, 2024-04-24T09:25:19.880 INFO:teuthology.orchestra.run.smithi149.stdout: "subtrees": 6, 2024-04-24T09:25:19.881 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse": 199, 2024-04-24T09:25:19.881 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_hit": 75, 2024-04-24T09:25:19.881 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_forward": 74, 2024-04-24T09:25:19.881 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_discover": 19, 2024-04-24T09:25:19.881 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_dir_fetch": 0, 2024-04-24T09:25:19.881 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_remote_ino": 0, 2024-04-24T09:25:19.881 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_lock": 0, 2024-04-24T09:25:19.881 INFO:teuthology.orchestra.run.smithi149.stdout: "load_cent": 0, 2024-04-24T09:25:19.881 INFO:teuthology.orchestra.run.smithi149.stdout: "q": 0, 2024-04-24T09:25:19.882 INFO:teuthology.orchestra.run.smithi149.stdout: "exported": 0, 2024-04-24T09:25:19.882 INFO:teuthology.orchestra.run.smithi149.stdout: "exported_inodes": 0, 2024-04-24T09:25:19.882 INFO:teuthology.orchestra.run.smithi149.stdout: "imported": 3, 2024-04-24T09:25:19.882 INFO:teuthology.orchestra.run.smithi149.stdout: "imported_inodes": 24, 2024-04-24T09:25:19.882 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_dir_fetch": 0, 2024-04-24T09:25:19.882 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_backtrace_fetch": 0, 2024-04-24T09:25:19.882 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_peer_discover": 0, 2024-04-24T09:25:19.882 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rfiles": 4, 2024-04-24T09:25:19.883 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rbytes": 234, 2024-04-24T09:25:19.883 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rsnaps": 0, 2024-04-24T09:25:19.883 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_fetch": 0, 2024-04-24T09:25:19.883 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_set_tag": 0, 2024-04-24T09:25:19.883 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T09:25:19.883 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_inotable_repaired": 0, 2024-04-24T09:25:19.883 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_inodes": 0, 2024-04-24T09:25:19.883 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T09:25:19.883 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dirfrag_rstats": 0, 2024-04-24T09:25:19.884 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_file_inodes": 0, 2024-04-24T09:25:19.884 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_inode_file_caps": 0, 2024-04-24T09:25:19.884 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_revoke": 1, 2024-04-24T09:25:19.884 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_grant": 17, 2024-04-24T09:25:19.884 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T09:25:19.884 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T09:25:19.884 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T09:25:19.884 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps": 32, 2024-04-24T09:25:19.885 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps_dirty": 24, 2024-04-24T09:25:19.885 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_cap_release": 8, 2024-04-24T09:25:19.885 INFO:teuthology.orchestra.run.smithi149.stdout: "process_request_cap_release": 20 2024-04-24T09:25:19.885 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.885 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_cache": { 2024-04-24T09:25:19.885 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update": 12, 2024-04-24T09:25:19.885 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update_receipt": 66, 2024-04-24T09:25:19.885 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_try_discover": 41, 2024-04-24T09:25:19.885 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_send_discover": 47, 2024-04-24T09:25:19.886 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_handle_discover": 12, 2024-04-24T09:25:19.886 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays": 0, 2024-04-24T09:25:19.886 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_delayed": 0, 2024-04-24T09:25:19.886 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_enqueuing": 0, 2024-04-24T09:25:19.886 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_created": 7, 2024-04-24T09:25:19.886 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_enqueued": 7, 2024-04-24T09:25:19.886 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_reintegrated": 0, 2024-04-24T09:25:19.886 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_migrated": 0, 2024-04-24T09:25:19.886 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_processing": 0, 2024-04-24T09:25:19.887 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_enqueued": 0, 2024-04-24T09:25:19.887 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_prioritized": 0, 2024-04-24T09:25:19.887 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_started": 0, 2024-04-24T09:25:19.887 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_completed": 0, 2024-04-24T09:25:19.887 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_quiesce_path": 0, 2024-04-24T09:25:19.887 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_quiesce_inode": 0, 2024-04-24T09:25:19.887 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T09:25:19.887 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_exportdir": 0, 2024-04-24T09:25:19.888 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_flush": 0, 2024-04-24T09:25:19.888 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragmentdir": 0, 2024-04-24T09:25:19.888 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragstats": 0, 2024-04-24T09:25:19.888 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_inodestats": 0 2024-04-24T09:25:19.888 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.888 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_log": { 2024-04-24T09:25:19.888 INFO:teuthology.orchestra.run.smithi149.stdout: "evlrg": 0, 2024-04-24T09:25:19.888 INFO:teuthology.orchestra.run.smithi149.stdout: "evadd": 73, 2024-04-24T09:25:19.888 INFO:teuthology.orchestra.run.smithi149.stdout: "evex": 0, 2024-04-24T09:25:19.889 INFO:teuthology.orchestra.run.smithi149.stdout: "evtrm": 0, 2024-04-24T09:25:19.889 INFO:teuthology.orchestra.run.smithi149.stdout: "ev": 73, 2024-04-24T09:25:19.889 INFO:teuthology.orchestra.run.smithi149.stdout: "evexg": 0, 2024-04-24T09:25:19.889 INFO:teuthology.orchestra.run.smithi149.stdout: "evexd": 0, 2024-04-24T09:25:19.889 INFO:teuthology.orchestra.run.smithi149.stdout: "segadd": 2, 2024-04-24T09:25:19.889 INFO:teuthology.orchestra.run.smithi149.stdout: "segex": 0, 2024-04-24T09:25:19.889 INFO:teuthology.orchestra.run.smithi149.stdout: "segtrm": 0, 2024-04-24T09:25:19.889 INFO:teuthology.orchestra.run.smithi149.stdout: "seg": 2, 2024-04-24T09:25:19.889 INFO:teuthology.orchestra.run.smithi149.stdout: "segmjr": 2, 2024-04-24T09:25:19.890 INFO:teuthology.orchestra.run.smithi149.stdout: "segexg": 0, 2024-04-24T09:25:19.890 INFO:teuthology.orchestra.run.smithi149.stdout: "segexd": 0, 2024-04-24T09:25:19.890 INFO:teuthology.orchestra.run.smithi149.stdout: "expos": 4194304, 2024-04-24T09:25:19.890 INFO:teuthology.orchestra.run.smithi149.stdout: "wrpos": 4317718, 2024-04-24T09:25:19.890 INFO:teuthology.orchestra.run.smithi149.stdout: "rdpos": 0, 2024-04-24T09:25:19.890 INFO:teuthology.orchestra.run.smithi149.stdout: "jlat": { 2024-04-24T09:25:19.890 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 26, 2024-04-24T09:25:19.890 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.047092022, 2024-04-24T09:25:19.891 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.001811231 2024-04-24T09:25:19.891 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.891 INFO:teuthology.orchestra.run.smithi149.stdout: "replayed": 0 2024-04-24T09:25:19.891 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.891 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_mem": { 2024-04-24T09:25:19.891 INFO:teuthology.orchestra.run.smithi149.stdout: "ino": 63, 2024-04-24T09:25:19.891 INFO:teuthology.orchestra.run.smithi149.stdout: "ino+": 73, 2024-04-24T09:25:19.891 INFO:teuthology.orchestra.run.smithi149.stdout: "ino-": 10, 2024-04-24T09:25:19.892 INFO:teuthology.orchestra.run.smithi149.stdout: "dir": 49, 2024-04-24T09:25:19.892 INFO:teuthology.orchestra.run.smithi149.stdout: "dir+": 56, 2024-04-24T09:25:19.892 INFO:teuthology.orchestra.run.smithi149.stdout: "dir-": 7, 2024-04-24T09:25:19.892 INFO:teuthology.orchestra.run.smithi149.stdout: "dn": 65, 2024-04-24T09:25:19.892 INFO:teuthology.orchestra.run.smithi149.stdout: "dn+": 83, 2024-04-24T09:25:19.892 INFO:teuthology.orchestra.run.smithi149.stdout: "dn-": 18, 2024-04-24T09:25:19.892 INFO:teuthology.orchestra.run.smithi149.stdout: "cap": 4, 2024-04-24T09:25:19.892 INFO:teuthology.orchestra.run.smithi149.stdout: "cap+": 31, 2024-04-24T09:25:19.892 INFO:teuthology.orchestra.run.smithi149.stdout: "cap-": 27, 2024-04-24T09:25:19.893 INFO:teuthology.orchestra.run.smithi149.stdout: "rss": 53628, 2024-04-24T09:25:19.893 INFO:teuthology.orchestra.run.smithi149.stdout: "heap": 223608 2024-04-24T09:25:19.893 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.893 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_server": { 2024-04-24T09:25:19.893 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_client_request": 135, 2024-04-24T09:25:19.893 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_server_request": 0, 2024-04-24T09:25:19.893 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_request": 116, 2024-04-24T09:25:19.893 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_session": 6, 2024-04-24T09:25:19.893 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_peer_request": 13, 2024-04-24T09:25:19.894 INFO:teuthology.orchestra.run.smithi149.stdout: "req_create_latency": { 2024-04-24T09:25:19.894 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 4, 2024-04-24T09:25:19.894 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.009060468, 2024-04-24T09:25:19.894 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.002265117 2024-04-24T09:25:19.894 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.894 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getattr_latency": { 2024-04-24T09:25:19.894 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1, 2024-04-24T09:25:19.894 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.001686570, 2024-04-24T09:25:19.894 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.001686570 2024-04-24T09:25:19.895 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.895 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getfilelock_latency": { 2024-04-24T09:25:19.895 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:19.895 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:19.895 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:19.895 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.895 INFO:teuthology.orchestra.run.smithi149.stdout: "req_link_latency": { 2024-04-24T09:25:19.895 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:19.896 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:19.896 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:19.896 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.896 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookup_latency": { 2024-04-24T09:25:19.896 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 23, 2024-04-24T09:25:19.896 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.007159519, 2024-04-24T09:25:19.896 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000311283 2024-04-24T09:25:19.896 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.896 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookuphash_latency": { 2024-04-24T09:25:19.897 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:19.897 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:19.897 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:19.897 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.897 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupino_latency": { 2024-04-24T09:25:19.897 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:19.897 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:19.897 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:19.897 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.898 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupname_latency": { 2024-04-24T09:25:19.898 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:19.898 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:19.898 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:19.898 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.898 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupparent_latency": { 2024-04-24T09:25:19.898 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:19.899 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:19.899 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:19.899 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.899 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupsnap_latency": { 2024-04-24T09:25:19.899 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:19.899 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:19.899 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:19.899 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.899 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lssnap_latency": { 2024-04-24T09:25:19.900 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:19.900 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:19.900 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:19.900 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.900 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mkdir_latency": { 2024-04-24T09:25:19.901 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:19.901 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:19.901 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:19.901 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.901 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mknod_latency": { 2024-04-24T09:25:19.901 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:19.902 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:19.902 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:19.902 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.902 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mksnap_latency": { 2024-04-24T09:25:19.902 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:19.902 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:19.902 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:19.902 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.903 INFO:teuthology.orchestra.run.smithi149.stdout: "req_open_latency": { 2024-04-24T09:25:19.903 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 2, 2024-04-24T09:25:19.903 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000620824, 2024-04-24T09:25:19.903 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000310412 2024-04-24T09:25:19.903 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.903 INFO:teuthology.orchestra.run.smithi149.stdout: "req_readdir_latency": { 2024-04-24T09:25:19.903 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 3, 2024-04-24T09:25:19.903 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.001738142, 2024-04-24T09:25:19.903 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000579380 2024-04-24T09:25:19.904 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.904 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rename_latency": { 2024-04-24T09:25:19.904 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:19.904 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:19.904 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:19.904 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.904 INFO:teuthology.orchestra.run.smithi149.stdout: "req_renamesnap_latency": { 2024-04-24T09:25:19.904 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:19.904 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:19.904 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:19.905 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.905 INFO:teuthology.orchestra.run.smithi149.stdout: "req_snapdiff_latency": { 2024-04-24T09:25:19.905 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:19.905 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:19.905 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:19.905 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.905 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmdir_latency": { 2024-04-24T09:25:19.905 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:19.906 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:19.906 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:19.906 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.906 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmsnap_latency": { 2024-04-24T09:25:19.906 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:19.906 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:19.906 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:19.906 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.906 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmxattr_latency": { 2024-04-24T09:25:19.907 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:19.907 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:19.907 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:19.907 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.907 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setattr_latency": { 2024-04-24T09:25:19.907 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:19.907 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:19.907 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:19.907 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.908 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setdirlayout_latency": { 2024-04-24T09:25:19.908 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:19.908 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:19.908 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:19.908 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.908 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setfilelock_latency": { 2024-04-24T09:25:19.908 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:19.908 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:19.908 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:19.909 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.909 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setlayout_latency": { 2024-04-24T09:25:19.909 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:19.909 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:19.909 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:19.909 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.909 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setxattr_latency": { 2024-04-24T09:25:19.909 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:19.910 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:19.910 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:19.910 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.910 INFO:teuthology.orchestra.run.smithi149.stdout: "req_symlink_latency": { 2024-04-24T09:25:19.910 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1, 2024-04-24T09:25:19.910 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000884197, 2024-04-24T09:25:19.910 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000884197 2024-04-24T09:25:19.910 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.910 INFO:teuthology.orchestra.run.smithi149.stdout: "req_unlink_latency": { 2024-04-24T09:25:19.911 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 7, 2024-04-24T09:25:19.911 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.016065921, 2024-04-24T09:25:19.911 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.002295131 2024-04-24T09:25:19.911 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.911 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_revoke_eviction": 0, 2024-04-24T09:25:19.911 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_acquisition_throttle": 0, 2024-04-24T09:25:19.911 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getvxattr_latency": { 2024-04-24T09:25:19.911 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:19.911 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:19.912 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:19.912 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-04-24T09:25:19.912 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.912 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_sessions": { 2024-04-24T09:25:19.912 INFO:teuthology.orchestra.run.smithi149.stdout: "session_count": 1, 2024-04-24T09:25:19.912 INFO:teuthology.orchestra.run.smithi149.stdout: "session_add": 2, 2024-04-24T09:25:19.912 INFO:teuthology.orchestra.run.smithi149.stdout: "session_remove": 1, 2024-04-24T09:25:19.912 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_open": 1, 2024-04-24T09:25:19.912 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_stale": 0, 2024-04-24T09:25:19.913 INFO:teuthology.orchestra.run.smithi149.stdout: "total_load": 29, 2024-04-24T09:25:19.913 INFO:teuthology.orchestra.run.smithi149.stdout: "average_load": 29, 2024-04-24T09:25:19.913 INFO:teuthology.orchestra.run.smithi149.stdout: "avg_session_uptime": 106, 2024-04-24T09:25:19.913 INFO:teuthology.orchestra.run.smithi149.stdout: "mdthresh_evicted": 0 2024-04-24T09:25:19.913 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.913 INFO:teuthology.orchestra.run.smithi149.stdout: "mempool": { 2024-04-24T09:25:19.913 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_bytes": 0, 2024-04-24T09:25:19.913 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_items": 0, 2024-04-24T09:25:19.914 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T09:25:19.914 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_items": 0, 2024-04-24T09:25:19.914 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T09:25:19.914 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_items": 0, 2024-04-24T09:25:19.914 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T09:25:19.914 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T09:25:19.914 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T09:25:19.914 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T09:25:19.914 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T09:25:19.915 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_items": 0, 2024-04-24T09:25:19.915 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T09:25:19.915 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T09:25:19.915 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_bytes": 0, 2024-04-24T09:25:19.915 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_items": 0, 2024-04-24T09:25:19.915 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_bytes": 0, 2024-04-24T09:25:19.915 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_items": 0, 2024-04-24T09:25:19.915 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T09:25:19.915 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T09:25:19.916 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T09:25:19.916 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T09:25:19.916 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T09:25:19.916 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_items": 0, 2024-04-24T09:25:19.916 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_bytes": 0, 2024-04-24T09:25:19.916 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_items": 0, 2024-04-24T09:25:19.916 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T09:25:19.916 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T09:25:19.916 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_bytes": 0, 2024-04-24T09:25:19.917 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_items": 0, 2024-04-24T09:25:19.917 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_bytes": 0, 2024-04-24T09:25:19.917 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_items": 0, 2024-04-24T09:25:19.917 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T09:25:19.917 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_items": 0, 2024-04-24T09:25:19.917 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T09:25:19.917 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_items": 0, 2024-04-24T09:25:19.917 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_bytes": 310937, 2024-04-24T09:25:19.918 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_items": 177, 2024-04-24T09:25:19.918 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_bytes": 440, 2024-04-24T09:25:19.918 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_items": 5, 2024-04-24T09:25:19.918 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_bytes": 0, 2024-04-24T09:25:19.918 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_items": 0, 2024-04-24T09:25:19.918 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_bytes": 0, 2024-04-24T09:25:19.918 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_items": 0, 2024-04-24T09:25:19.918 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_bytes": 0, 2024-04-24T09:25:19.918 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_items": 0, 2024-04-24T09:25:19.919 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_bytes": 7624, 2024-04-24T09:25:19.919 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_items": 160, 2024-04-24T09:25:19.919 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T09:25:19.919 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_items": 0, 2024-04-24T09:25:19.919 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_bytes": 0, 2024-04-24T09:25:19.919 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_items": 0, 2024-04-24T09:25:19.919 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_bytes": 259486, 2024-04-24T09:25:19.919 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_items": 2011, 2024-04-24T09:25:19.919 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_bytes": 0, 2024-04-24T09:25:19.920 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_items": 0, 2024-04-24T09:25:19.920 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_bytes": 0, 2024-04-24T09:25:19.920 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_items": 0 2024-04-24T09:25:19.920 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.920 INFO:teuthology.orchestra.run.smithi149.stdout: "objecter": { 2024-04-24T09:25:19.920 INFO:teuthology.orchestra.run.smithi149.stdout: "op_active": 0, 2024-04-24T09:25:19.920 INFO:teuthology.orchestra.run.smithi149.stdout: "op_laggy": 0, 2024-04-24T09:25:19.920 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send": 98, 2024-04-24T09:25:19.920 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send_bytes": 139691, 2024-04-24T09:25:19.921 INFO:teuthology.orchestra.run.smithi149.stdout: "op_resend": 31, 2024-04-24T09:25:19.921 INFO:teuthology.orchestra.run.smithi149.stdout: "op_reply": 98, 2024-04-24T09:25:19.921 INFO:teuthology.orchestra.run.smithi149.stdout: "op_latency": { 2024-04-24T09:25:19.921 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 98, 2024-04-24T09:25:19.921 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.425991277, 2024-04-24T09:25:19.921 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.004346849 2024-04-24T09:25:19.921 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.921 INFO:teuthology.orchestra.run.smithi149.stdout: "op_inflight": 0, 2024-04-24T09:25:19.922 INFO:teuthology.orchestra.run.smithi149.stdout: "oplen_avg": { 2024-04-24T09:25:19.922 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 98, 2024-04-24T09:25:19.922 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 134 2024-04-24T09:25:19.922 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.922 INFO:teuthology.orchestra.run.smithi149.stdout: "op": 98, 2024-04-24T09:25:19.922 INFO:teuthology.orchestra.run.smithi149.stdout: "op_r": 1, 2024-04-24T09:25:19.922 INFO:teuthology.orchestra.run.smithi149.stdout: "op_w": 97, 2024-04-24T09:25:19.923 INFO:teuthology.orchestra.run.smithi149.stdout: "op_rmw": 0, 2024-04-24T09:25:19.923 INFO:teuthology.orchestra.run.smithi149.stdout: "op_pg": 0, 2024-04-24T09:25:19.923 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_stat": 0, 2024-04-24T09:25:19.923 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_create": 10, 2024-04-24T09:25:19.923 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_read": 1, 2024-04-24T09:25:19.923 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_write": 27, 2024-04-24T09:25:19.923 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writefull": 14, 2024-04-24T09:25:19.923 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writesame": 0, 2024-04-24T09:25:19.923 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_append": 0, 2024-04-24T09:25:19.924 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_zero": 0, 2024-04-24T09:25:19.924 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_truncate": 0, 2024-04-24T09:25:19.924 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_delete": 19, 2024-04-24T09:25:19.924 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_mapext": 0, 2024-04-24T09:25:19.924 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_sparse_read": 0, 2024-04-24T09:25:19.924 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_clonerange": 0, 2024-04-24T09:25:19.924 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_getxattr": 0, 2024-04-24T09:25:19.924 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_setxattr": 20, 2024-04-24T09:25:19.925 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_cmpxattr": 0, 2024-04-24T09:25:19.925 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_rmxattr": 0, 2024-04-24T09:25:19.925 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_resetxattrs": 0, 2024-04-24T09:25:19.925 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_call": 0, 2024-04-24T09:25:19.925 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_watch": 0, 2024-04-24T09:25:19.925 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_notify": 0, 2024-04-24T09:25:19.925 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T09:25:19.925 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls": 0, 2024-04-24T09:25:19.925 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls_filter": 0, 2024-04-24T09:25:19.926 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_other": 0, 2024-04-24T09:25:19.926 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_active": 0, 2024-04-24T09:25:19.926 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_send": 0, 2024-04-24T09:25:19.926 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_resend": 0, 2024-04-24T09:25:19.926 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_ping": 0, 2024-04-24T09:25:19.926 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_active": 0, 2024-04-24T09:25:19.926 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_send": 0, 2024-04-24T09:25:19.926 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_resend": 0, 2024-04-24T09:25:19.926 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_active": 0, 2024-04-24T09:25:19.927 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_send": 0, 2024-04-24T09:25:19.927 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_resend": 0, 2024-04-24T09:25:19.927 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_active": 0, 2024-04-24T09:25:19.927 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_send": 0, 2024-04-24T09:25:19.927 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_resend": 0, 2024-04-24T09:25:19.927 INFO:teuthology.orchestra.run.smithi149.stdout: "command_active": 0, 2024-04-24T09:25:19.927 INFO:teuthology.orchestra.run.smithi149.stdout: "command_send": 0, 2024-04-24T09:25:19.927 INFO:teuthology.orchestra.run.smithi149.stdout: "command_resend": 0, 2024-04-24T09:25:19.927 INFO:teuthology.orchestra.run.smithi149.stdout: "map_epoch": 88, 2024-04-24T09:25:19.928 INFO:teuthology.orchestra.run.smithi149.stdout: "map_full": 0, 2024-04-24T09:25:19.928 INFO:teuthology.orchestra.run.smithi149.stdout: "map_inc": 2, 2024-04-24T09:25:19.928 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_sessions": 12, 2024-04-24T09:25:19.928 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_open": 12, 2024-04-24T09:25:19.928 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_close": 0, 2024-04-24T09:25:19.928 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_laggy": 0, 2024-04-24T09:25:19.928 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_wr": 37, 2024-04-24T09:25:19.928 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_rd": 0, 2024-04-24T09:25:19.928 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_del": 6 2024-04-24T09:25:19.929 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.929 INFO:teuthology.orchestra.run.smithi149.stdout: "oft": { 2024-04-24T09:25:19.929 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_objs": 1, 2024-04-24T09:25:19.929 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_kv_pairs": 0, 2024-04-24T09:25:19.929 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_updates": 0, 2024-04-24T09:25:19.929 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_removes": 0 2024-04-24T09:25:19.929 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.929 INFO:teuthology.orchestra.run.smithi149.stdout: "osd-slow-ops": { 2024-04-24T09:25:19.930 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_ops_count": 0 2024-04-24T09:25:19.930 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.930 INFO:teuthology.orchestra.run.smithi149.stdout: "purge_queue": { 2024-04-24T09:25:19.930 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops": 0, 2024-04-24T09:25:19.930 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops_high_water": 6, 2024-04-24T09:25:19.930 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing": 0, 2024-04-24T09:25:19.930 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_high_water": 6, 2024-04-24T09:25:19.930 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed_ops": 7, 2024-04-24T09:25:19.930 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed": 7, 2024-04-24T09:25:19.931 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_item_in_journal": 0 2024-04-24T09:25:19.931 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.931 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T09:25:19.931 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-04-24T09:25:19.931 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-04-24T09:25:19.931 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-04-24T09:25:19.931 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 1262, 2024-04-24T09:25:19.931 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 1123752, 2024-04-24T09:25:19.931 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-04-24T09:25:19.932 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 1262, 2024-04-24T09:25:19.932 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-04-24T09:25:19.932 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-04-24T09:25:19.932 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 1262, 2024-04-24T09:25:19.932 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 1123752, 2024-04-24T09:25:19.932 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-04-24T09:25:19.932 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:19.932 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:19.932 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:19.933 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-04-24T09:25:19.933 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.933 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_bytes": { 2024-04-24T09:25:19.933 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-04-24T09:25:19.933 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-04-24T09:25:19.933 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-04-24T09:25:19.933 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-04-24T09:25:19.933 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-04-24T09:25:19.933 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-04-24T09:25:19.934 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-04-24T09:25:19.934 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 98, 2024-04-24T09:25:19.934 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 140398, 2024-04-24T09:25:19.934 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 98, 2024-04-24T09:25:19.934 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 140398, 2024-04-24T09:25:19.934 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-04-24T09:25:19.934 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:19.935 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:19.935 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:19.935 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-04-24T09:25:19.935 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.935 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_ops": { 2024-04-24T09:25:19.935 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-04-24T09:25:19.935 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 1024, 2024-04-24T09:25:19.935 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-04-24T09:25:19.935 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-04-24T09:25:19.936 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-04-24T09:25:19.936 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-04-24T09:25:19.936 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-04-24T09:25:19.936 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 98, 2024-04-24T09:25:19.936 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 98, 2024-04-24T09:25:19.936 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 98, 2024-04-24T09:25:19.937 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 98, 2024-04-24T09:25:19.937 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-04-24T09:25:19.937 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:19.937 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:19.937 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:19.937 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-04-24T09:25:19.937 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.937 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle": { 2024-04-24T09:25:19.938 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-04-24T09:25:19.938 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-04-24T09:25:19.938 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-04-24T09:25:19.938 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 7, 2024-04-24T09:25:19.938 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 707, 2024-04-24T09:25:19.938 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-04-24T09:25:19.938 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 7, 2024-04-24T09:25:19.938 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-04-24T09:25:19.938 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-04-24T09:25:19.939 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 1, 2024-04-24T09:25:19.939 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 707, 2024-04-24T09:25:19.939 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-04-24T09:25:19.939 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:19.939 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:19.939 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:19.939 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-04-24T09:25:19.939 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:19.939 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle-0x5625fc2ac500": { 2024-04-24T09:25:19.940 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-04-24T09:25:19.940 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-04-24T09:25:19.940 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-04-24T09:25:19.940 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 73, 2024-04-24T09:25:19.940 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 123414, 2024-04-24T09:25:19.940 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-04-24T09:25:19.940 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 73, 2024-04-24T09:25:19.940 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-04-24T09:25:19.941 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-04-24T09:25:19.941 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 26, 2024-04-24T09:25:19.941 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 123414, 2024-04-24T09:25:19.941 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-04-24T09:25:19.941 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:19.941 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:19.941 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:19.941 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-04-24T09:25:19.941 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-04-24T09:25:19.942 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-04-24T09:25:20.624 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-04-24T09:25:20.624 INFO:tasks.check_counter:Daemon mds.i mds.imported=3 2024-04-24T09:25:20.624 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=12 2024-04-24T09:25:20.625 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=66 2024-04-24T09:25:20.625 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=0 2024-04-24T09:25:20.625 DEBUG:tasks.check_counter:Getting stats from l 2024-04-24T09:25:20.625 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-24T09:25:20.864 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T09:25:21.085 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:20 smithi149 ceph-mon[32592]: pgmap v509: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-24T09:25:21.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:20 smithi090 ceph-mon[27825]: pgmap v509: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-24T09:25:21.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:20 smithi114 ceph-mon[32501]: pgmap v509: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-24T09:25:22.583 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-04-24T09:25:22.583 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T09:25:22.583 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 410, 2024-04-24T09:25:22.583 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 412, 2024-04-24T09:25:22.584 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 135095, 2024-04-24T09:25:22.584 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 130381, 2024-04-24T09:25:22.584 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 6, 2024-04-24T09:25:22.584 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 2, 2024-04-24T09:25:22.584 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.051429477, 2024-04-24T09:25:22.584 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.016036322, 2024-04-24T09:25:22.584 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.013700950, 2024-04-24T09:25:22.584 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.015902263, 2024-04-24T09:25:22.585 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:25:22.585 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 410, 2024-04-24T09:25:22.585 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.023568240, 2024-04-24T09:25:22.585 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000057483 2024-04-24T09:25:22.585 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.585 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:25:22.585 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:22.585 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:22.585 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:22.586 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.586 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T09:25:22.586 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T09:25:22.586 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.586 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T09:25:22.586 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 217, 2024-04-24T09:25:22.586 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 303, 2024-04-24T09:25:22.586 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 53862, 2024-04-24T09:25:22.587 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 367574, 2024-04-24T09:25:22.587 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 6, 2024-04-24T09:25:22.587 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 4, 2024-04-24T09:25:22.587 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.026299468, 2024-04-24T09:25:22.587 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.010893944, 2024-04-24T09:25:22.587 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.005690325, 2024-04-24T09:25:22.587 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.007014840, 2024-04-24T09:25:22.587 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:25:22.587 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 300, 2024-04-24T09:25:22.588 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.017028404, 2024-04-24T09:25:22.588 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000056761 2024-04-24T09:25:22.588 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.588 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:25:22.588 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:22.588 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:22.588 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:22.588 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.589 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-24T09:25:22.589 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 299136 2024-04-24T09:25:22.589 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.589 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T09:25:22.589 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 475, 2024-04-24T09:25:22.589 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 460, 2024-04-24T09:25:22.589 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 953260, 2024-04-24T09:25:22.589 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 178124, 2024-04-24T09:25:22.589 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 3, 2024-04-24T09:25:22.590 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 2, 2024-04-24T09:25:22.590 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.055481469, 2024-04-24T09:25:22.590 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.017763598, 2024-04-24T09:25:22.590 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.028118561, 2024-04-24T09:25:22.590 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.011820313, 2024-04-24T09:25:22.590 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:25:22.590 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 460, 2024-04-24T09:25:22.590 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.022776429, 2024-04-24T09:25:22.590 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000049513 2024-04-24T09:25:22.591 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.591 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:25:22.591 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:22.591 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:22.591 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:22.591 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.591 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 793168, 2024-04-24T09:25:22.591 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 70640 2024-04-24T09:25:22.592 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.592 INFO:teuthology.orchestra.run.smithi149.stdout: "cct": { 2024-04-24T09:25:22.592 INFO:teuthology.orchestra.run.smithi149.stdout: "total_workers": 1, 2024-04-24T09:25:22.592 INFO:teuthology.orchestra.run.smithi149.stdout: "unhealthy_workers": 0 2024-04-24T09:25:22.592 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.592 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-MDSRank": { 2024-04-24T09:25:22.592 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-04-24T09:25:22.592 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-04-24T09:25:22.592 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1695, 2024-04-24T09:25:22.593 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.052055830, 2024-04-24T09:25:22.593 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000030711 2024-04-24T09:25:22.593 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-04-24T09:25:22.593 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.593 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-PurgeQueue": { 2024-04-24T09:25:22.593 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-04-24T09:25:22.593 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-04-24T09:25:22.593 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:22.594 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:22.594 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:22.594 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-04-24T09:25:22.594 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.594 INFO:teuthology.orchestra.run.smithi149.stdout: "mds": { 2024-04-24T09:25:22.594 INFO:teuthology.orchestra.run.smithi149.stdout: "request": 0, 2024-04-24T09:25:22.594 INFO:teuthology.orchestra.run.smithi149.stdout: "reply": 0, 2024-04-24T09:25:22.594 INFO:teuthology.orchestra.run.smithi149.stdout: "reply_latency": { 2024-04-24T09:25:22.594 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:22.595 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:22.595 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:22.595 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.595 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_reply": 0, 2024-04-24T09:25:22.595 INFO:teuthology.orchestra.run.smithi149.stdout: "forward": 0, 2024-04-24T09:25:22.595 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_complete": 0, 2024-04-24T09:25:22.595 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_keys": 0, 2024-04-24T09:25:22.595 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_commit": 0, 2024-04-24T09:25:22.596 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_split": 0, 2024-04-24T09:25:22.596 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_merge": 0, 2024-04-24T09:25:22.596 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes": 28, 2024-04-24T09:25:22.596 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_top": 1, 2024-04-24T09:25:22.596 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_bottom": 27, 2024-04-24T09:25:22.596 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pin_tail": 0, 2024-04-24T09:25:22.596 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pinned": 27, 2024-04-24T09:25:22.596 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_expired": 0, 2024-04-24T09:25:22.596 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_with_caps": 0, 2024-04-24T09:25:22.597 INFO:teuthology.orchestra.run.smithi149.stdout: "caps": 0, 2024-04-24T09:25:22.597 INFO:teuthology.orchestra.run.smithi149.stdout: "subtrees": 3, 2024-04-24T09:25:22.597 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse": 0, 2024-04-24T09:25:22.597 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_hit": 0, 2024-04-24T09:25:22.597 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_forward": 0, 2024-04-24T09:25:22.597 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_discover": 0, 2024-04-24T09:25:22.597 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_dir_fetch": 0, 2024-04-24T09:25:22.597 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_remote_ino": 0, 2024-04-24T09:25:22.597 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_lock": 0, 2024-04-24T09:25:22.598 INFO:teuthology.orchestra.run.smithi149.stdout: "load_cent": 0, 2024-04-24T09:25:22.598 INFO:teuthology.orchestra.run.smithi149.stdout: "q": 0, 2024-04-24T09:25:22.598 INFO:teuthology.orchestra.run.smithi149.stdout: "exported": 0, 2024-04-24T09:25:22.598 INFO:teuthology.orchestra.run.smithi149.stdout: "exported_inodes": 0, 2024-04-24T09:25:22.598 INFO:teuthology.orchestra.run.smithi149.stdout: "imported": 0, 2024-04-24T09:25:22.598 INFO:teuthology.orchestra.run.smithi149.stdout: "imported_inodes": 0, 2024-04-24T09:25:22.598 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_dir_fetch": 0, 2024-04-24T09:25:22.598 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_backtrace_fetch": 0, 2024-04-24T09:25:22.599 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_peer_discover": 0, 2024-04-24T09:25:22.599 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rfiles": 0, 2024-04-24T09:25:22.599 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rbytes": 0, 2024-04-24T09:25:22.599 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rsnaps": 0, 2024-04-24T09:25:22.599 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_fetch": 0, 2024-04-24T09:25:22.599 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_set_tag": 0, 2024-04-24T09:25:22.599 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T09:25:22.599 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_inotable_repaired": 0, 2024-04-24T09:25:22.599 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_inodes": 0, 2024-04-24T09:25:22.600 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T09:25:22.600 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dirfrag_rstats": 0, 2024-04-24T09:25:22.600 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_file_inodes": 0, 2024-04-24T09:25:22.600 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_inode_file_caps": 0, 2024-04-24T09:25:22.600 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_revoke": 0, 2024-04-24T09:25:22.600 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_grant": 0, 2024-04-24T09:25:22.600 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T09:25:22.600 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T09:25:22.600 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T09:25:22.601 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps": 0, 2024-04-24T09:25:22.601 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps_dirty": 0, 2024-04-24T09:25:22.601 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_cap_release": 0, 2024-04-24T09:25:22.601 INFO:teuthology.orchestra.run.smithi149.stdout: "process_request_cap_release": 0 2024-04-24T09:25:22.601 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.601 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_cache": { 2024-04-24T09:25:22.601 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update": 0, 2024-04-24T09:25:22.601 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update_receipt": 0, 2024-04-24T09:25:22.601 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_try_discover": 0, 2024-04-24T09:25:22.602 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_send_discover": 0, 2024-04-24T09:25:22.602 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_handle_discover": 0, 2024-04-24T09:25:22.602 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays": 0, 2024-04-24T09:25:22.602 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_delayed": 0, 2024-04-24T09:25:22.602 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_enqueuing": 0, 2024-04-24T09:25:22.602 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_created": 0, 2024-04-24T09:25:22.602 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_enqueued": 0, 2024-04-24T09:25:22.602 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_reintegrated": 0, 2024-04-24T09:25:22.603 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_migrated": 0, 2024-04-24T09:25:22.603 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_processing": 0, 2024-04-24T09:25:22.603 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_enqueued": 0, 2024-04-24T09:25:22.603 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_prioritized": 0, 2024-04-24T09:25:22.603 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_started": 0, 2024-04-24T09:25:22.603 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_completed": 0, 2024-04-24T09:25:22.603 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_quiesce_path": 0, 2024-04-24T09:25:22.603 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_quiesce_inode": 0, 2024-04-24T09:25:22.603 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T09:25:22.603 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_exportdir": 0, 2024-04-24T09:25:22.604 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_flush": 0, 2024-04-24T09:25:22.604 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragmentdir": 0, 2024-04-24T09:25:22.604 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragstats": 0, 2024-04-24T09:25:22.604 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_inodestats": 0 2024-04-24T09:25:22.604 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.604 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_log": { 2024-04-24T09:25:22.604 INFO:teuthology.orchestra.run.smithi149.stdout: "evlrg": 0, 2024-04-24T09:25:22.605 INFO:teuthology.orchestra.run.smithi149.stdout: "evadd": 0, 2024-04-24T09:25:22.605 INFO:teuthology.orchestra.run.smithi149.stdout: "evex": 0, 2024-04-24T09:25:22.605 INFO:teuthology.orchestra.run.smithi149.stdout: "evtrm": 0, 2024-04-24T09:25:22.605 INFO:teuthology.orchestra.run.smithi149.stdout: "ev": 55, 2024-04-24T09:25:22.605 INFO:teuthology.orchestra.run.smithi149.stdout: "evexg": 0, 2024-04-24T09:25:22.605 INFO:teuthology.orchestra.run.smithi149.stdout: "evexd": 0, 2024-04-24T09:25:22.605 INFO:teuthology.orchestra.run.smithi149.stdout: "segadd": 0, 2024-04-24T09:25:22.605 INFO:teuthology.orchestra.run.smithi149.stdout: "segex": 0, 2024-04-24T09:25:22.605 INFO:teuthology.orchestra.run.smithi149.stdout: "segtrm": 0, 2024-04-24T09:25:22.606 INFO:teuthology.orchestra.run.smithi149.stdout: "seg": 2, 2024-04-24T09:25:22.606 INFO:teuthology.orchestra.run.smithi149.stdout: "segmjr": 2, 2024-04-24T09:25:22.606 INFO:teuthology.orchestra.run.smithi149.stdout: "segexg": 0, 2024-04-24T09:25:22.606 INFO:teuthology.orchestra.run.smithi149.stdout: "segexd": 0, 2024-04-24T09:25:22.606 INFO:teuthology.orchestra.run.smithi149.stdout: "expos": 4194304, 2024-04-24T09:25:22.606 INFO:teuthology.orchestra.run.smithi149.stdout: "wrpos": 4266212, 2024-04-24T09:25:22.606 INFO:teuthology.orchestra.run.smithi149.stdout: "rdpos": 4265638, 2024-04-24T09:25:22.606 INFO:teuthology.orchestra.run.smithi149.stdout: "jlat": { 2024-04-24T09:25:22.607 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:22.607 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:22.607 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:22.607 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.607 INFO:teuthology.orchestra.run.smithi149.stdout: "replayed": 55 2024-04-24T09:25:22.607 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.607 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_mem": { 2024-04-24T09:25:22.607 INFO:teuthology.orchestra.run.smithi149.stdout: "ino": 25, 2024-04-24T09:25:22.607 INFO:teuthology.orchestra.run.smithi149.stdout: "ino+": 30, 2024-04-24T09:25:22.608 INFO:teuthology.orchestra.run.smithi149.stdout: "ino-": 5, 2024-04-24T09:25:22.608 INFO:teuthology.orchestra.run.smithi149.stdout: "dir": 12, 2024-04-24T09:25:22.608 INFO:teuthology.orchestra.run.smithi149.stdout: "dir+": 12, 2024-04-24T09:25:22.608 INFO:teuthology.orchestra.run.smithi149.stdout: "dir-": 0, 2024-04-24T09:25:22.608 INFO:teuthology.orchestra.run.smithi149.stdout: "dn": 28, 2024-04-24T09:25:22.608 INFO:teuthology.orchestra.run.smithi149.stdout: "dn+": 28, 2024-04-24T09:25:22.608 INFO:teuthology.orchestra.run.smithi149.stdout: "dn-": 0, 2024-04-24T09:25:22.608 INFO:teuthology.orchestra.run.smithi149.stdout: "cap": 0, 2024-04-24T09:25:22.608 INFO:teuthology.orchestra.run.smithi149.stdout: "cap+": 0, 2024-04-24T09:25:22.609 INFO:teuthology.orchestra.run.smithi149.stdout: "cap-": 0, 2024-04-24T09:25:22.609 INFO:teuthology.orchestra.run.smithi149.stdout: "rss": 49380, 2024-04-24T09:25:22.609 INFO:teuthology.orchestra.run.smithi149.stdout: "heap": 215416 2024-04-24T09:25:22.609 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.609 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_server": { 2024-04-24T09:25:22.609 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_client_request": 0, 2024-04-24T09:25:22.609 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_server_request": 0, 2024-04-24T09:25:22.609 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_request": 0, 2024-04-24T09:25:22.609 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_session": 0, 2024-04-24T09:25:22.610 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_peer_request": 0, 2024-04-24T09:25:22.610 INFO:teuthology.orchestra.run.smithi149.stdout: "req_create_latency": { 2024-04-24T09:25:22.610 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:22.610 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:22.610 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:22.610 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.610 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getattr_latency": { 2024-04-24T09:25:22.610 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:22.610 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:22.611 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:22.611 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.611 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getfilelock_latency": { 2024-04-24T09:25:22.611 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:22.611 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:22.611 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:22.611 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.611 INFO:teuthology.orchestra.run.smithi149.stdout: "req_link_latency": { 2024-04-24T09:25:22.611 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:22.612 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:22.612 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:22.612 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.612 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookup_latency": { 2024-04-24T09:25:22.612 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:22.612 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:22.612 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:22.612 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.612 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookuphash_latency": { 2024-04-24T09:25:22.613 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:22.613 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:22.613 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:22.613 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.613 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupino_latency": { 2024-04-24T09:25:22.613 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:22.613 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:22.613 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:22.613 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.613 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupname_latency": { 2024-04-24T09:25:22.614 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:22.614 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:22.614 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:22.614 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.614 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupparent_latency": { 2024-04-24T09:25:22.614 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:22.614 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:22.614 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:22.615 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.615 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupsnap_latency": { 2024-04-24T09:25:22.615 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:22.615 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:22.615 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:22.615 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.615 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lssnap_latency": { 2024-04-24T09:25:22.615 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:22.615 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:22.616 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:22.616 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.616 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mkdir_latency": { 2024-04-24T09:25:22.616 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:22.617 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:22.617 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:22.617 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.617 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mknod_latency": { 2024-04-24T09:25:22.617 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:22.617 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:22.617 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:22.618 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.618 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mksnap_latency": { 2024-04-24T09:25:22.618 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:22.618 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:22.618 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:22.618 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.618 INFO:teuthology.orchestra.run.smithi149.stdout: "req_open_latency": { 2024-04-24T09:25:22.618 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:22.618 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:22.619 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:22.619 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.619 INFO:teuthology.orchestra.run.smithi149.stdout: "req_readdir_latency": { 2024-04-24T09:25:22.619 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:22.619 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:22.619 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:22.619 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.619 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rename_latency": { 2024-04-24T09:25:22.619 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:22.620 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:22.620 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:22.620 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.620 INFO:teuthology.orchestra.run.smithi149.stdout: "req_renamesnap_latency": { 2024-04-24T09:25:22.620 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:22.620 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:22.620 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:22.620 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.621 INFO:teuthology.orchestra.run.smithi149.stdout: "req_snapdiff_latency": { 2024-04-24T09:25:22.621 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:22.621 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:22.621 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:22.621 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.621 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmdir_latency": { 2024-04-24T09:25:22.621 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:22.621 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:22.621 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:22.622 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.622 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmsnap_latency": { 2024-04-24T09:25:22.622 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:22.622 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:22.622 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:22.622 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.622 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmxattr_latency": { 2024-04-24T09:25:22.622 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:22.623 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:22.623 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:22.623 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.623 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setattr_latency": { 2024-04-24T09:25:22.623 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:22.623 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:22.623 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:22.623 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.624 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setdirlayout_latency": { 2024-04-24T09:25:22.624 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:22.624 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:22.624 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:22.624 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.624 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setfilelock_latency": { 2024-04-24T09:25:22.624 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:22.624 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:22.624 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:22.625 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.625 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setlayout_latency": { 2024-04-24T09:25:22.625 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:22.625 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:22.625 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:22.625 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.625 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setxattr_latency": { 2024-04-24T09:25:22.625 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:22.625 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:22.626 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:22.626 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.626 INFO:teuthology.orchestra.run.smithi149.stdout: "req_symlink_latency": { 2024-04-24T09:25:22.626 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:22.626 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:22.626 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:22.626 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.626 INFO:teuthology.orchestra.run.smithi149.stdout: "req_unlink_latency": { 2024-04-24T09:25:22.627 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:22.627 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:22.627 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:22.627 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.627 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_revoke_eviction": 0, 2024-04-24T09:25:22.627 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_acquisition_throttle": 0, 2024-04-24T09:25:22.627 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getvxattr_latency": { 2024-04-24T09:25:22.627 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:22.627 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:22.628 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:22.628 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-04-24T09:25:22.628 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.628 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_sessions": { 2024-04-24T09:25:22.628 INFO:teuthology.orchestra.run.smithi149.stdout: "session_count": 1, 2024-04-24T09:25:22.628 INFO:teuthology.orchestra.run.smithi149.stdout: "session_add": 1, 2024-04-24T09:25:22.628 INFO:teuthology.orchestra.run.smithi149.stdout: "session_remove": 0, 2024-04-24T09:25:22.628 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_open": 1, 2024-04-24T09:25:22.628 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_stale": 0, 2024-04-24T09:25:22.629 INFO:teuthology.orchestra.run.smithi149.stdout: "total_load": 0, 2024-04-24T09:25:22.629 INFO:teuthology.orchestra.run.smithi149.stdout: "average_load": 0, 2024-04-24T09:25:22.629 INFO:teuthology.orchestra.run.smithi149.stdout: "avg_session_uptime": 1758, 2024-04-24T09:25:22.629 INFO:teuthology.orchestra.run.smithi149.stdout: "mdthresh_evicted": 0 2024-04-24T09:25:22.629 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.629 INFO:teuthology.orchestra.run.smithi149.stdout: "mempool": { 2024-04-24T09:25:22.629 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_bytes": 0, 2024-04-24T09:25:22.629 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_items": 0, 2024-04-24T09:25:22.629 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T09:25:22.630 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_items": 0, 2024-04-24T09:25:22.630 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T09:25:22.630 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_items": 0, 2024-04-24T09:25:22.630 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T09:25:22.630 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T09:25:22.630 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T09:25:22.630 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T09:25:22.630 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T09:25:22.631 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_items": 0, 2024-04-24T09:25:22.631 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T09:25:22.631 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T09:25:22.631 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_bytes": 0, 2024-04-24T09:25:22.631 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_items": 0, 2024-04-24T09:25:22.631 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_bytes": 0, 2024-04-24T09:25:22.631 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_items": 0, 2024-04-24T09:25:22.631 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T09:25:22.631 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T09:25:22.632 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T09:25:22.632 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T09:25:22.632 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T09:25:22.632 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_items": 0, 2024-04-24T09:25:22.632 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_bytes": 0, 2024-04-24T09:25:22.632 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_items": 0, 2024-04-24T09:25:22.632 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T09:25:22.632 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T09:25:22.633 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_bytes": 0, 2024-04-24T09:25:22.633 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_items": 0, 2024-04-24T09:25:22.633 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_bytes": 0, 2024-04-24T09:25:22.633 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_items": 0, 2024-04-24T09:25:22.633 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T09:25:22.633 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_items": 0, 2024-04-24T09:25:22.633 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T09:25:22.633 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_items": 0, 2024-04-24T09:25:22.633 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_bytes": 189846, 2024-04-24T09:25:22.634 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_items": 65, 2024-04-24T09:25:22.634 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_bytes": 0, 2024-04-24T09:25:22.634 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_items": 0, 2024-04-24T09:25:22.634 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_bytes": 0, 2024-04-24T09:25:22.634 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_items": 0, 2024-04-24T09:25:22.634 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_bytes": 0, 2024-04-24T09:25:22.634 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_items": 0, 2024-04-24T09:25:22.634 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_bytes": 0, 2024-04-24T09:25:22.634 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_items": 0, 2024-04-24T09:25:22.635 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_bytes": 7624, 2024-04-24T09:25:22.635 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_items": 160, 2024-04-24T09:25:22.635 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T09:25:22.635 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_items": 0, 2024-04-24T09:25:22.635 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_bytes": 0, 2024-04-24T09:25:22.635 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_items": 0, 2024-04-24T09:25:22.635 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_bytes": 92838, 2024-04-24T09:25:22.635 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_items": 611, 2024-04-24T09:25:22.635 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_bytes": 0, 2024-04-24T09:25:22.636 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_items": 0, 2024-04-24T09:25:22.636 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_bytes": 0, 2024-04-24T09:25:22.636 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_items": 0 2024-04-24T09:25:22.636 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.636 INFO:teuthology.orchestra.run.smithi149.stdout: "objecter": { 2024-04-24T09:25:22.636 INFO:teuthology.orchestra.run.smithi149.stdout: "op_active": 0, 2024-04-24T09:25:22.636 INFO:teuthology.orchestra.run.smithi149.stdout: "op_laggy": 0, 2024-04-24T09:25:22.636 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send": 963, 2024-04-24T09:25:22.636 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send_bytes": 21, 2024-04-24T09:25:22.636 INFO:teuthology.orchestra.run.smithi149.stdout: "op_resend": 3, 2024-04-24T09:25:22.637 INFO:teuthology.orchestra.run.smithi149.stdout: "op_reply": 963, 2024-04-24T09:25:22.637 INFO:teuthology.orchestra.run.smithi149.stdout: "op_latency": { 2024-04-24T09:25:22.637 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 963, 2024-04-24T09:25:22.637 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.797983675, 2024-04-24T09:25:22.637 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000828643 2024-04-24T09:25:22.637 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.637 INFO:teuthology.orchestra.run.smithi149.stdout: "op_inflight": 0, 2024-04-24T09:25:22.637 INFO:teuthology.orchestra.run.smithi149.stdout: "oplen_avg": { 2024-04-24T09:25:22.637 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 963, 2024-04-24T09:25:22.638 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 964 2024-04-24T09:25:22.638 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.638 INFO:teuthology.orchestra.run.smithi149.stdout: "op": 963, 2024-04-24T09:25:22.638 INFO:teuthology.orchestra.run.smithi149.stdout: "op_r": 963, 2024-04-24T09:25:22.638 INFO:teuthology.orchestra.run.smithi149.stdout: "op_w": 0, 2024-04-24T09:25:22.638 INFO:teuthology.orchestra.run.smithi149.stdout: "op_rmw": 0, 2024-04-24T09:25:22.638 INFO:teuthology.orchestra.run.smithi149.stdout: "op_pg": 0, 2024-04-24T09:25:22.638 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_stat": 578, 2024-04-24T09:25:22.638 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_create": 0, 2024-04-24T09:25:22.639 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_read": 383, 2024-04-24T09:25:22.639 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_write": 0, 2024-04-24T09:25:22.639 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writefull": 0, 2024-04-24T09:25:22.639 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writesame": 0, 2024-04-24T09:25:22.639 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_append": 0, 2024-04-24T09:25:22.639 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_zero": 0, 2024-04-24T09:25:22.639 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_truncate": 0, 2024-04-24T09:25:22.640 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_delete": 0, 2024-04-24T09:25:22.640 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_mapext": 0, 2024-04-24T09:25:22.640 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_sparse_read": 0, 2024-04-24T09:25:22.640 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_clonerange": 0, 2024-04-24T09:25:22.640 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_getxattr": 1, 2024-04-24T09:25:22.640 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_setxattr": 0, 2024-04-24T09:25:22.640 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_cmpxattr": 0, 2024-04-24T09:25:22.641 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_rmxattr": 0, 2024-04-24T09:25:22.641 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_resetxattrs": 0, 2024-04-24T09:25:22.641 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_call": 0, 2024-04-24T09:25:22.641 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_watch": 0, 2024-04-24T09:25:22.641 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_notify": 0, 2024-04-24T09:25:22.641 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T09:25:22.641 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls": 0, 2024-04-24T09:25:22.641 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls_filter": 0, 2024-04-24T09:25:22.641 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_other": 0, 2024-04-24T09:25:22.642 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_active": 0, 2024-04-24T09:25:22.642 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_send": 0, 2024-04-24T09:25:22.642 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_resend": 0, 2024-04-24T09:25:22.642 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_ping": 0, 2024-04-24T09:25:22.642 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_active": 0, 2024-04-24T09:25:22.642 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_send": 0, 2024-04-24T09:25:22.642 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_resend": 0, 2024-04-24T09:25:22.642 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_active": 0, 2024-04-24T09:25:22.643 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_send": 0, 2024-04-24T09:25:22.643 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_resend": 0, 2024-04-24T09:25:22.643 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_active": 0, 2024-04-24T09:25:22.643 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_send": 0, 2024-04-24T09:25:22.643 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_resend": 0, 2024-04-24T09:25:22.643 INFO:teuthology.orchestra.run.smithi149.stdout: "command_active": 0, 2024-04-24T09:25:22.643 INFO:teuthology.orchestra.run.smithi149.stdout: "command_send": 0, 2024-04-24T09:25:22.643 INFO:teuthology.orchestra.run.smithi149.stdout: "command_resend": 0, 2024-04-24T09:25:22.643 INFO:teuthology.orchestra.run.smithi149.stdout: "map_epoch": 88, 2024-04-24T09:25:22.644 INFO:teuthology.orchestra.run.smithi149.stdout: "map_full": 0, 2024-04-24T09:25:22.644 INFO:teuthology.orchestra.run.smithi149.stdout: "map_inc": 2, 2024-04-24T09:25:22.644 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_sessions": 5, 2024-04-24T09:25:22.644 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_open": 5, 2024-04-24T09:25:22.644 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_close": 0, 2024-04-24T09:25:22.644 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_laggy": 0, 2024-04-24T09:25:22.644 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_wr": 0, 2024-04-24T09:25:22.644 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_rd": 2, 2024-04-24T09:25:22.644 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_del": 0 2024-04-24T09:25:22.645 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.645 INFO:teuthology.orchestra.run.smithi149.stdout: "oft": { 2024-04-24T09:25:22.645 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_objs": 0, 2024-04-24T09:25:22.645 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_kv_pairs": 0, 2024-04-24T09:25:22.645 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_updates": 0, 2024-04-24T09:25:22.645 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_removes": 0 2024-04-24T09:25:22.645 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.645 INFO:teuthology.orchestra.run.smithi149.stdout: "osd-slow-ops": { 2024-04-24T09:25:22.645 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_ops_count": 0 2024-04-24T09:25:22.646 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.646 INFO:teuthology.orchestra.run.smithi149.stdout: "purge_queue": { 2024-04-24T09:25:22.646 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops": 0, 2024-04-24T09:25:22.646 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops_high_water": 0, 2024-04-24T09:25:22.646 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing": 0, 2024-04-24T09:25:22.646 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_high_water": 0, 2024-04-24T09:25:22.646 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed_ops": 0, 2024-04-24T09:25:22.646 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed": 0, 2024-04-24T09:25:22.646 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_item_in_journal": 0 2024-04-24T09:25:22.647 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.647 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T09:25:22.647 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-04-24T09:25:22.647 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-04-24T09:25:22.647 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-04-24T09:25:22.647 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 1098, 2024-04-24T09:25:22.647 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 1033561, 2024-04-24T09:25:22.647 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-04-24T09:25:22.647 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 1098, 2024-04-24T09:25:22.648 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-04-24T09:25:22.648 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-04-24T09:25:22.648 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 1098, 2024-04-24T09:25:22.648 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 1033561, 2024-04-24T09:25:22.648 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-04-24T09:25:22.648 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:22.648 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:22.648 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:22.648 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-04-24T09:25:22.648 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.649 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_bytes": { 2024-04-24T09:25:22.649 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-04-24T09:25:22.649 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-04-24T09:25:22.649 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-04-24T09:25:22.649 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-04-24T09:25:22.649 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-04-24T09:25:22.649 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-04-24T09:25:22.649 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-04-24T09:25:22.649 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 963, 2024-04-24T09:25:22.650 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 71913, 2024-04-24T09:25:22.650 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 963, 2024-04-24T09:25:22.650 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 71913, 2024-04-24T09:25:22.650 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-04-24T09:25:22.650 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:22.650 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:22.650 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:22.650 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-04-24T09:25:22.650 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.651 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_ops": { 2024-04-24T09:25:22.651 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-04-24T09:25:22.651 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 1024, 2024-04-24T09:25:22.651 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-04-24T09:25:22.651 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-04-24T09:25:22.651 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-04-24T09:25:22.651 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-04-24T09:25:22.651 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-04-24T09:25:22.651 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 963, 2024-04-24T09:25:22.651 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 963, 2024-04-24T09:25:22.652 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 963, 2024-04-24T09:25:22.652 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 963, 2024-04-24T09:25:22.652 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-04-24T09:25:22.652 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:22.652 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:22.652 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:22.652 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-04-24T09:25:22.652 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.652 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle": { 2024-04-24T09:25:22.653 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-04-24T09:25:22.653 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-04-24T09:25:22.653 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-04-24T09:25:22.653 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-04-24T09:25:22.653 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-04-24T09:25:22.653 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-04-24T09:25:22.653 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-04-24T09:25:22.653 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-04-24T09:25:22.654 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-04-24T09:25:22.654 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 0, 2024-04-24T09:25:22.654 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 0, 2024-04-24T09:25:22.654 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-04-24T09:25:22.654 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:22.654 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:22.654 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:22.654 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-04-24T09:25:22.654 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-04-24T09:25:22.655 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle-0x55b94ae6a6e0": { 2024-04-24T09:25:22.655 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-04-24T09:25:22.655 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-04-24T09:25:22.655 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-04-24T09:25:22.655 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-04-24T09:25:22.655 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-04-24T09:25:22.655 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-04-24T09:25:22.655 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-04-24T09:25:22.656 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-04-24T09:25:22.656 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-04-24T09:25:22.656 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 0, 2024-04-24T09:25:22.656 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 0, 2024-04-24T09:25:22.656 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-04-24T09:25:22.656 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-04-24T09:25:22.656 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-04-24T09:25:22.656 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-04-24T09:25:22.656 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-04-24T09:25:22.657 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-04-24T09:25:22.657 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-04-24T09:25:22.861 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:22 smithi149 ceph-mon[32592]: pgmap v510: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 9 op/s 2024-04-24T09:25:23.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:22 smithi090 ceph-mon[27825]: pgmap v510: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 9 op/s 2024-04-24T09:25:23.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:22 smithi114 ceph-mon[32501]: pgmap v510: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 9 op/s 2024-04-24T09:25:23.465 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-04-24T09:25:23.465 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-04-24T09:25:23.466 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-04-24T09:25:23.466 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-04-24T09:25:23.466 INFO:tasks.check_counter:Daemon mds.l mds.dir_split=0 2024-04-24T09:25:23.466 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-24T09:25:23.476 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-24T09:25:23.476 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-24T09:25:23.477 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T09:25:23.536 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-24T09:25:23.536 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T09:25:23.589 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T09:25:23.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:23.796+0000 7feee3200640 1 -- 172.21.15.90:0/414238048 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feedc105e10 msgr2=0x7feedc10c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:23.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:23.796+0000 7feee3200640 1 --2- 172.21.15.90:0/414238048 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feedc105e10 0x7feedc10c6a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7feed4009ca0 tx=0x7feed402a8b0 comp rx=0 tx=0).stop 2024-04-24T09:25:23.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:23.796+0000 7feee3200640 1 -- 172.21.15.90:0/414238048 shutdown_connections 2024-04-24T09:25:23.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:23.796+0000 7feee3200640 1 --2- 172.21.15.90:0/414238048 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feedc105e10 0x7feedc10c6a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:23.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:23.796+0000 7feee3200640 1 --2- 172.21.15.90:0/414238048 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feedc105450 0x7feedc1058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:23.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:23.796+0000 7feee3200640 1 --2- 172.21.15.90:0/414238048 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feedc104260 0x7feedc104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:23.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:23.796+0000 7feee3200640 1 -- 172.21.15.90:0/414238048 >> 172.21.15.90:0/414238048 conn(0x7feedc0ff7f0 msgr2=0x7feedc101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:25:23.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:23.796+0000 7feee3200640 1 -- 172.21.15.90:0/414238048 shutdown_connections 2024-04-24T09:25:23.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:23.796+0000 7feee3200640 1 -- 172.21.15.90:0/414238048 wait complete. 2024-04-24T09:25:23.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:23.797+0000 7feee3200640 1 Processor -- start 2024-04-24T09:25:23.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:23.797+0000 7feee3200640 1 -- start start 2024-04-24T09:25:23.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:23.797+0000 7feee3200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feedc104260 0x7feedc19a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:25:23.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:23.797+0000 7feee3200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feedc105450 0x7feedc19aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:25:23.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:23.797+0000 7feee3200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feedc105e10 0x7feedc1a1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:25:23.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:23.797+0000 7feee3200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7feedc10ff10 con 0x7feedc105450 2024-04-24T09:25:23.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:23.797+0000 7feee3200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7feedc10fd90 con 0x7feedc104260 2024-04-24T09:25:23.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:23.797+0000 7feee3200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7feedc110090 con 0x7feedc105e10 2024-04-24T09:25:23.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:23.798+0000 7feed3400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feedc105450 0x7feedc19aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:25:23.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:23.798+0000 7feed3400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feedc105450 0x7feedc19aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47338/0 (socket says 172.21.15.90:47338) 2024-04-24T09:25:23.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:23.798+0000 7feed3400640 1 -- 172.21.15.90:0/4228129706 learned_addr learned my addr 172.21.15.90:0/4228129706 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:25:23.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:23.798+0000 7feedbe00640 1 --2- 172.21.15.90:0/4228129706 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feedc104260 0x7feedc19a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:25:23.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:23.798+0000 7feee1200640 1 --2- 172.21.15.90:0/4228129706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feedc105e10 0x7feedc1a1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:25:23.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:23.798+0000 7feedbe00640 1 -- 172.21.15.90:0/4228129706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feedc105e10 msgr2=0x7feedc1a1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:23.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:23.798+0000 7feedbe00640 1 --2- 172.21.15.90:0/4228129706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feedc105e10 0x7feedc1a1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:23.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:23.798+0000 7feedbe00640 1 -- 172.21.15.90:0/4228129706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feedc105450 msgr2=0x7feedc19aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:23.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:23.798+0000 7feedbe00640 1 --2- 172.21.15.90:0/4228129706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feedc105450 0x7feedc19aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:23.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:23.798+0000 7feedbe00640 1 -- 172.21.15.90:0/4228129706 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feedc1a21a0 con 0x7feedc104260 2024-04-24T09:25:23.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:23.798+0000 7feed3400640 1 --2- 172.21.15.90:0/4228129706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feedc105450 0x7feedc19aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:25:23.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:23.799+0000 7feedbe00640 1 --2- 172.21.15.90:0/4228129706 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feedc104260 0x7feedc19a580 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7feecc00e9c0 tx=0x7feecc00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:25:23.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:23.799+0000 7feed9600640 1 -- 172.21.15.90:0/4228129706 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc00ccc0 con 0x7feedc104260 2024-04-24T09:25:23.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:23.799+0000 7feed9600640 1 -- 172.21.15.90:0/4228129706 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feecc0035f0 con 0x7feedc104260 2024-04-24T09:25:23.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:23.799+0000 7feee3200640 1 -- 172.21.15.90:0/4228129706 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feedc1a2490 con 0x7feedc104260 2024-04-24T09:25:23.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:23.799+0000 7feed9600640 1 -- 172.21.15.90:0/4228129706 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc00ce60 con 0x7feedc104260 2024-04-24T09:25:23.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:23.799+0000 7feee3200640 1 -- 172.21.15.90:0/4228129706 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feedc1a2a30 con 0x7feedc104260 2024-04-24T09:25:23.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:23.800+0000 7feee3200640 1 -- 172.21.15.90:0/4228129706 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feedc10b4b0 con 0x7feedc104260 2024-04-24T09:25:23.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:23.802+0000 7feed9600640 1 -- 172.21.15.90:0/4228129706 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7feecc0037a0 con 0x7feedc104260 2024-04-24T09:25:23.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:23.803+0000 7feed9600640 1 --2- 172.21.15.90:0/4228129706 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7feebc0a3c30 0x7feebc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:25:23.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:23.803+0000 7feed9600640 1 -- 172.21.15.90:0/4228129706 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7feecc0d06c0 con 0x7feedc104260 2024-04-24T09:25:23.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:23.804+0000 7feed3400640 1 --2- 172.21.15.90:0/4228129706 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7feebc0a3c30 0x7feebc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:25:23.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:23.804+0000 7feed3400640 1 --2- 172.21.15.90:0/4228129706 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7feebc0a3c30 0x7feebc0a60f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7feedc19ba90 tx=0x7feec4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:25:23.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:23.808+0000 7feed9600640 1 -- 172.21.15.90:0/4228129706 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feecc09d0a0 con 0x7feedc104260 2024-04-24T09:25:24.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.011+0000 7feee3200640 1 -- 172.21.15.90:0/4228129706 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feedc02d620 con 0x7feedc104260 2024-04-24T09:25:24.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.013+0000 7feed9600640 1 -- 172.21.15.90:0/4228129706 <== mon.2 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22652 (secure 0 0 0) 0x7feecc09c9a0 con 0x7feedc104260 2024-04-24T09:25:24.014 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:25:24.014 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":88,"fsid":"30ee9be2-021a-11ef-bc93-c7b262605968","created":"2024-04-24T09:09:49.496019+0000","modified":"2024-04-24T09:19:58.609640+0000","last_up_change":"2024-04-24T09:18:10.455632+0000","last_in_change":"2024-04-24T09:17:50.600366+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T09:13:45.548774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T09:19:07.185580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T09:19:08.374818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13a59b9f-b46e-4ec4-90af-bb7a3d5c753b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6803","nonce":2864288066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6805","nonce":2864288066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6809","nonce":2864288066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6807","nonce":2864288066}]},"public_addr":"172.21.15.90:6803/2864288066","cluster_addr":"172.21.15.90:6805/2864288066","heartbeat_back_addr":"172.21.15.90:6809/2864288066","heartbeat_front_addr":"172.21.15.90:6807/2864288066","state":["exists","up"]},{"osd":1,"uuid":"1567ce8f-24e6-42cf-bf64-f6f0c534a9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6801","nonce":344222414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6803","nonce":344222414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6807","nonce":344222414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6805","nonce":344222414}]},"public_addr":"172.21.15.114:6801/344222414","cluster_addr":"172.21.15.114:6803/344222414","heartbeat_back_addr":"172.21.15.114:6807/344222414","heartbeat_front_addr":"172.21.15.114:6805/344222414","state":["exists","up"]},{"osd":2,"uuid":"f861b304-9c7d-4629-8522-37f33a747945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6801","nonce":3257067419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6803","nonce":3257067419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6807","nonce":3257067419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6805","nonce":3257067419}]},"public_addr":"172.21.15.149:6801/3257067419","cluster_addr":"172.21.15.149:6803/3257067419","heartbeat_back_addr":"172.21.15.149:6807/3257067419","heartbeat_front_addr":"172.21.15.149:6805/3257067419","state":["exists","up"]},{"osd":3,"uuid":"9377ab6e-5996-45fb-93a9-475a2e5f17a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6811","nonce":3068524970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6813","nonce":3068524970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6817","nonce":3068524970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6815","nonce":3068524970}]},"public_addr":"172.21.15.90:6811/3068524970","cluster_addr":"172.21.15.90:6813/3068524970","heartbeat_back_addr":"172.21.15.90:6817/3068524970","heartbeat_front_addr":"172.21.15.90:6815/3068524970","state":["exists","up"]},{"osd":4,"uuid":"e2ddd19e-e641-4d61-b342-02b5f12b1c4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6809","nonce":2782015522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6811","nonce":2782015522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6815","nonce":2782015522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6813","nonce":2782015522}]},"public_addr":"172.21.15.114:6809/2782015522","cluster_addr":"172.21.15.114:6811/2782015522","heartbeat_back_addr":"172.21.15.114:6815/2782015522","heartbeat_front_addr":"172.21.15.114:6813/2782015522","state":["exists","up"]},{"osd":5,"uuid":"b6ff9d10-5210-4b46-a192-6dcbf9d1f156","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6809","nonce":4266460479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6811","nonce":4266460479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6815","nonce":4266460479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6813","nonce":4266460479}]},"public_addr":"172.21.15.149:6809/4266460479","cluster_addr":"172.21.15.149:6811/4266460479","heartbeat_back_addr":"172.21.15.149:6815/4266460479","heartbeat_front_addr":"172.21.15.149:6813/4266460479","state":["exists","up"]},{"osd":6,"uuid":"4287b5a1-f953-421c-bf62-5e390bee834d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6819","nonce":1121461276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6821","nonce":1121461276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6825","nonce":1121461276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6823","nonce":1121461276}]},"public_addr":"172.21.15.90:6819/1121461276","cluster_addr":"172.21.15.90:6821/1121461276","heartbeat_back_addr":"172.21.15.90:6825/1121461276","heartbeat_front_addr":"172.21.15.90:6823/1121461276","state":["exists","up"]},{"osd":7,"uuid":"a54e507d-d6ba-4bf2-9a7f-a7b95c3859bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6817","nonce":3222546280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6819","nonce":3222546280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6823","nonce":3222546280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6821","nonce":3222546280}]},"public_addr":"172.21.15.114:6817/3222546280","cluster_addr":"172.21.15.114:6819/3222546280","heartbeat_back_addr":"172.21.15.114:6823/3222546280","heartbeat_front_addr":"172.21.15.114:6821/3222546280","state":["exists","up"]},{"osd":8,"uuid":"9950b010-5865-4109-ae37-582b1dbd2f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6817","nonce":157615149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6819","nonce":157615149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6823","nonce":157615149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6821","nonce":157615149}]},"public_addr":"172.21.15.149:6817/157615149","cluster_addr":"172.21.15.149:6819/157615149","heartbeat_back_addr":"172.21.15.149:6823/157615149","heartbeat_front_addr":"172.21.15.149:6821/157615149","state":["exists","up"]},{"osd":9,"uuid":"31e2635c-e234-421f-9757-592fd673c936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6827","nonce":1307318076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6829","nonce":1307318076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6833","nonce":1307318076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6831","nonce":1307318076}]},"public_addr":"172.21.15.90:6827/1307318076","cluster_addr":"172.21.15.90:6829/1307318076","heartbeat_back_addr":"172.21.15.90:6833/1307318076","heartbeat_front_addr":"172.21.15.90:6831/1307318076","state":["exists","up"]},{"osd":10,"uuid":"46a3b2a6-e6e5-4578-b4bb-f3e998b9bc95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6825","nonce":1719802140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6827","nonce":1719802140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6831","nonce":1719802140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6829","nonce":1719802140}]},"public_addr":"172.21.15.114:6825/1719802140","cluster_addr":"172.21.15.114:6827/1719802140","heartbeat_back_addr":"172.21.15.114:6831/1719802140","heartbeat_front_addr":"172.21.15.114:6829/1719802140","state":["exists","up"]},{"osd":11,"uuid":"ccd4456b-e947-46cf-a779-148ffa289b58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6825","nonce":1737275166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6827","nonce":1737275166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6831","nonce":1737275166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6829","nonce":1737275166}]},"public_addr":"172.21.15.149:6825/1737275166","cluster_addr":"172.21.15.149:6827/1737275166","heartbeat_back_addr":"172.21.15.149:6831/1737275166","heartbeat_front_addr":"172.21.15.149:6829/1737275166","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:12:53.015078+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:13:18.047053+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:13:41.935891+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:14:15.095388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:14:42.512355+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:15:08.113775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:15:43.828550+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:16:10.901909+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:16:36.422962+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:17:13.924446+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:17:42.814401+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:18:08.672793+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/3090852580":"2024-04-25T09:10:57.377655+0000","172.21.15.90:0/687197394":"2024-04-25T09:10:57.377655+0000","172.21.15.90:6801/1809038230":"2024-04-25T09:10:57.377655+0000","172.21.15.90:6800/1809038230":"2024-04-25T09:10:57.377655+0000","172.21.15.90:0/2964149071":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/3617388113":"2024-04-25T09:10:22.599674+0000","172.21.15.90:6801/3089140842":"2024-04-25T09:10:22.599674+0000","172.21.15.90:6800/3089140842":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/2840416149":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/305432710":"2024-04-25T09:10:57.377655+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T09:25:24.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.017+0000 7feee3200640 1 -- 172.21.15.90:0/4228129706 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7feebc0a3c30 msgr2=0x7feebc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:24.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.017+0000 7feee3200640 1 --2- 172.21.15.90:0/4228129706 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7feebc0a3c30 0x7feebc0a60f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7feedc19ba90 tx=0x7feec4005f90 comp rx=0 tx=0).stop 2024-04-24T09:25:24.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.017+0000 7feee3200640 1 -- 172.21.15.90:0/4228129706 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feedc104260 msgr2=0x7feedc19a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:24.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.017+0000 7feee3200640 1 --2- 172.21.15.90:0/4228129706 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feedc104260 0x7feedc19a580 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7feecc00e9c0 tx=0x7feecc00ee80 comp rx=0 tx=0).stop 2024-04-24T09:25:24.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.017+0000 7feee3200640 1 -- 172.21.15.90:0/4228129706 shutdown_connections 2024-04-24T09:25:24.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.017+0000 7feee3200640 1 --2- 172.21.15.90:0/4228129706 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7feebc0a3c30 0x7feebc0a60f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:24.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.017+0000 7feee3200640 1 --2- 172.21.15.90:0/4228129706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feedc105e10 0x7feedc1a1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:24.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.017+0000 7feee3200640 1 --2- 172.21.15.90:0/4228129706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feedc105450 0x7feedc19aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:24.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.017+0000 7feee3200640 1 --2- 172.21.15.90:0/4228129706 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feedc104260 0x7feedc19a580 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:24.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.017+0000 7feee3200640 1 -- 172.21.15.90:0/4228129706 >> 172.21.15.90:0/4228129706 conn(0x7feedc0ff7f0 msgr2=0x7feedc108680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:25:24.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.017+0000 7feee3200640 1 -- 172.21.15.90:0/4228129706 shutdown_connections 2024-04-24T09:25:24.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.017+0000 7feee3200640 1 -- 172.21.15.90:0/4228129706 wait complete. 2024-04-24T09:25:24.037 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T09:13:45.548774+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T09:19:07.185580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T09:19:08.374818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T09:25:24.038 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T09:25:24.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.245+0000 7f4f8f200640 1 -- 172.21.15.90:0/349340191 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f88105470 msgr2=0x7f4f881058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:24.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.245+0000 7f4f8f200640 1 --2- 172.21.15.90:0/349340191 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f88105470 0x7f4f881058f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4f70009900 tx=0x7f4f7002a410 comp rx=0 tx=0).stop 2024-04-24T09:25:24.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.245+0000 7f4f8f200640 1 -- 172.21.15.90:0/349340191 shutdown_connections 2024-04-24T09:25:24.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.245+0000 7f4f8f200640 1 --2- 172.21.15.90:0/349340191 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f88105e30 0x7f4f8810c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:24.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.245+0000 7f4f8f200640 1 --2- 172.21.15.90:0/349340191 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f88105470 0x7f4f881058f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:24.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.245+0000 7f4f8f200640 1 --2- 172.21.15.90:0/349340191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f88104280 0x7f4f881046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:24.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.245+0000 7f4f8f200640 1 -- 172.21.15.90:0/349340191 >> 172.21.15.90:0/349340191 conn(0x7f4f880ff7f0 msgr2=0x7f4f88101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:25:24.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.245+0000 7f4f8f200640 1 -- 172.21.15.90:0/349340191 shutdown_connections 2024-04-24T09:25:24.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.245+0000 7f4f8f200640 1 -- 172.21.15.90:0/349340191 wait complete. 2024-04-24T09:25:24.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.246+0000 7f4f8f200640 1 Processor -- start 2024-04-24T09:25:24.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.246+0000 7f4f8f200640 1 -- start start 2024-04-24T09:25:24.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.246+0000 7f4f8f200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f88104280 0x7f4f8819a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:25:24.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.247+0000 7f4f8f200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f88105470 0x7f4f8819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:25:24.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.247+0000 7f4f8f200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f88105e30 0x7f4f881a1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:25:24.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.247+0000 7f4f8f200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4f8810ff60 con 0x7f4f88105470 2024-04-24T09:25:24.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.247+0000 7f4f8f200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4f8810fde0 con 0x7f4f88104280 2024-04-24T09:25:24.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.247+0000 7f4f8f200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4f881100e0 con 0x7f4f88105e30 2024-04-24T09:25:24.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.247+0000 7f4f87e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f88104280 0x7f4f8819a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:25:24.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.247+0000 7f4f87e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f88104280 0x7f4f8819a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.90:36244/0 (socket says 172.21.15.90:36244) 2024-04-24T09:25:24.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.247+0000 7f4f87e00640 1 -- 172.21.15.90:0/3765312486 learned_addr learned my addr 172.21.15.90:0/3765312486 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:25:24.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.247+0000 7f4f87400640 1 --2- 172.21.15.90:0/3765312486 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f88105470 0x7f4f8819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:25:24.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.247+0000 7f4f8d200640 1 --2- 172.21.15.90:0/3765312486 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f88105e30 0x7f4f881a1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:25:24.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.247+0000 7f4f87e00640 1 -- 172.21.15.90:0/3765312486 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f88105e30 msgr2=0x7f4f881a1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:24.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.247+0000 7f4f87e00640 1 --2- 172.21.15.90:0/3765312486 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f88105e30 0x7f4f881a1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:24.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.247+0000 7f4f87e00640 1 -- 172.21.15.90:0/3765312486 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f88105470 msgr2=0x7f4f8819ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:24.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.247+0000 7f4f87e00640 1 --2- 172.21.15.90:0/3765312486 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f88105470 0x7f4f8819ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:24.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.247+0000 7f4f87e00640 1 -- 172.21.15.90:0/3765312486 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f881a2200 con 0x7f4f88104280 2024-04-24T09:25:24.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.248+0000 7f4f87e00640 1 --2- 172.21.15.90:0/3765312486 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f88104280 0x7f4f8819a5e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4f7800e9c0 tx=0x7f4f7800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:25:24.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.248+0000 7f4f84c00640 1 -- 172.21.15.90:0/3765312486 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f7800cd80 con 0x7f4f88104280 2024-04-24T09:25:24.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.248+0000 7f4f8f200640 1 -- 172.21.15.90:0/3765312486 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f881a24f0 con 0x7f4f88104280 2024-04-24T09:25:24.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.248+0000 7f4f84c00640 1 -- 172.21.15.90:0/3765312486 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f780035f0 con 0x7f4f88104280 2024-04-24T09:25:24.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.248+0000 7f4f84c00640 1 -- 172.21.15.90:0/3765312486 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f78010690 con 0x7f4f88104280 2024-04-24T09:25:24.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.248+0000 7f4f8f200640 1 -- 172.21.15.90:0/3765312486 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f881a29c0 con 0x7f4f88104280 2024-04-24T09:25:24.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.249+0000 7f4f8f200640 1 -- 172.21.15.90:0/3765312486 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f8810b5f0 con 0x7f4f88104280 2024-04-24T09:25:24.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.251+0000 7f4f84c00640 1 -- 172.21.15.90:0/3765312486 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f4f780037a0 con 0x7f4f88104280 2024-04-24T09:25:24.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.253+0000 7f4f84c00640 1 --2- 172.21.15.90:0/3765312486 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f4f6c0a3bb0 0x7f4f6c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:25:24.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.253+0000 7f4f84c00640 1 -- 172.21.15.90:0/3765312486 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f4f780d0780 con 0x7f4f88104280 2024-04-24T09:25:24.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.253+0000 7f4f87400640 1 --2- 172.21.15.90:0/3765312486 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f4f6c0a3bb0 0x7f4f6c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:25:24.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.254+0000 7f4f87400640 1 --2- 172.21.15.90:0/3765312486 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f4f6c0a3bb0 0x7f4f6c0a6070 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4f70031440 tx=0x7f4f700315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:25:24.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.255+0000 7f4f84c00640 1 -- 172.21.15.90:0/3765312486 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4f7809d120 con 0x7f4f88104280 2024-04-24T09:25:24.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.459+0000 7f4f8f200640 1 -- 172.21.15.90:0/3765312486 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4f88062c00 con 0x7f4f88104280 2024-04-24T09:25:24.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.460+0000 7f4f84c00640 1 -- 172.21.15.90:0/3765312486 <== mon.2 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f4f7809ca20 con 0x7f4f88104280 2024-04-24T09:25:24.460 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 1 2024-04-24T09:25:24.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.462+0000 7f4f8f200640 1 -- 172.21.15.90:0/3765312486 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f4f6c0a3bb0 msgr2=0x7f4f6c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:24.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.462+0000 7f4f8f200640 1 --2- 172.21.15.90:0/3765312486 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f4f6c0a3bb0 0x7f4f6c0a6070 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4f70031440 tx=0x7f4f700315b0 comp rx=0 tx=0).stop 2024-04-24T09:25:24.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.462+0000 7f4f8f200640 1 -- 172.21.15.90:0/3765312486 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f88104280 msgr2=0x7f4f8819a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:24.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.462+0000 7f4f8f200640 1 --2- 172.21.15.90:0/3765312486 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f88104280 0x7f4f8819a5e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4f7800e9c0 tx=0x7f4f7800ee80 comp rx=0 tx=0).stop 2024-04-24T09:25:24.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.463+0000 7f4f8f200640 1 -- 172.21.15.90:0/3765312486 shutdown_connections 2024-04-24T09:25:24.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.463+0000 7f4f8f200640 1 --2- 172.21.15.90:0/3765312486 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f4f6c0a3bb0 0x7f4f6c0a6070 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:24.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.463+0000 7f4f8f200640 1 --2- 172.21.15.90:0/3765312486 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f88105e30 0x7f4f881a1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:24.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.463+0000 7f4f8f200640 1 --2- 172.21.15.90:0/3765312486 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f88105470 0x7f4f8819ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:24.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.463+0000 7f4f8f200640 1 --2- 172.21.15.90:0/3765312486 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f88104280 0x7f4f8819a5e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:24.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.463+0000 7f4f8f200640 1 -- 172.21.15.90:0/3765312486 >> 172.21.15.90:0/3765312486 conn(0x7f4f880ff7f0 msgr2=0x7f4f881087c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:25:24.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.463+0000 7f4f8f200640 1 -- 172.21.15.90:0/3765312486 shutdown_connections 2024-04-24T09:25:24.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.463+0000 7f4f8f200640 1 -- 172.21.15.90:0/3765312486 wait complete. 2024-04-24T09:25:24.483 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T09:25:24.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.694+0000 7f716d800640 1 -- 172.21.15.90:0/2093810791 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7168104140 msgr2=0x7f7168104560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:24.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.694+0000 7f716d800640 1 --2- 172.21.15.90:0/2093810791 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7168104140 0x7f7168104560 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7158009be0 tx=0x7f715802a6c0 comp rx=0 tx=0).stop 2024-04-24T09:25:24.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.694+0000 7f716d800640 1 -- 172.21.15.90:0/2093810791 shutdown_connections 2024-04-24T09:25:24.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.694+0000 7f716d800640 1 --2- 172.21.15.90:0/2093810791 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7168105cf0 0x7f716810c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:24.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.694+0000 7f716d800640 1 --2- 172.21.15.90:0/2093810791 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7168105330 0x7f71681057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:24.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.694+0000 7f716d800640 1 --2- 172.21.15.90:0/2093810791 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7168104140 0x7f7168104560 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:24.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.694+0000 7f716d800640 1 -- 172.21.15.90:0/2093810791 >> 172.21.15.90:0/2093810791 conn(0x7f71680ff6b0 msgr2=0x7f7168101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:25:24.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.694+0000 7f716d800640 1 -- 172.21.15.90:0/2093810791 shutdown_connections 2024-04-24T09:25:24.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.694+0000 7f716d800640 1 -- 172.21.15.90:0/2093810791 wait complete. 2024-04-24T09:25:24.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.695+0000 7f716d800640 1 Processor -- start 2024-04-24T09:25:24.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.695+0000 7f716d800640 1 -- start start 2024-04-24T09:25:24.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.696+0000 7f716d800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7168104140 0x7f716819a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:25:24.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.696+0000 7f716d800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7168105330 0x7f716819a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:25:24.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.696+0000 7f716d800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7168105cf0 0x7f71681a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:25:24.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.696+0000 7f716d800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f716810fd80 con 0x7f7168104140 2024-04-24T09:25:24.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.696+0000 7f716d800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f716810fc00 con 0x7f7168105330 2024-04-24T09:25:24.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.696+0000 7f716d800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f716810ff00 con 0x7f7168105cf0 2024-04-24T09:25:24.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.696+0000 7f7167e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7168104140 0x7f716819a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:25:24.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.696+0000 7f7167e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7168104140 0x7f716819a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47362/0 (socket says 172.21.15.90:47362) 2024-04-24T09:25:24.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.696+0000 7f7167e00640 1 -- 172.21.15.90:0/32024455 learned_addr learned my addr 172.21.15.90:0/32024455 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:25:24.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.696+0000 7f715f400640 1 --2- 172.21.15.90:0/32024455 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7168105330 0x7f716819a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:25:24.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.696+0000 7f716ce00640 1 --2- 172.21.15.90:0/32024455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7168105cf0 0x7f71681a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:25:24.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.697+0000 7f7167e00640 1 -- 172.21.15.90:0/32024455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7168105cf0 msgr2=0x7f71681a19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:24.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.697+0000 7f7167e00640 1 --2- 172.21.15.90:0/32024455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7168105cf0 0x7f71681a19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:24.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.697+0000 7f7167e00640 1 -- 172.21.15.90:0/32024455 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7168105330 msgr2=0x7f716819a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:24.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.697+0000 7f7167e00640 1 --2- 172.21.15.90:0/32024455 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7168105330 0x7f716819a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:24.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.697+0000 7f7167e00640 1 -- 172.21.15.90:0/32024455 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71681a2050 con 0x7f7168104140 2024-04-24T09:25:24.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.697+0000 7f715f400640 1 --2- 172.21.15.90:0/32024455 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7168105330 0x7f716819a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:25:24.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.697+0000 7f7167e00640 1 --2- 172.21.15.90:0/32024455 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7168104140 0x7f716819a430 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7158030960 tx=0x7f71580305b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:25:24.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.697+0000 7f7165600640 1 -- 172.21.15.90:0/32024455 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7158041020 con 0x7f7168104140 2024-04-24T09:25:24.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.697+0000 7f7165600640 1 -- 172.21.15.90:0/32024455 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7158037040 con 0x7f7168104140 2024-04-24T09:25:24.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.697+0000 7f716d800640 1 -- 172.21.15.90:0/32024455 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71681a22e0 con 0x7f7168104140 2024-04-24T09:25:24.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.697+0000 7f7165600640 1 -- 172.21.15.90:0/32024455 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7158036070 con 0x7f7168104140 2024-04-24T09:25:24.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.698+0000 7f716d800640 1 -- 172.21.15.90:0/32024455 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71681a27d0 con 0x7f7168104140 2024-04-24T09:25:24.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.699+0000 7f716d800640 1 -- 172.21.15.90:0/32024455 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f716810b4e0 con 0x7f7168104140 2024-04-24T09:25:24.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.701+0000 7f7165600640 1 -- 172.21.15.90:0/32024455 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f7158044020 con 0x7f7168104140 2024-04-24T09:25:24.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.702+0000 7f7165600640 1 --2- 172.21.15.90:0/32024455 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f714c0a3bb0 0x7f714c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:25:24.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.702+0000 7f7165600640 1 -- 172.21.15.90:0/32024455 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f71580f0f40 con 0x7f7168104140 2024-04-24T09:25:24.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.702+0000 7f715f400640 1 --2- 172.21.15.90:0/32024455 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f714c0a3bb0 0x7f714c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:25:24.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.703+0000 7f715f400640 1 --2- 172.21.15.90:0/32024455 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f714c0a3bb0 0x7f714c0a6070 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f71500095f0 tx=0x7f7150005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:25:24.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.706+0000 7f7165600640 1 -- 172.21.15.90:0/32024455 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f715804b020 con 0x7f7168104140 2024-04-24T09:25:24.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.912+0000 7f716d800640 1 -- 172.21.15.90:0/32024455 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7168062c00 con 0x7f7168104140 2024-04-24T09:25:24.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.913+0000 7f7165600640 1 -- 172.21.15.90:0/32024455 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f71580bd8e0 con 0x7f7168104140 2024-04-24T09:25:24.913 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 32 2024-04-24T09:25:24.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.916+0000 7f716d800640 1 -- 172.21.15.90:0/32024455 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f714c0a3bb0 msgr2=0x7f714c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:24.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.916+0000 7f716d800640 1 --2- 172.21.15.90:0/32024455 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f714c0a3bb0 0x7f714c0a6070 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f71500095f0 tx=0x7f7150005f90 comp rx=0 tx=0).stop 2024-04-24T09:25:24.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.916+0000 7f716d800640 1 -- 172.21.15.90:0/32024455 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7168104140 msgr2=0x7f716819a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:24.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.916+0000 7f716d800640 1 --2- 172.21.15.90:0/32024455 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7168104140 0x7f716819a430 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7158030960 tx=0x7f71580305b0 comp rx=0 tx=0).stop 2024-04-24T09:25:24.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.916+0000 7f716d800640 1 -- 172.21.15.90:0/32024455 shutdown_connections 2024-04-24T09:25:24.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.916+0000 7f716d800640 1 --2- 172.21.15.90:0/32024455 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f714c0a3bb0 0x7f714c0a6070 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:24.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.916+0000 7f716d800640 1 --2- 172.21.15.90:0/32024455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7168105cf0 0x7f71681a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:24.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.916+0000 7f716d800640 1 --2- 172.21.15.90:0/32024455 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7168105330 0x7f716819a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:24.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.916+0000 7f716d800640 1 --2- 172.21.15.90:0/32024455 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7168104140 0x7f716819a430 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:24.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.916+0000 7f716d800640 1 -- 172.21.15.90:0/32024455 >> 172.21.15.90:0/32024455 conn(0x7f71680ff6b0 msgr2=0x7f71681086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:25:24.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.916+0000 7f716d800640 1 -- 172.21.15.90:0/32024455 shutdown_connections 2024-04-24T09:25:24.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:24.916+0000 7f716d800640 1 -- 172.21.15.90:0/32024455 wait complete. 2024-04-24T09:25:24.936 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T09:25:25.007 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:24 smithi090 ceph-mon[27825]: pgmap v511: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 511 B/s wr, 10 op/s 2024-04-24T09:25:25.007 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:24 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/4228129706' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:25:25.008 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:24 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/3765312486' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:25:25.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:24 smithi149 ceph-mon[32592]: pgmap v511: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 511 B/s wr, 10 op/s 2024-04-24T09:25:25.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:24 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/4228129706' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:25:25.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:24 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/3765312486' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:25:25.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.145+0000 7f7a01200640 1 -- 172.21.15.90:0/3219718936 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f79fc105330 msgr2=0x7f79fc1057b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:25.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.145+0000 7f7a01200640 1 --2- 172.21.15.90:0/3219718936 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f79fc105330 0x7f79fc1057b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f79e8009900 tx=0x7f79e802a3f0 comp rx=0 tx=0).stop 2024-04-24T09:25:25.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.145+0000 7f7a01200640 1 -- 172.21.15.90:0/3219718936 shutdown_connections 2024-04-24T09:25:25.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.145+0000 7f7a01200640 1 --2- 172.21.15.90:0/3219718936 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79fc105cf0 0x7f79fc10c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:25.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.145+0000 7f7a01200640 1 --2- 172.21.15.90:0/3219718936 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f79fc105330 0x7f79fc1057b0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:25.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.145+0000 7f7a01200640 1 --2- 172.21.15.90:0/3219718936 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79fc104140 0x7f79fc104560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:25.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.145+0000 7f7a01200640 1 -- 172.21.15.90:0/3219718936 >> 172.21.15.90:0/3219718936 conn(0x7f79fc0ff6b0 msgr2=0x7f79fc101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:25:25.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.145+0000 7f7a01200640 1 -- 172.21.15.90:0/3219718936 shutdown_connections 2024-04-24T09:25:25.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.145+0000 7f7a01200640 1 -- 172.21.15.90:0/3219718936 wait complete. 2024-04-24T09:25:25.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.146+0000 7f7a01200640 1 Processor -- start 2024-04-24T09:25:25.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.147+0000 7f7a01200640 1 -- start start 2024-04-24T09:25:25.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.147+0000 7f7a01200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79fc104140 0x7f79fc19a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:25:25.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.147+0000 7f7a01200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f79fc105330 0x7f79fc19a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:25:25.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.147+0000 7f7a01200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79fc105cf0 0x7f79fc1a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:25:25.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.147+0000 7f7a01200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f79fc10fe20 con 0x7f79fc105cf0 2024-04-24T09:25:25.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.147+0000 7f7a01200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f79fc10fca0 con 0x7f79fc104140 2024-04-24T09:25:25.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.147+0000 7f7a01200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f79fc10ffa0 con 0x7f79fc105330 2024-04-24T09:25:25.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.147+0000 7f79faa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f79fc105330 0x7f79fc19a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:25:25.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.147+0000 7f79faa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f79fc105330 0x7f79fc19a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:43894/0 (socket says 172.21.15.90:43894) 2024-04-24T09:25:25.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.147+0000 7f79faa00640 1 -- 172.21.15.90:0/1555835401 learned_addr learned my addr 172.21.15.90:0/1555835401 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:25:25.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.147+0000 7f79fbe00640 1 --2- 172.21.15.90:0/1555835401 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79fc105cf0 0x7f79fc1a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:25:25.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.147+0000 7f79fb400640 1 --2- 172.21.15.90:0/1555835401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79fc104140 0x7f79fc19a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:25:25.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.148+0000 7f79faa00640 1 -- 172.21.15.90:0/1555835401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79fc104140 msgr2=0x7f79fc19a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:25.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.148+0000 7f79faa00640 1 --2- 172.21.15.90:0/1555835401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79fc104140 0x7f79fc19a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:25.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.148+0000 7f79faa00640 1 -- 172.21.15.90:0/1555835401 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79fc105cf0 msgr2=0x7f79fc1a19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:25.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.148+0000 7f79faa00640 1 --2- 172.21.15.90:0/1555835401 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79fc105cf0 0x7f79fc1a19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:25.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.148+0000 7f79faa00640 1 -- 172.21.15.90:0/1555835401 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79fc1a2050 con 0x7f79fc105330 2024-04-24T09:25:25.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.148+0000 7f79faa00640 1 --2- 172.21.15.90:0/1555835401 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f79fc105330 0x7f79fc19a970 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f79e8031440 tx=0x7f79e8032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:25:25.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.148+0000 7f79f3e00640 1 -- 172.21.15.90:0/1555835401 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e803f030 con 0x7f79fc105330 2024-04-24T09:25:25.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.149+0000 7f7a01200640 1 -- 172.21.15.90:0/1555835401 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79fc1a22e0 con 0x7f79fc105330 2024-04-24T09:25:25.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.149+0000 7f79f3e00640 1 -- 172.21.15.90:0/1555835401 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79e8003ce0 con 0x7f79fc105330 2024-04-24T09:25:25.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.149+0000 7f79f3e00640 1 -- 172.21.15.90:0/1555835401 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e8035070 con 0x7f79fc105330 2024-04-24T09:25:25.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.149+0000 7f7a01200640 1 -- 172.21.15.90:0/1555835401 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79fc1a2820 con 0x7f79fc105330 2024-04-24T09:25:25.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.150+0000 7f7a01200640 1 -- 172.21.15.90:0/1555835401 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79fc10b4b0 con 0x7f79fc105330 2024-04-24T09:25:25.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.152+0000 7f79f3e00640 1 -- 172.21.15.90:0/1555835401 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f79e8042030 con 0x7f79fc105330 2024-04-24T09:25:25.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.153+0000 7f79f3e00640 1 --2- 172.21.15.90:0/1555835401 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f79e00a3bb0 0x7f79e00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:25:25.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.154+0000 7f79f3e00640 1 -- 172.21.15.90:0/1555835401 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f79e80f0cf0 con 0x7f79fc105330 2024-04-24T09:25:25.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.154+0000 7f79fb400640 1 --2- 172.21.15.90:0/1555835401 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f79e00a3bb0 0x7f79e00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:25:25.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.154+0000 7f79fb400640 1 --2- 172.21.15.90:0/1555835401 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f79e00a3bb0 0x7f79e00a6070 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f79fc105190 tx=0x7f79ec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:25:25.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.156+0000 7f79f3e00640 1 -- 172.21.15.90:0/1555835401 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f79e80bd690 con 0x7f79fc105330 2024-04-24T09:25:25.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:24 smithi114 ceph-mon[32501]: pgmap v511: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 511 B/s wr, 10 op/s 2024-04-24T09:25:25.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:24 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/4228129706' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:25:25.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:24 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/3765312486' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:25:25.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.358+0000 7f7a01200640 1 -- 172.21.15.90:0/1555835401 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f79fc02d820 con 0x7f79fc105330 2024-04-24T09:25:25.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.359+0000 7f79f3e00640 1 -- 172.21.15.90:0/1555835401 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f79e80bcf90 con 0x7f79fc105330 2024-04-24T09:25:25.359 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 64 2024-04-24T09:25:25.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.362+0000 7f7a01200640 1 -- 172.21.15.90:0/1555835401 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f79e00a3bb0 msgr2=0x7f79e00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:25.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.362+0000 7f7a01200640 1 --2- 172.21.15.90:0/1555835401 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f79e00a3bb0 0x7f79e00a6070 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f79fc105190 tx=0x7f79ec005f90 comp rx=0 tx=0).stop 2024-04-24T09:25:25.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.362+0000 7f7a01200640 1 -- 172.21.15.90:0/1555835401 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f79fc105330 msgr2=0x7f79fc19a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:25.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.362+0000 7f7a01200640 1 --2- 172.21.15.90:0/1555835401 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f79fc105330 0x7f79fc19a970 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f79e8031440 tx=0x7f79e8032ce0 comp rx=0 tx=0).stop 2024-04-24T09:25:25.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.362+0000 7f7a01200640 1 -- 172.21.15.90:0/1555835401 shutdown_connections 2024-04-24T09:25:25.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.362+0000 7f7a01200640 1 --2- 172.21.15.90:0/1555835401 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f79e00a3bb0 0x7f79e00a6070 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:25.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.362+0000 7f7a01200640 1 --2- 172.21.15.90:0/1555835401 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79fc105cf0 0x7f79fc1a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:25.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.362+0000 7f7a01200640 1 --2- 172.21.15.90:0/1555835401 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f79fc105330 0x7f79fc19a970 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:25.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.362+0000 7f7a01200640 1 --2- 172.21.15.90:0/1555835401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79fc104140 0x7f79fc19a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:25.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.363+0000 7f7a01200640 1 -- 172.21.15.90:0/1555835401 >> 172.21.15.90:0/1555835401 conn(0x7f79fc0ff6b0 msgr2=0x7f79fc108680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:25:25.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.363+0000 7f7a01200640 1 -- 172.21.15.90:0/1555835401 shutdown_connections 2024-04-24T09:25:25.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.363+0000 7f7a01200640 1 -- 172.21.15.90:0/1555835401 wait complete. 2024-04-24T09:25:25.382 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T09:25:25.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.588+0000 7f3abfe00640 1 -- 172.21.15.90:0/3845044563 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ac00769c0 msgr2=0x7f3ac0076de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:25.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.589+0000 7f3abfe00640 1 --2- 172.21.15.90:0/3845044563 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ac00769c0 0x7f3ac0076de0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0009ca0 tx=0x7f3ab002a8c0 comp rx=0 tx=0).stop 2024-04-24T09:25:25.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.589+0000 7f3abfe00640 1 -- 172.21.15.90:0/3845044563 shutdown_connections 2024-04-24T09:25:25.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.589+0000 7f3abfe00640 1 --2- 172.21.15.90:0/3845044563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ac00785a0 0x7f3ac0078e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:25.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.589+0000 7f3abfe00640 1 --2- 172.21.15.90:0/3845044563 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ac0077c00 0x7f3ac0078060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:25.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.589+0000 7f3abfe00640 1 --2- 172.21.15.90:0/3845044563 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ac00769c0 0x7f3ac0076de0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:25.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.589+0000 7f3abfe00640 1 -- 172.21.15.90:0/3845044563 >> 172.21.15.90:0/3845044563 conn(0x7f3ac00ff6b0 msgr2=0x7f3ac0101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:25:25.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.589+0000 7f3abfe00640 1 -- 172.21.15.90:0/3845044563 shutdown_connections 2024-04-24T09:25:25.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.589+0000 7f3abfe00640 1 -- 172.21.15.90:0/3845044563 wait complete. 2024-04-24T09:25:25.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.590+0000 7f3abfe00640 1 Processor -- start 2024-04-24T09:25:25.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.590+0000 7f3abfe00640 1 -- start start 2024-04-24T09:25:25.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.590+0000 7f3abfe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ac00769c0 0x7f3ac019e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:25:25.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.590+0000 7f3abfe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ac0077c00 0x7f3ac019eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:25:25.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.590+0000 7f3abfe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ac00785a0 0x7f3ac01a5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:25:25.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.590+0000 7f3abfe00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac01141d0 con 0x7f3ac00769c0 2024-04-24T09:25:25.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.590+0000 7f3abfe00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac0114050 con 0x7f3ac0077c00 2024-04-24T09:25:25.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.590+0000 7f3abfe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac0114350 con 0x7f3ac00785a0 2024-04-24T09:25:25.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.591+0000 7f3abea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ac00769c0 0x7f3ac019e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:25:25.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.591+0000 7f3abea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ac00769c0 0x7f3ac019e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47402/0 (socket says 172.21.15.90:47402) 2024-04-24T09:25:25.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.591+0000 7f3abea00640 1 -- 172.21.15.90:0/1696921352 learned_addr learned my addr 172.21.15.90:0/1696921352 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:25:25.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.591+0000 7f3abe000640 1 --2- 172.21.15.90:0/1696921352 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ac0077c00 0x7f3ac019eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:25:25.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.591+0000 7f3abf400640 1 --2- 172.21.15.90:0/1696921352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ac00785a0 0x7f3ac01a5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:25:25.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.591+0000 7f3abea00640 1 -- 172.21.15.90:0/1696921352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ac00785a0 msgr2=0x7f3ac01a5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:25.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.591+0000 7f3abea00640 1 --2- 172.21.15.90:0/1696921352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ac00785a0 0x7f3ac01a5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:25.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.591+0000 7f3abea00640 1 -- 172.21.15.90:0/1696921352 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ac0077c00 msgr2=0x7f3ac019eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:25.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.591+0000 7f3abea00640 1 --2- 172.21.15.90:0/1696921352 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ac0077c00 0x7f3ac019eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:25.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.591+0000 7f3abea00640 1 -- 172.21.15.90:0/1696921352 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ac01a6480 con 0x7f3ac00769c0 2024-04-24T09:25:25.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.591+0000 7f3abf400640 1 --2- 172.21.15.90:0/1696921352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ac00785a0 0x7f3ac01a5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:25:25.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.591+0000 7f3abea00640 1 --2- 172.21.15.90:0/1696921352 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ac00769c0 0x7f3ac019e860 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0000f80 tx=0x7f3ab0031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:25:25.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.592+0000 7f3ab7400640 1 -- 172.21.15.90:0/1696921352 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab002fa20 con 0x7f3ac00769c0 2024-04-24T09:25:25.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.592+0000 7f3ab7400640 1 -- 172.21.15.90:0/1696921352 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ab002fbc0 con 0x7f3ac00769c0 2024-04-24T09:25:25.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.592+0000 7f3ab7400640 1 -- 172.21.15.90:0/1696921352 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab003a6f0 con 0x7f3ac00769c0 2024-04-24T09:25:25.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.592+0000 7f3abfe00640 1 -- 172.21.15.90:0/1696921352 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ac01a6710 con 0x7f3ac00769c0 2024-04-24T09:25:25.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.592+0000 7f3abfe00640 1 -- 172.21.15.90:0/1696921352 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ac01a6cb0 con 0x7f3ac00769c0 2024-04-24T09:25:25.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.593+0000 7f3abfe00640 1 -- 172.21.15.90:0/1696921352 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ac0110ab0 con 0x7f3ac00769c0 2024-04-24T09:25:25.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.595+0000 7f3ab7400640 1 -- 172.21.15.90:0/1696921352 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f3ab003a890 con 0x7f3ac00769c0 2024-04-24T09:25:25.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.596+0000 7f3ab7400640 1 --2- 172.21.15.90:0/1696921352 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f3aa40a3c30 0x7f3aa40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:25:25.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.596+0000 7f3ab7400640 1 -- 172.21.15.90:0/1696921352 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f3ab00f1930 con 0x7f3ac00769c0 2024-04-24T09:25:25.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.597+0000 7f3abe000640 1 --2- 172.21.15.90:0/1696921352 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f3aa40a3c30 0x7f3aa40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:25:25.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.597+0000 7f3abe000640 1 --2- 172.21.15.90:0/1696921352 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f3aa40a3c30 0x7f3aa40a60f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3ac019fd70 tx=0x7f3aa8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:25:25.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.600+0000 7f3ab7400640 1 -- 172.21.15.90:0/1696921352 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3ab00be360 con 0x7f3ac00769c0 2024-04-24T09:25:25.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.825+0000 7f3abfe00640 1 -- 172.21.15.90:0/1696921352 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3ac002d620 con 0x7f3ac00769c0 2024-04-24T09:25:25.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.826+0000 7f3ab7400640 1 -- 172.21.15.90:0/1696921352 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) ==== 81+0+590 (secure 0 0 0) 0x7f3ab00bdc60 con 0x7f3ac00769c0 2024-04-24T09:25:25.826 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/3090852580 2024-04-25T09:10:57.377655+0000 2024-04-24T09:25:25.826 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/687197394 2024-04-25T09:10:57.377655+0000 2024-04-24T09:25:25.826 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/1809038230 2024-04-25T09:10:57.377655+0000 2024-04-24T09:25:25.826 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/1809038230 2024-04-25T09:10:57.377655+0000 2024-04-24T09:25:25.827 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2964149071 2024-04-25T09:10:22.599674+0000 2024-04-24T09:25:25.827 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/3617388113 2024-04-25T09:10:22.599674+0000 2024-04-24T09:25:25.827 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/3089140842 2024-04-25T09:10:22.599674+0000 2024-04-24T09:25:25.827 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/3089140842 2024-04-25T09:10:22.599674+0000 2024-04-24T09:25:25.827 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2840416149 2024-04-25T09:10:22.599674+0000 2024-04-24T09:25:25.827 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/305432710 2024-04-25T09:10:57.377655+0000 2024-04-24T09:25:25.828 INFO:teuthology.orchestra.run.smithi090.stderr:listed 10 entries 2024-04-24T09:25:25.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.828+0000 7f3abfe00640 1 -- 172.21.15.90:0/1696921352 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f3aa40a3c30 msgr2=0x7f3aa40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:25.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.828+0000 7f3abfe00640 1 --2- 172.21.15.90:0/1696921352 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f3aa40a3c30 0x7f3aa40a60f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3ac019fd70 tx=0x7f3aa8005f90 comp rx=0 tx=0).stop 2024-04-24T09:25:25.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.828+0000 7f3abfe00640 1 -- 172.21.15.90:0/1696921352 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ac00769c0 msgr2=0x7f3ac019e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:25.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.828+0000 7f3abfe00640 1 --2- 172.21.15.90:0/1696921352 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ac00769c0 0x7f3ac019e860 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0000f80 tx=0x7f3ab0031ee0 comp rx=0 tx=0).stop 2024-04-24T09:25:25.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.829+0000 7f3abfe00640 1 -- 172.21.15.90:0/1696921352 shutdown_connections 2024-04-24T09:25:25.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.829+0000 7f3abfe00640 1 --2- 172.21.15.90:0/1696921352 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f3aa40a3c30 0x7f3aa40a60f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:25.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.829+0000 7f3abfe00640 1 --2- 172.21.15.90:0/1696921352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ac00785a0 0x7f3ac01a5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:25.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.829+0000 7f3abfe00640 1 --2- 172.21.15.90:0/1696921352 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ac0077c00 0x7f3ac019eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:25.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.829+0000 7f3abfe00640 1 --2- 172.21.15.90:0/1696921352 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ac00769c0 0x7f3ac019e860 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:25.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.829+0000 7f3abfe00640 1 -- 172.21.15.90:0/1696921352 >> 172.21.15.90:0/1696921352 conn(0x7f3ac00ff6b0 msgr2=0x7f3ac01011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:25:25.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.829+0000 7f3abfe00640 1 -- 172.21.15.90:0/1696921352 shutdown_connections 2024-04-24T09:25:25.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:25.829+0000 7f3abfe00640 1 -- 172.21.15.90:0/1696921352 wait complete. 2024-04-24T09:25:25.847 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-24T09:25:25.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:25:25.847 DEBUG:teuthology.orchestra.run.smithi090:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-24T09:25:26.075 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi090.front.sepia.ceph.com 2024-04-24T09:25:26.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:25:26.075 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T09:25:26.076 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:25 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/32024455' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:25:26.076 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:25 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/1555835401' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:25:26.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:25 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/32024455' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:25:26.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:25 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/1555835401' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:25:26.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:25:26.141 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list 2024-04-24T09:25:26.175 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:25 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/32024455' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:25:26.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:25 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/1555835401' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:25:26.177 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T09:25:26.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:25:26.178 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T09:25:26.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:25:26.280 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link delete ceph-brx 2024-04-24T09:25:26.380 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-24T09:25:26.392 INFO:tasks.cephadm:Teardown begin 2024-04-24T09:25:26.392 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-24T09:25:26.393 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:25:26.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.568+0000 7fc623200640 1 -- 172.21.15.90:0/3591179748 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc624108480 msgr2=0x7fc6241088a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:26.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.568+0000 7fc623200640 1 --2- 172.21.15.90:0/3591179748 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc624108480 0x7fc6241088a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc614009900 tx=0x7fc61402a3f0 comp rx=0 tx=0).stop 2024-04-24T09:25:26.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.568+0000 7fc623200640 1 -- 172.21.15.90:0/3591179748 shutdown_connections 2024-04-24T09:25:26.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.568+0000 7fc623200640 1 --2- 172.21.15.90:0/3591179748 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc62410a030 0x7fc6241108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:26.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.568+0000 7fc623200640 1 --2- 172.21.15.90:0/3591179748 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc624109670 0x7fc624109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:26.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.568+0000 7fc623200640 1 --2- 172.21.15.90:0/3591179748 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc624108480 0x7fc6241088a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:26.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.568+0000 7fc623200640 1 -- 172.21.15.90:0/3591179748 >> 172.21.15.90:0/3591179748 conn(0x7fc624077440 msgr2=0x7fc624079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:25:26.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.569+0000 7fc623200640 1 -- 172.21.15.90:0/3591179748 shutdown_connections 2024-04-24T09:25:26.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.569+0000 7fc623200640 1 -- 172.21.15.90:0/3591179748 wait complete. 2024-04-24T09:25:26.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.570+0000 7fc623200640 1 Processor -- start 2024-04-24T09:25:26.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.570+0000 7fc623200640 1 -- start start 2024-04-24T09:25:26.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.570+0000 7fc623200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc624108480 0x7fc62419e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:25:26.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.570+0000 7fc623200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc624109670 0x7fc62419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:25:26.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.570+0000 7fc623200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc62410a030 0x7fc6241a5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:25:26.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.570+0000 7fc623200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc624114110 con 0x7fc624108480 2024-04-24T09:25:26.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.570+0000 7fc623200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc624113f90 con 0x7fc624109670 2024-04-24T09:25:26.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.570+0000 7fc623200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc624114290 con 0x7fc62410a030 2024-04-24T09:25:26.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.571+0000 7fc621e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc624108480 0x7fc62419e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:25:26.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.571+0000 7fc61be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc624109670 0x7fc62419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:25:26.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.571+0000 7fc621e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc624108480 0x7fc62419e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40026/0 (socket says 172.21.15.90:40026) 2024-04-24T09:25:26.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.571+0000 7fc61be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc624109670 0x7fc62419ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.90:53600/0 (socket says 172.21.15.90:53600) 2024-04-24T09:25:26.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.571+0000 7fc61be00640 1 -- 172.21.15.90:0/4011959531 learned_addr learned my addr 172.21.15.90:0/4011959531 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:25:26.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.571+0000 7fc622800640 1 --2- 172.21.15.90:0/4011959531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc62410a030 0x7fc6241a5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:25:26.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.571+0000 7fc61be00640 1 -- 172.21.15.90:0/4011959531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc62410a030 msgr2=0x7fc6241a5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:26.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.571+0000 7fc61be00640 1 --2- 172.21.15.90:0/4011959531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc62410a030 0x7fc6241a5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:26.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.571+0000 7fc61be00640 1 -- 172.21.15.90:0/4011959531 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc624108480 msgr2=0x7fc62419e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:26.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.571+0000 7fc61be00640 1 --2- 172.21.15.90:0/4011959531 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc624108480 0x7fc62419e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:26.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.571+0000 7fc61be00640 1 -- 172.21.15.90:0/4011959531 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6241a63c0 con 0x7fc624109670 2024-04-24T09:25:26.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.571+0000 7fc622800640 1 --2- 172.21.15.90:0/4011959531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc62410a030 0x7fc6241a5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:25:26.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.571+0000 7fc621e00640 1 --2- 172.21.15.90:0/4011959531 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc624108480 0x7fc62419e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:25:26.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.571+0000 7fc61be00640 1 --2- 172.21.15.90:0/4011959531 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc624109670 0x7fc62419ed30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc60c00e9f0 tx=0x7fc60c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:25:26.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.572+0000 7fc61aa00640 1 -- 172.21.15.90:0/4011959531 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60c00ccc0 con 0x7fc624109670 2024-04-24T09:25:26.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.572+0000 7fc61aa00640 1 -- 172.21.15.90:0/4011959531 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc60c00ce60 con 0x7fc624109670 2024-04-24T09:25:26.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.572+0000 7fc623200640 1 -- 172.21.15.90:0/4011959531 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6241a66b0 con 0x7fc624109670 2024-04-24T09:25:26.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.572+0000 7fc61aa00640 1 -- 172.21.15.90:0/4011959531 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60c010690 con 0x7fc624109670 2024-04-24T09:25:26.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.572+0000 7fc623200640 1 -- 172.21.15.90:0/4011959531 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6241a6bb0 con 0x7fc624109670 2024-04-24T09:25:26.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.573+0000 7fc623200640 1 -- 172.21.15.90:0/4011959531 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc62410f870 con 0x7fc624109670 2024-04-24T09:25:26.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.575+0000 7fc61aa00640 1 -- 172.21.15.90:0/4011959531 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fc60c0046c0 con 0x7fc624109670 2024-04-24T09:25:26.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.576+0000 7fc61aa00640 1 --2- 172.21.15.90:0/4011959531 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fc6080a3bb0 0x7fc6080a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:25:26.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.576+0000 7fc61aa00640 1 -- 172.21.15.90:0/4011959531 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fc60c0d07f0 con 0x7fc624109670 2024-04-24T09:25:26.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.576+0000 7fc621e00640 1 --2- 172.21.15.90:0/4011959531 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fc6080a3bb0 0x7fc6080a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:25:26.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.577+0000 7fc621e00640 1 --2- 172.21.15.90:0/4011959531 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fc6080a3bb0 0x7fc6080a6070 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc614005ab0 tx=0x7fc614000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:25:26.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.578+0000 7fc61aa00640 1 -- 172.21.15.90:0/4011959531 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc60c09d190 con 0x7fc624109670 2024-04-24T09:25:26.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.826+0000 7fc623200640 1 -- 172.21.15.90:0/4011959531 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc62402d870 con 0x7fc624109670 2024-04-24T09:25:26.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.827+0000 7fc61aa00640 1 -- 172.21.15.90:0/4011959531 <== mon.2 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11933 (secure 0 0 0) 0x7fc60c0bd0d0 con 0x7fc624109670 2024-04-24T09:25:26.828 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:25:26.828 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/227251251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":227251251},{"type":"v1","addr":"172.21.15.90:6837","nonce":227251251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/1904617971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":1904617971},{"type":"v1","addr":"172.21.15.90:6839","nonce":1904617971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T09:19:09.430875+0000","modified":"2024-04-24T09:24:03.813872+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24547,"mds_2":24545,"mds_3":24526,"mds_4":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6833/219377141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":219377141},{"type":"v1","addr":"172.21.15.114:6833","nonce":219377141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"j","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.90:6835/2396995634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2396995634},{"type":"v1","addr":"172.21.15.90:6835","nonce":2396995634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/779642417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":779642417},{"type":"v1","addr":"172.21.15.149:6833","nonce":779642417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6835/2565340800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":2565340800},{"type":"v1","addr":"172.21.15.114:6835","nonce":2565340800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6837/875324280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":875324280},{"type":"v1","addr":"172.21.15.114:6837","nonce":875324280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/3157375624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3157375624},{"type":"v1","addr":"172.21.15.149:6835","nonce":3157375624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.149:6837/2250310587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2250310587},{"type":"v1","addr":"172.21.15.149:6837","nonce":2250310587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.90:6841/397593398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":397593398},{"type":"v1","addr":"172.21.15.90:6841","nonce":397593398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.149:6839/652734263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":652734263},{"type":"v1","addr":"172.21.15.149:6839","nonce":652734263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6839/1025403710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":1025403710},{"type":"v1","addr":"172.21.15.114:6839","nonce":1025403710}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24545,24557,24446,24526,24547]},"id":1}]} 2024-04-24T09:25:26.829 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 31 2024-04-24T09:25:26.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.830+0000 7fc623200640 1 -- 172.21.15.90:0/4011959531 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fc6080a3bb0 msgr2=0x7fc6080a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:26.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.830+0000 7fc623200640 1 --2- 172.21.15.90:0/4011959531 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fc6080a3bb0 0x7fc6080a6070 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc614005ab0 tx=0x7fc614000bc0 comp rx=0 tx=0).stop 2024-04-24T09:25:26.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.830+0000 7fc623200640 1 -- 172.21.15.90:0/4011959531 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc624109670 msgr2=0x7fc62419ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:26.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.830+0000 7fc623200640 1 --2- 172.21.15.90:0/4011959531 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc624109670 0x7fc62419ed30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc60c00e9f0 tx=0x7fc60c00edf0 comp rx=0 tx=0).stop 2024-04-24T09:25:26.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.830+0000 7fc623200640 1 -- 172.21.15.90:0/4011959531 shutdown_connections 2024-04-24T09:25:26.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.830+0000 7fc623200640 1 --2- 172.21.15.90:0/4011959531 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fc6080a3bb0 0x7fc6080a6070 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:26.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.830+0000 7fc623200640 1 --2- 172.21.15.90:0/4011959531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc62410a030 0x7fc6241a5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:26.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.830+0000 7fc623200640 1 --2- 172.21.15.90:0/4011959531 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc624109670 0x7fc62419ed30 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:26.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.830+0000 7fc623200640 1 --2- 172.21.15.90:0/4011959531 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc624108480 0x7fc62419e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:26.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.830+0000 7fc623200640 1 -- 172.21.15.90:0/4011959531 >> 172.21.15.90:0/4011959531 conn(0x7fc624077440 msgr2=0x7fc62410ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:25:26.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.830+0000 7fc623200640 1 -- 172.21.15.90:0/4011959531 shutdown_connections 2024-04-24T09:25:26.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:26.830+0000 7fc623200640 1 -- 172.21.15.90:0/4011959531 wait complete. 2024-04-24T09:25:26.850 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:25:27.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.052+0000 7fc132000640 1 -- 172.21.15.90:0/2020475196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc12c109850 msgr2=0x7fc12c109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:27.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.052+0000 7fc132000640 1 --2- 172.21.15.90:0/2020475196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc12c109850 0x7fc12c109cd0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc114009900 tx=0x7fc11402a410 comp rx=0 tx=0).stop 2024-04-24T09:25:27.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.052+0000 7fc132000640 1 -- 172.21.15.90:0/2020475196 shutdown_connections 2024-04-24T09:25:27.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.052+0000 7fc132000640 1 --2- 172.21.15.90:0/2020475196 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc12c10a210 0x7fc12c110aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:27.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.052+0000 7fc132000640 1 --2- 172.21.15.90:0/2020475196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc12c109850 0x7fc12c109cd0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:27.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.052+0000 7fc132000640 1 --2- 172.21.15.90:0/2020475196 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc12c108660 0x7fc12c108a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:27.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.052+0000 7fc132000640 1 -- 172.21.15.90:0/2020475196 >> 172.21.15.90:0/2020475196 conn(0x7fc12c077620 msgr2=0x7fc12c079a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:25:27.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.052+0000 7fc132000640 1 -- 172.21.15.90:0/2020475196 shutdown_connections 2024-04-24T09:25:27.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.052+0000 7fc132000640 1 -- 172.21.15.90:0/2020475196 wait complete. 2024-04-24T09:25:27.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.053+0000 7fc132000640 1 Processor -- start 2024-04-24T09:25:27.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.053+0000 7fc132000640 1 -- start start 2024-04-24T09:25:27.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.054+0000 7fc132000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc12c108660 0x7fc12c19e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:25:27.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.054+0000 7fc132000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc12c109850 0x7fc12c19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:25:27.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.054+0000 7fc132000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc12c10a210 0x7fc12c1a5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:25:27.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.054+0000 7fc132000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc12c114310 con 0x7fc12c109850 2024-04-24T09:25:27.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.054+0000 7fc132000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc12c114190 con 0x7fc12c10a210 2024-04-24T09:25:27.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.054+0000 7fc132000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc12c114490 con 0x7fc12c108660 2024-04-24T09:25:27.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.054+0000 7fc123e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc12c109850 0x7fc12c19ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:25:27.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.054+0000 7fc123e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc12c109850 0x7fc12c19ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40042/0 (socket says 172.21.15.90:40042) 2024-04-24T09:25:27.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.054+0000 7fc123e00640 1 -- 172.21.15.90:0/2038911741 learned_addr learned my addr 172.21.15.90:0/2038911741 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:25:27.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.054+0000 7fc12b400640 1 --2- 172.21.15.90:0/2038911741 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc12c108660 0x7fc12c19e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:25:27.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.054+0000 7fc12be00640 1 --2- 172.21.15.90:0/2038911741 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc12c10a210 0x7fc12c1a5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:25:27.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.054+0000 7fc123e00640 1 -- 172.21.15.90:0/2038911741 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc12c108660 msgr2=0x7fc12c19e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:27.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.054+0000 7fc123e00640 1 --2- 172.21.15.90:0/2038911741 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc12c108660 0x7fc12c19e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:27.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.054+0000 7fc123e00640 1 -- 172.21.15.90:0/2038911741 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc12c10a210 msgr2=0x7fc12c1a5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:27.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.054+0000 7fc123e00640 1 --2- 172.21.15.90:0/2038911741 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc12c10a210 0x7fc12c1a5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:27.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.054+0000 7fc123e00640 1 -- 172.21.15.90:0/2038911741 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc12c1a65e0 con 0x7fc12c109850 2024-04-24T09:25:27.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.054+0000 7fc12b400640 1 --2- 172.21.15.90:0/2038911741 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc12c108660 0x7fc12c19e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:25:27.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.055+0000 7fc123e00640 1 --2- 172.21.15.90:0/2038911741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc12c109850 0x7fc12c19ef00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc114031440 tx=0x7fc114032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:25:27.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.055+0000 7fc128c00640 1 -- 172.21.15.90:0/2038911741 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11403f030 con 0x7fc12c109850 2024-04-24T09:25:27.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.055+0000 7fc132000640 1 -- 172.21.15.90:0/2038911741 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc12c1a6870 con 0x7fc12c109850 2024-04-24T09:25:27.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.055+0000 7fc128c00640 1 -- 172.21.15.90:0/2038911741 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc114003ce0 con 0x7fc12c109850 2024-04-24T09:25:27.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.055+0000 7fc128c00640 1 -- 172.21.15.90:0/2038911741 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc114035070 con 0x7fc12c109850 2024-04-24T09:25:27.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.055+0000 7fc132000640 1 -- 172.21.15.90:0/2038911741 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc12c1a6d60 con 0x7fc12c109850 2024-04-24T09:25:27.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.056+0000 7fc132000640 1 -- 172.21.15.90:0/2038911741 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc12c10f920 con 0x7fc12c109850 2024-04-24T09:25:27.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.058+0000 7fc128c00640 1 -- 172.21.15.90:0/2038911741 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fc114042030 con 0x7fc12c109850 2024-04-24T09:25:27.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.060+0000 7fc128c00640 1 --2- 172.21.15.90:0/2038911741 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fc1100a3bb0 0x7fc1100a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:25:27.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.060+0000 7fc128c00640 1 -- 172.21.15.90:0/2038911741 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fc1140f0b20 con 0x7fc12c109850 2024-04-24T09:25:27.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.060+0000 7fc12b400640 1 --2- 172.21.15.90:0/2038911741 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fc1100a3bb0 0x7fc1100a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:25:27.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.061+0000 7fc12b400640 1 --2- 172.21.15.90:0/2038911741 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fc1100a3bb0 0x7fc1100a6070 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc12c1096b0 tx=0x7fc11c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:25:27.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.064+0000 7fc128c00640 1 -- 172.21.15.90:0/2038911741 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc1140bce70 con 0x7fc12c109850 2024-04-24T09:25:27.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:26 smithi090 ceph-mon[27825]: pgmap v512: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-24T09:25:27.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:26 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/1696921352' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T09:25:27.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:26 smithi149 ceph-mon[32592]: pgmap v512: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-24T09:25:27.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:26 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/1696921352' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T09:25:27.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:26 smithi114 ceph-mon[32501]: pgmap v512: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-24T09:25:27.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:26 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/1696921352' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T09:25:27.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.306+0000 7fc132000640 1 -- 172.21.15.90:0/2038911741 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc12c062c00 con 0x7fc12c109850 2024-04-24T09:25:27.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.307+0000 7fc128c00640 1 -- 172.21.15.90:0/2038911741 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11933 (secure 0 0 0) 0x7fc1140c1170 con 0x7fc12c109850 2024-04-24T09:25:27.308 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:25:27.308 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/227251251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":227251251},{"type":"v1","addr":"172.21.15.90:6837","nonce":227251251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/1904617971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":1904617971},{"type":"v1","addr":"172.21.15.90:6839","nonce":1904617971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T09:19:09.430875+0000","modified":"2024-04-24T09:24:03.813872+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24547,"mds_2":24545,"mds_3":24526,"mds_4":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6833/219377141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":219377141},{"type":"v1","addr":"172.21.15.114:6833","nonce":219377141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"j","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.90:6835/2396995634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2396995634},{"type":"v1","addr":"172.21.15.90:6835","nonce":2396995634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/779642417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":779642417},{"type":"v1","addr":"172.21.15.149:6833","nonce":779642417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6835/2565340800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":2565340800},{"type":"v1","addr":"172.21.15.114:6835","nonce":2565340800}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6837/875324280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":875324280},{"type":"v1","addr":"172.21.15.114:6837","nonce":875324280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/3157375624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3157375624},{"type":"v1","addr":"172.21.15.149:6835","nonce":3157375624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.149:6837/2250310587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2250310587},{"type":"v1","addr":"172.21.15.149:6837","nonce":2250310587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"a","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"172.21.15.90:6841/397593398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":397593398},{"type":"v1","addr":"172.21.15.90:6841","nonce":397593398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.149:6839/652734263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":652734263},{"type":"v1","addr":"172.21.15.149:6839","nonce":652734263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.114:6839/1025403710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":1025403710},{"type":"v1","addr":"172.21.15.114:6839","nonce":1025403710}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24557,"qdb_cluster":[24545,24557,24446,24526,24547]},"id":1}]} 2024-04-24T09:25:27.308 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 31 2024-04-24T09:25:27.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.310+0000 7fc132000640 1 -- 172.21.15.90:0/2038911741 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fc1100a3bb0 msgr2=0x7fc1100a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:27.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.310+0000 7fc132000640 1 --2- 172.21.15.90:0/2038911741 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fc1100a3bb0 0x7fc1100a6070 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc12c1096b0 tx=0x7fc11c005f90 comp rx=0 tx=0).stop 2024-04-24T09:25:27.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.310+0000 7fc132000640 1 -- 172.21.15.90:0/2038911741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc12c109850 msgr2=0x7fc12c19ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:27.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.310+0000 7fc132000640 1 --2- 172.21.15.90:0/2038911741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc12c109850 0x7fc12c19ef00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc114031440 tx=0x7fc114032ce0 comp rx=0 tx=0).stop 2024-04-24T09:25:27.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.310+0000 7fc132000640 1 -- 172.21.15.90:0/2038911741 shutdown_connections 2024-04-24T09:25:27.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.310+0000 7fc132000640 1 --2- 172.21.15.90:0/2038911741 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fc1100a3bb0 0x7fc1100a6070 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:27.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.310+0000 7fc132000640 1 --2- 172.21.15.90:0/2038911741 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc12c10a210 0x7fc12c1a5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:27.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.310+0000 7fc132000640 1 --2- 172.21.15.90:0/2038911741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc12c109850 0x7fc12c19ef00 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:27.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.310+0000 7fc132000640 1 --2- 172.21.15.90:0/2038911741 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc12c108660 0x7fc12c19e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:27.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.310+0000 7fc132000640 1 -- 172.21.15.90:0/2038911741 >> 172.21.15.90:0/2038911741 conn(0x7fc12c077620 msgr2=0x7fc12c10caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:25:27.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.310+0000 7fc132000640 1 -- 172.21.15.90:0/2038911741 shutdown_connections 2024-04-24T09:25:27.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.310+0000 7fc132000640 1 -- 172.21.15.90:0/2038911741 wait complete. 2024-04-24T09:25:27.329 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T09:25:27.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.515+0000 7fe692600640 1 -- 172.21.15.90:0/3180803566 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe68c10e600 msgr2=0x7fe68c114e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:27.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.515+0000 7fe692600640 1 --2- 172.21.15.90:0/3180803566 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe68c10e600 0x7fe68c114e90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe684009900 tx=0x7fe68402a3f0 comp rx=0 tx=0).stop 2024-04-24T09:25:27.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.515+0000 7fe692600640 1 -- 172.21.15.90:0/3180803566 shutdown_connections 2024-04-24T09:25:27.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.515+0000 7fe692600640 1 --2- 172.21.15.90:0/3180803566 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe68c10e600 0x7fe68c114e90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:27.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.515+0000 7fe692600640 1 --2- 172.21.15.90:0/3180803566 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe68c10dc40 0x7fe68c10e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:27.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.515+0000 7fe692600640 1 --2- 172.21.15.90:0/3180803566 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe68c10ca50 0x7fe68c10ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:27.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.515+0000 7fe692600640 1 -- 172.21.15.90:0/3180803566 >> 172.21.15.90:0/3180803566 conn(0x7fe68c06caf0 msgr2=0x7fe68c06cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:25:27.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.515+0000 7fe692600640 1 -- 172.21.15.90:0/3180803566 shutdown_connections 2024-04-24T09:25:27.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.515+0000 7fe692600640 1 -- 172.21.15.90:0/3180803566 wait complete. 2024-04-24T09:25:27.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.516+0000 7fe692600640 1 Processor -- start 2024-04-24T09:25:27.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.516+0000 7fe692600640 1 -- start start 2024-04-24T09:25:27.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.517+0000 7fe692600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe68c10ca50 0x7fe68c1a2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:25:27.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.517+0000 7fe692600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe68c10dc40 0x7fe68c1a32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:25:27.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.517+0000 7fe692600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe68c10e600 0x7fe68c1aa320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:25:27.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.517+0000 7fe692600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c118720 con 0x7fe68c10dc40 2024-04-24T09:25:27.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.517+0000 7fe692600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c1185a0 con 0x7fe68c10ca50 2024-04-24T09:25:27.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.517+0000 7fe692600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c1188a0 con 0x7fe68c10e600 2024-04-24T09:25:27.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.517+0000 7fe682a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe68c10dc40 0x7fe68c1a32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:25:27.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.517+0000 7fe68be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe68c10e600 0x7fe68c1aa320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:25:27.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.517+0000 7fe68be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe68c10e600 0x7fe68c1aa320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:51510/0 (socket says 172.21.15.90:51510) 2024-04-24T09:25:27.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.517+0000 7fe68b400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe68c10ca50 0x7fe68c1a2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:25:27.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.517+0000 7fe68be00640 1 -- 172.21.15.90:0/2179442711 learned_addr learned my addr 172.21.15.90:0/2179442711 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:25:27.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.517+0000 7fe682a00640 1 -- 172.21.15.90:0/2179442711 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe68c10e600 msgr2=0x7fe68c1aa320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:27.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.517+0000 7fe682a00640 1 --2- 172.21.15.90:0/2179442711 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe68c10e600 0x7fe68c1aa320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:27.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.517+0000 7fe682a00640 1 -- 172.21.15.90:0/2179442711 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe68c10ca50 msgr2=0x7fe68c1a2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:27.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.517+0000 7fe682a00640 1 --2- 172.21.15.90:0/2179442711 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe68c10ca50 0x7fe68c1a2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:27.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.517+0000 7fe682a00640 1 -- 172.21.15.90:0/2179442711 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe68c1aa990 con 0x7fe68c10dc40 2024-04-24T09:25:27.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.518+0000 7fe682a00640 1 --2- 172.21.15.90:0/2179442711 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe68c10dc40 0x7fe68c1a32b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe67400e9f0 tx=0x7fe67400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:25:27.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.518+0000 7fe688c00640 1 -- 172.21.15.90:0/2179442711 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67400ccc0 con 0x7fe68c10dc40 2024-04-24T09:25:27.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.518+0000 7fe692600640 1 -- 172.21.15.90:0/2179442711 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe68c1aac80 con 0x7fe68c10dc40 2024-04-24T09:25:27.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.518+0000 7fe688c00640 1 -- 172.21.15.90:0/2179442711 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe674010910 con 0x7fe68c10dc40 2024-04-24T09:25:27.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.518+0000 7fe688c00640 1 -- 172.21.15.90:0/2179442711 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe674018960 con 0x7fe68c10dc40 2024-04-24T09:25:27.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.518+0000 7fe692600640 1 -- 172.21.15.90:0/2179442711 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe68c1ab1d0 con 0x7fe68c10dc40 2024-04-24T09:25:27.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.519+0000 7fe692600640 1 -- 172.21.15.90:0/2179442711 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe68c113d10 con 0x7fe68c10dc40 2024-04-24T09:25:27.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.521+0000 7fe688c00640 1 -- 172.21.15.90:0/2179442711 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fe674010ac0 con 0x7fe68c10dc40 2024-04-24T09:25:27.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.523+0000 7fe688c00640 1 --2- 172.21.15.90:0/2179442711 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fe6700a3bb0 0x7fe6700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:25:27.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.523+0000 7fe688c00640 1 -- 172.21.15.90:0/2179442711 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fe6740d0c40 con 0x7fe68c10dc40 2024-04-24T09:25:27.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.523+0000 7fe68b400640 1 --2- 172.21.15.90:0/2179442711 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fe6700a3bb0 0x7fe6700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:25:27.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.524+0000 7fe68b400640 1 --2- 172.21.15.90:0/2179442711 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fe6700a3bb0 0x7fe6700a6070 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe68c10daa0 tx=0x7fe67c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:25:27.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.527+0000 7fe688c00640 1 -- 172.21.15.90:0/2179442711 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe67409cf90 con 0x7fe68c10dc40 2024-04-24T09:25:27.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.730+0000 7fe692600640 1 -- 172.21.15.90:0/2179442711 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe68c02d7a0 con 0x7fe68c10dc40 2024-04-24T09:25:27.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.734+0000 7fe688c00640 1 -- 172.21.15.90:0/2179442711 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22652 (secure 0 0 0) 0x7fe6740a1290 con 0x7fe68c10dc40 2024-04-24T09:25:27.735 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-24T09:25:27.735 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":88,"fsid":"30ee9be2-021a-11ef-bc93-c7b262605968","created":"2024-04-24T09:09:49.496019+0000","modified":"2024-04-24T09:19:58.609640+0000","last_up_change":"2024-04-24T09:18:10.455632+0000","last_in_change":"2024-04-24T09:17:50.600366+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T09:13:45.548774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T09:19:07.185580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T09:19:08.374818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"13a59b9f-b46e-4ec4-90af-bb7a3d5c753b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6803","nonce":2864288066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6805","nonce":2864288066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6809","nonce":2864288066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":2864288066},{"type":"v1","addr":"172.21.15.90:6807","nonce":2864288066}]},"public_addr":"172.21.15.90:6803/2864288066","cluster_addr":"172.21.15.90:6805/2864288066","heartbeat_back_addr":"172.21.15.90:6809/2864288066","heartbeat_front_addr":"172.21.15.90:6807/2864288066","state":["exists","up"]},{"osd":1,"uuid":"1567ce8f-24e6-42cf-bf64-f6f0c534a9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6801","nonce":344222414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6803","nonce":344222414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6807","nonce":344222414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":344222414},{"type":"v1","addr":"172.21.15.114:6805","nonce":344222414}]},"public_addr":"172.21.15.114:6801/344222414","cluster_addr":"172.21.15.114:6803/344222414","heartbeat_back_addr":"172.21.15.114:6807/344222414","heartbeat_front_addr":"172.21.15.114:6805/344222414","state":["exists","up"]},{"osd":2,"uuid":"f861b304-9c7d-4629-8522-37f33a747945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6801","nonce":3257067419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6803","nonce":3257067419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6807","nonce":3257067419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3257067419},{"type":"v1","addr":"172.21.15.149:6805","nonce":3257067419}]},"public_addr":"172.21.15.149:6801/3257067419","cluster_addr":"172.21.15.149:6803/3257067419","heartbeat_back_addr":"172.21.15.149:6807/3257067419","heartbeat_front_addr":"172.21.15.149:6805/3257067419","state":["exists","up"]},{"osd":3,"uuid":"9377ab6e-5996-45fb-93a9-475a2e5f17a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6811","nonce":3068524970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6813","nonce":3068524970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6817","nonce":3068524970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":3068524970},{"type":"v1","addr":"172.21.15.90:6815","nonce":3068524970}]},"public_addr":"172.21.15.90:6811/3068524970","cluster_addr":"172.21.15.90:6813/3068524970","heartbeat_back_addr":"172.21.15.90:6817/3068524970","heartbeat_front_addr":"172.21.15.90:6815/3068524970","state":["exists","up"]},{"osd":4,"uuid":"e2ddd19e-e641-4d61-b342-02b5f12b1c4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6809","nonce":2782015522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6811","nonce":2782015522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6815","nonce":2782015522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2782015522},{"type":"v1","addr":"172.21.15.114:6813","nonce":2782015522}]},"public_addr":"172.21.15.114:6809/2782015522","cluster_addr":"172.21.15.114:6811/2782015522","heartbeat_back_addr":"172.21.15.114:6815/2782015522","heartbeat_front_addr":"172.21.15.114:6813/2782015522","state":["exists","up"]},{"osd":5,"uuid":"b6ff9d10-5210-4b46-a192-6dcbf9d1f156","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6809","nonce":4266460479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6811","nonce":4266460479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6815","nonce":4266460479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":4266460479},{"type":"v1","addr":"172.21.15.149:6813","nonce":4266460479}]},"public_addr":"172.21.15.149:6809/4266460479","cluster_addr":"172.21.15.149:6811/4266460479","heartbeat_back_addr":"172.21.15.149:6815/4266460479","heartbeat_front_addr":"172.21.15.149:6813/4266460479","state":["exists","up"]},{"osd":6,"uuid":"4287b5a1-f953-421c-bf62-5e390bee834d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6819","nonce":1121461276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6821","nonce":1121461276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6825","nonce":1121461276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1121461276},{"type":"v1","addr":"172.21.15.90:6823","nonce":1121461276}]},"public_addr":"172.21.15.90:6819/1121461276","cluster_addr":"172.21.15.90:6821/1121461276","heartbeat_back_addr":"172.21.15.90:6825/1121461276","heartbeat_front_addr":"172.21.15.90:6823/1121461276","state":["exists","up"]},{"osd":7,"uuid":"a54e507d-d6ba-4bf2-9a7f-a7b95c3859bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6817","nonce":3222546280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6819","nonce":3222546280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6823","nonce":3222546280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":3222546280},{"type":"v1","addr":"172.21.15.114:6821","nonce":3222546280}]},"public_addr":"172.21.15.114:6817/3222546280","cluster_addr":"172.21.15.114:6819/3222546280","heartbeat_back_addr":"172.21.15.114:6823/3222546280","heartbeat_front_addr":"172.21.15.114:6821/3222546280","state":["exists","up"]},{"osd":8,"uuid":"9950b010-5865-4109-ae37-582b1dbd2f41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6817","nonce":157615149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6819","nonce":157615149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6823","nonce":157615149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":157615149},{"type":"v1","addr":"172.21.15.149:6821","nonce":157615149}]},"public_addr":"172.21.15.149:6817/157615149","cluster_addr":"172.21.15.149:6819/157615149","heartbeat_back_addr":"172.21.15.149:6823/157615149","heartbeat_front_addr":"172.21.15.149:6821/157615149","state":["exists","up"]},{"osd":9,"uuid":"31e2635c-e234-421f-9757-592fd673c936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6827","nonce":1307318076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6829","nonce":1307318076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6833","nonce":1307318076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1307318076},{"type":"v1","addr":"172.21.15.90:6831","nonce":1307318076}]},"public_addr":"172.21.15.90:6827/1307318076","cluster_addr":"172.21.15.90:6829/1307318076","heartbeat_back_addr":"172.21.15.90:6833/1307318076","heartbeat_front_addr":"172.21.15.90:6831/1307318076","state":["exists","up"]},{"osd":10,"uuid":"46a3b2a6-e6e5-4578-b4bb-f3e998b9bc95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6825","nonce":1719802140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6827","nonce":1719802140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6831","nonce":1719802140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":1719802140},{"type":"v1","addr":"172.21.15.114:6829","nonce":1719802140}]},"public_addr":"172.21.15.114:6825/1719802140","cluster_addr":"172.21.15.114:6827/1719802140","heartbeat_back_addr":"172.21.15.114:6831/1719802140","heartbeat_front_addr":"172.21.15.114:6829/1719802140","state":["exists","up"]},{"osd":11,"uuid":"ccd4456b-e947-46cf-a779-148ffa289b58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6825","nonce":1737275166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6827","nonce":1737275166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6831","nonce":1737275166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":1737275166},{"type":"v1","addr":"172.21.15.149:6829","nonce":1737275166}]},"public_addr":"172.21.15.149:6825/1737275166","cluster_addr":"172.21.15.149:6827/1737275166","heartbeat_back_addr":"172.21.15.149:6831/1737275166","heartbeat_front_addr":"172.21.15.149:6829/1737275166","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:12:53.015078+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:13:18.047053+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:13:41.935891+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:14:15.095388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:14:42.512355+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:15:08.113775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:15:43.828550+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:16:10.901909+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:16:36.422962+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:17:13.924446+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:17:42.814401+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T09:18:08.672793+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/3090852580":"2024-04-25T09:10:57.377655+0000","172.21.15.90:0/687197394":"2024-04-25T09:10:57.377655+0000","172.21.15.90:6801/1809038230":"2024-04-25T09:10:57.377655+0000","172.21.15.90:6800/1809038230":"2024-04-25T09:10:57.377655+0000","172.21.15.90:0/2964149071":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/3617388113":"2024-04-25T09:10:22.599674+0000","172.21.15.90:6801/3089140842":"2024-04-25T09:10:22.599674+0000","172.21.15.90:6800/3089140842":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/2840416149":"2024-04-25T09:10:22.599674+0000","172.21.15.90:0/305432710":"2024-04-25T09:10:57.377655+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T09:25:27.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.737+0000 7fe692600640 1 -- 172.21.15.90:0/2179442711 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fe6700a3bb0 msgr2=0x7fe6700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:27.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.737+0000 7fe692600640 1 --2- 172.21.15.90:0/2179442711 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fe6700a3bb0 0x7fe6700a6070 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe68c10daa0 tx=0x7fe67c009500 comp rx=0 tx=0).stop 2024-04-24T09:25:27.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.737+0000 7fe692600640 1 -- 172.21.15.90:0/2179442711 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe68c10dc40 msgr2=0x7fe68c1a32b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:27.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.737+0000 7fe692600640 1 --2- 172.21.15.90:0/2179442711 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe68c10dc40 0x7fe68c1a32b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe67400e9f0 tx=0x7fe67400eeb0 comp rx=0 tx=0).stop 2024-04-24T09:25:27.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.737+0000 7fe692600640 1 -- 172.21.15.90:0/2179442711 shutdown_connections 2024-04-24T09:25:27.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.737+0000 7fe692600640 1 --2- 172.21.15.90:0/2179442711 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fe6700a3bb0 0x7fe6700a6070 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:27.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.737+0000 7fe692600640 1 --2- 172.21.15.90:0/2179442711 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe68c10e600 0x7fe68c1aa320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:27.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.737+0000 7fe692600640 1 --2- 172.21.15.90:0/2179442711 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe68c10dc40 0x7fe68c1a32b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:27.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.737+0000 7fe692600640 1 --2- 172.21.15.90:0/2179442711 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe68c10ca50 0x7fe68c1a2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:27.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.737+0000 7fe692600640 1 -- 172.21.15.90:0/2179442711 >> 172.21.15.90:0/2179442711 conn(0x7fe68c06caf0 msgr2=0x7fe68c110ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:25:27.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.737+0000 7fe692600640 1 -- 172.21.15.90:0/2179442711 shutdown_connections 2024-04-24T09:25:27.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.737+0000 7fe692600640 1 -- 172.21.15.90:0/2179442711 wait complete. 2024-04-24T09:25:27.757 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-24T09:25:27.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.951+0000 7fe75d000640 1 -- 172.21.15.90:0/3112047460 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7580769c0 msgr2=0x7fe758076de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:27.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.951+0000 7fe75d000640 1 --2- 172.21.15.90:0/3112047460 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7580769c0 0x7fe758076de0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe750009860 tx=0x7fe75002a340 comp rx=0 tx=0).stop 2024-04-24T09:25:27.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.951+0000 7fe75d000640 1 -- 172.21.15.90:0/3112047460 shutdown_connections 2024-04-24T09:25:27.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.951+0000 7fe75d000640 1 --2- 172.21.15.90:0/3112047460 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe7580785a0 0x7fe758078e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:27.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.951+0000 7fe75d000640 1 --2- 172.21.15.90:0/3112047460 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe758077c00 0x7fe758078060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:27.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.951+0000 7fe75d000640 1 --2- 172.21.15.90:0/3112047460 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7580769c0 0x7fe758076de0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:27.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.951+0000 7fe75d000640 1 -- 172.21.15.90:0/3112047460 >> 172.21.15.90:0/3112047460 conn(0x7fe7580ff7b0 msgr2=0x7fe758101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:25:27.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.951+0000 7fe75d000640 1 -- 172.21.15.90:0/3112047460 shutdown_connections 2024-04-24T09:25:27.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.951+0000 7fe75d000640 1 -- 172.21.15.90:0/3112047460 wait complete. 2024-04-24T09:25:27.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.952+0000 7fe75d000640 1 Processor -- start 2024-04-24T09:25:27.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.953+0000 7fe75d000640 1 -- start start 2024-04-24T09:25:27.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.953+0000 7fe75d000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7580769c0 0x7fe75819e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:25:27.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.953+0000 7fe75d000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe758077c00 0x7fe75819ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:25:27.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.953+0000 7fe75d000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe7580785a0 0x7fe7581a5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:25:27.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.953+0000 7fe75d000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe7581142f0 con 0x7fe758077c00 2024-04-24T09:25:27.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.953+0000 7fe75d000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe758114170 con 0x7fe7580785a0 2024-04-24T09:25:27.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.953+0000 7fe75d000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe758114470 con 0x7fe7580769c0 2024-04-24T09:25:27.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.953+0000 7fe755400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe758077c00 0x7fe75819ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:25:27.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.953+0000 7fe755e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7580769c0 0x7fe75819e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:25:27.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.953+0000 7fe755400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe758077c00 0x7fe75819ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40074/0 (socket says 172.21.15.90:40074) 2024-04-24T09:25:27.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.953+0000 7fe755400640 1 -- 172.21.15.90:0/2455607947 learned_addr learned my addr 172.21.15.90:0/2455607947 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:25:27.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.953+0000 7fe756800640 1 --2- 172.21.15.90:0/2455607947 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe7580785a0 0x7fe7581a5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:25:27.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.954+0000 7fe755e00640 1 -- 172.21.15.90:0/2455607947 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe7580785a0 msgr2=0x7fe7581a5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:27.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.954+0000 7fe755e00640 1 --2- 172.21.15.90:0/2455607947 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe7580785a0 0x7fe7581a5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:27.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.954+0000 7fe755e00640 1 -- 172.21.15.90:0/2455607947 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe758077c00 msgr2=0x7fe75819ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:27.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.954+0000 7fe755e00640 1 --2- 172.21.15.90:0/2455607947 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe758077c00 0x7fe75819ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:27.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.954+0000 7fe755e00640 1 -- 172.21.15.90:0/2455607947 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7581a6560 con 0x7fe7580769c0 2024-04-24T09:25:27.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.954+0000 7fe755e00640 1 --2- 172.21.15.90:0/2455607947 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7580769c0 0x7fe75819e940 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe750035040 tx=0x7fe750033d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:25:27.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.954+0000 7fe74ea00640 1 -- 172.21.15.90:0/2455607947 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe750037040 con 0x7fe7580769c0 2024-04-24T09:25:27.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.954+0000 7fe74ea00640 1 -- 172.21.15.90:0/2455607947 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe750030850 con 0x7fe7580769c0 2024-04-24T09:25:27.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.954+0000 7fe74ea00640 1 -- 172.21.15.90:0/2455607947 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe750004920 con 0x7fe7580769c0 2024-04-24T09:25:27.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.954+0000 7fe75d000640 1 -- 172.21.15.90:0/2455607947 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7581a67f0 con 0x7fe7580769c0 2024-04-24T09:25:27.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.954+0000 7fe75d000640 1 -- 172.21.15.90:0/2455607947 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7581a6d00 con 0x7fe7580769c0 2024-04-24T09:25:27.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.955+0000 7fe75d000640 1 -- 172.21.15.90:0/2455607947 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe758110d70 con 0x7fe7580769c0 2024-04-24T09:25:27.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.958+0000 7fe74ea00640 1 -- 172.21.15.90:0/2455607947 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fe750004050 con 0x7fe7580769c0 2024-04-24T09:25:27.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.959+0000 7fe74ea00640 1 --2- 172.21.15.90:0/2455607947 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fe73c0a3c30 0x7fe73c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:25:27.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.959+0000 7fe74ea00640 1 -- 172.21.15.90:0/2455607947 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fe7500f0160 con 0x7fe7580769c0 2024-04-24T09:25:27.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.959+0000 7fe755400640 1 --2- 172.21.15.90:0/2455607947 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fe73c0a3c30 0x7fe73c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:25:27.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.960+0000 7fe755400640 1 --2- 172.21.15.90:0/2455607947 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fe73c0a3c30 0x7fe73c0a60f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe75819fe50 tx=0x7fe740010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:25:27.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:27.963+0000 7fe74ea00640 1 -- 172.21.15.90:0/2455607947 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe7500bcb00 con 0x7fe7580769c0 2024-04-24T09:25:28.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:27 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/4011959531' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:25:28.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:27 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/2038911741' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:25:28.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:27 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/2179442711' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:25:28.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:27 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/4011959531' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:25:28.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:27 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/2038911741' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:25:28.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:27 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/2179442711' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:25:28.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:27 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/4011959531' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:25:28.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:27 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/2038911741' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:25:28.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:27 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/2179442711' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:25:28.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.207+0000 7fe75d000640 1 -- 172.21.15.90:0/2455607947 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe75802d840 con 0x7fe7580769c0 2024-04-24T09:25:28.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.220+0000 7fe74ea00640 1 -- 172.21.15.90:0/2455607947 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v32) ==== 161+0+0 (secure 0 0 0) 0x7fe7500bc400 con 0x7fe7580769c0 2024-04-24T09:25:28.221 INFO:teuthology.orchestra.run.smithi090.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-24T09:25:28.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.223+0000 7fe75d000640 1 -- 172.21.15.90:0/2455607947 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fe73c0a3c30 msgr2=0x7fe73c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:28.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.223+0000 7fe75d000640 1 --2- 172.21.15.90:0/2455607947 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fe73c0a3c30 0x7fe73c0a60f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe75819fe50 tx=0x7fe740010040 comp rx=0 tx=0).stop 2024-04-24T09:25:28.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.223+0000 7fe75d000640 1 -- 172.21.15.90:0/2455607947 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7580769c0 msgr2=0x7fe75819e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:28.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.223+0000 7fe75d000640 1 --2- 172.21.15.90:0/2455607947 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7580769c0 0x7fe75819e940 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe750035040 tx=0x7fe750033d90 comp rx=0 tx=0).stop 2024-04-24T09:25:28.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.224+0000 7fe75d000640 1 -- 172.21.15.90:0/2455607947 shutdown_connections 2024-04-24T09:25:28.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.224+0000 7fe75d000640 1 --2- 172.21.15.90:0/2455607947 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7fe73c0a3c30 0x7fe73c0a60f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:28.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.224+0000 7fe75d000640 1 --2- 172.21.15.90:0/2455607947 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe7580785a0 0x7fe7581a5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:28.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.224+0000 7fe75d000640 1 --2- 172.21.15.90:0/2455607947 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe758077c00 0x7fe75819ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:28.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.224+0000 7fe75d000640 1 --2- 172.21.15.90:0/2455607947 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7580769c0 0x7fe75819e940 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:28.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.224+0000 7fe75d000640 1 -- 172.21.15.90:0/2455607947 >> 172.21.15.90:0/2455607947 conn(0x7fe7580ff7b0 msgr2=0x7fe758100f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:25:28.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.224+0000 7fe75d000640 1 -- 172.21.15.90:0/2455607947 shutdown_connections 2024-04-24T09:25:28.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.224+0000 7fe75d000640 1 -- 172.21.15.90:0/2455607947 wait complete. 2024-04-24T09:25:28.244 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-24T09:25:28.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.438+0000 7faf0fa00640 1 -- 172.21.15.90:0/306424556 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faf08104280 msgr2=0x7faf081046a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:28.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.438+0000 7faf0fa00640 1 --2- 172.21.15.90:0/306424556 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faf08104280 0x7faf081046a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7faef8009900 tx=0x7faef802a3f0 comp rx=0 tx=0).stop 2024-04-24T09:25:28.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.438+0000 7faf0fa00640 1 -- 172.21.15.90:0/306424556 shutdown_connections 2024-04-24T09:25:28.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.438+0000 7faf0fa00640 1 --2- 172.21.15.90:0/306424556 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf08105e30 0x7faf0810c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:28.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.438+0000 7faf0fa00640 1 --2- 172.21.15.90:0/306424556 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf08105470 0x7faf081058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:28.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.438+0000 7faf0fa00640 1 --2- 172.21.15.90:0/306424556 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faf08104280 0x7faf081046a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:28.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.438+0000 7faf0fa00640 1 -- 172.21.15.90:0/306424556 >> 172.21.15.90:0/306424556 conn(0x7faf080ff7f0 msgr2=0x7faf08101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:25:28.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.438+0000 7faf0fa00640 1 -- 172.21.15.90:0/306424556 shutdown_connections 2024-04-24T09:25:28.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.438+0000 7faf0fa00640 1 -- 172.21.15.90:0/306424556 wait complete. 2024-04-24T09:25:28.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.439+0000 7faf0fa00640 1 Processor -- start 2024-04-24T09:25:28.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.439+0000 7faf0fa00640 1 -- start start 2024-04-24T09:25:28.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.440+0000 7faf0fa00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf08104280 0x7faf0819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:25:28.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.440+0000 7faf0fa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf08105470 0x7faf0819aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:25:28.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.440+0000 7faf0fa00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faf08105e30 0x7faf081a1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:25:28.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.440+0000 7faf0fa00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7faf0810ff60 con 0x7faf08104280 2024-04-24T09:25:28.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.440+0000 7faf0fa00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7faf0810fde0 con 0x7faf08105e30 2024-04-24T09:25:28.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.440+0000 7faf0fa00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7faf081100e0 con 0x7faf08105470 2024-04-24T09:25:28.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.440+0000 7faf0d000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf08104280 0x7faf0819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:25:28.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.440+0000 7faf07e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf08105470 0x7faf0819aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:25:28.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.440+0000 7faf07e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf08105470 0x7faf0819aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:51562/0 (socket says 172.21.15.90:51562) 2024-04-24T09:25:28.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.440+0000 7faf07e00640 1 -- 172.21.15.90:0/3884361509 learned_addr learned my addr 172.21.15.90:0/3884361509 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:25:28.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.440+0000 7faf0da00640 1 --2- 172.21.15.90:0/3884361509 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faf08105e30 0x7faf081a1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:25:28.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.440+0000 7faf07e00640 1 -- 172.21.15.90:0/3884361509 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faf08105e30 msgr2=0x7faf081a1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:28.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.440+0000 7faf07e00640 1 --2- 172.21.15.90:0/3884361509 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faf08105e30 0x7faf081a1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:28.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.440+0000 7faf07e00640 1 -- 172.21.15.90:0/3884361509 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf08104280 msgr2=0x7faf0819a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:28.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.440+0000 7faf07e00640 1 --2- 172.21.15.90:0/3884361509 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf08104280 0x7faf0819a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:28.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.440+0000 7faf07e00640 1 -- 172.21.15.90:0/3884361509 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf081a21d0 con 0x7faf08105470 2024-04-24T09:25:28.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.441+0000 7faf07e00640 1 --2- 172.21.15.90:0/3884361509 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf08105470 0x7faf0819aaf0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7faef000e9f0 tx=0x7faef000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:25:28.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.441+0000 7faf05600640 1 -- 172.21.15.90:0/3884361509 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef000cd80 con 0x7faf08105470 2024-04-24T09:25:28.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.441+0000 7faf0fa00640 1 -- 172.21.15.90:0/3884361509 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf081a24c0 con 0x7faf08105470 2024-04-24T09:25:28.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.441+0000 7faf05600640 1 -- 172.21.15.90:0/3884361509 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faef0005500 con 0x7faf08105470 2024-04-24T09:25:28.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.441+0000 7faf05600640 1 -- 172.21.15.90:0/3884361509 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef0010690 con 0x7faf08105470 2024-04-24T09:25:28.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.441+0000 7faf0fa00640 1 -- 172.21.15.90:0/3884361509 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf081a2a10 con 0x7faf08105470 2024-04-24T09:25:28.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.442+0000 7faf0fa00640 1 -- 172.21.15.90:0/3884361509 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf0810b5a0 con 0x7faf08105470 2024-04-24T09:25:28.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.444+0000 7faf05600640 1 -- 172.21.15.90:0/3884361509 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7faef00046c0 con 0x7faf08105470 2024-04-24T09:25:28.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.446+0000 7faf05600640 1 --2- 172.21.15.90:0/3884361509 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7faeec0a3c00 0x7faeec0a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:25:28.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.446+0000 7faf05600640 1 -- 172.21.15.90:0/3884361509 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8670+0+0 (secure 0 0 0) 0x7faef00d0a10 con 0x7faf08105470 2024-04-24T09:25:28.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.446+0000 7faf0d000640 1 --2- 172.21.15.90:0/3884361509 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7faeec0a3c00 0x7faeec0a60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:25:28.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.447+0000 7faf0d000640 1 --2- 172.21.15.90:0/3884361509 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7faeec0a3c00 0x7faeec0a60c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7faef802a9e0 tx=0x7faef8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:25:28.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.449+0000 7faf05600640 1 -- 172.21.15.90:0/3884361509 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faef009d1e0 con 0x7faf08105470 2024-04-24T09:25:28.602 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 24 09:25:28 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-j[68052]: ignoring --setuser ceph since I am not root 2024-04-24T09:25:28.603 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 24 09:25:28 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-j[68052]: ignoring --setgroup ceph since I am not root 2024-04-24T09:25:28.603 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 24 09:25:28 smithi090 ceph-mds[68056]: -- 172.21.15.90:0/763799743 <== mon.2 v2:172.21.15.114:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5562834dfa40 con 0x5562834b2800 2024-04-24T09:25:28.603 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 24 09:25:28 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-j[68052]: starting mds.j at 2024-04-24T09:25:28.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:28 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mon-a[27821]: 2024-04-24T09:25:28.209+0000 7f6f3cc00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T09:25:28.604 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 24 09:25:28 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-a[69432]: ignoring --setuser ceph since I am not root 2024-04-24T09:25:28.604 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 24 09:25:28 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-a[69432]: ignoring --setgroup ceph since I am not root 2024-04-24T09:25:28.604 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 24 09:25:28 smithi090 ceph-mds[69436]: -- 172.21.15.90:0/3543308551 <== mon.2 v2:172.21.15.114:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559a73511a40 con 0x559a734e5000 2024-04-24T09:25:28.604 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 24 09:25:28 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-a[69432]: starting mds.a at 2024-04-24T09:25:28.611 INFO:journalctl@ceph.mds.c.smithi149.stdout:Apr 24 09:25:28 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-c[64043]: ignoring --setuser ceph since I am not root 2024-04-24T09:25:28.612 INFO:journalctl@ceph.mds.c.smithi149.stdout:Apr 24 09:25:28 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-c[64043]: ignoring --setgroup ceph since I am not root 2024-04-24T09:25:28.612 INFO:journalctl@ceph.mds.c.smithi149.stdout:Apr 24 09:25:28 smithi149 ceph-mds[64047]: -- 172.21.15.149:0/1531596515 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557a2cda7a40 con 0x557a2cd7a800 2024-04-24T09:25:28.612 INFO:journalctl@ceph.mds.c.smithi149.stdout:Apr 24 09:25:28 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-c[64043]: starting mds.c at 2024-04-24T09:25:28.613 INFO:journalctl@ceph.mds.l.smithi149.stdout:Apr 24 09:25:28 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-l[63416]: ignoring --setuser ceph since I am not root 2024-04-24T09:25:28.613 INFO:journalctl@ceph.mds.l.smithi149.stdout:Apr 24 09:25:28 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-l[63416]: ignoring --setgroup ceph since I am not root 2024-04-24T09:25:28.613 INFO:journalctl@ceph.mds.l.smithi149.stdout:Apr 24 09:25:28 smithi149 ceph-mds[63420]: -- 172.21.15.149:0/1277273952 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5570a9401a40 con 0x5570a93d4c00 2024-04-24T09:25:28.613 INFO:journalctl@ceph.mds.l.smithi149.stdout:Apr 24 09:25:28 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-l[63416]: starting mds.l at 2024-04-24T09:25:28.614 INFO:journalctl@ceph.mds.f.smithi149.stdout:Apr 24 09:25:28 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-f[63729]: ignoring --setuser ceph since I am not root 2024-04-24T09:25:28.614 INFO:journalctl@ceph.mds.f.smithi149.stdout:Apr 24 09:25:28 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-f[63729]: ignoring --setgroup ceph since I am not root 2024-04-24T09:25:28.614 INFO:journalctl@ceph.mds.f.smithi149.stdout:Apr 24 09:25:28 smithi149 ceph-mds[63738]: -- 172.21.15.149:0/203379986 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563c230b9a40 con 0x563c2308d000 2024-04-24T09:25:28.615 INFO:journalctl@ceph.mds.f.smithi149.stdout:Apr 24 09:25:28 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-f[63729]: starting mds.f at 2024-04-24T09:25:28.615 INFO:journalctl@ceph.mds.i.smithi149.stdout:Apr 24 09:25:28 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-i[64347]: ignoring --setuser ceph since I am not root 2024-04-24T09:25:28.615 INFO:journalctl@ceph.mds.i.smithi149.stdout:Apr 24 09:25:28 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-i[64347]: ignoring --setgroup ceph since I am not root 2024-04-24T09:25:28.615 INFO:journalctl@ceph.mds.i.smithi149.stdout:Apr 24 09:25:28 smithi149 ceph-mds[64351]: -- 172.21.15.149:0/3501759542 <== mon.2 v2:172.21.15.114:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cc6a76da40 con 0x55cc6a740c00 2024-04-24T09:25:28.615 INFO:journalctl@ceph.mds.i.smithi149.stdout:Apr 24 09:25:28 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-i[64347]: starting mds.i at 2024-04-24T09:25:28.676 INFO:journalctl@ceph.mds.b.smithi114.stdout:Apr 24 09:25:28 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-b[63608]: ignoring --setuser ceph since I am not root 2024-04-24T09:25:28.676 INFO:journalctl@ceph.mds.b.smithi114.stdout:Apr 24 09:25:28 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-b[63608]: ignoring --setgroup ceph since I am not root 2024-04-24T09:25:28.676 INFO:journalctl@ceph.mds.b.smithi114.stdout:Apr 24 09:25:28 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-b[63608]: starting mds.b at 2024-04-24T09:25:28.677 INFO:journalctl@ceph.mds.h.smithi114.stdout:Apr 24 09:25:28 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-h[63285]: ignoring --setuser ceph since I am not root 2024-04-24T09:25:28.677 INFO:journalctl@ceph.mds.h.smithi114.stdout:Apr 24 09:25:28 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-h[63285]: ignoring --setgroup ceph since I am not root 2024-04-24T09:25:28.677 INFO:journalctl@ceph.mds.h.smithi114.stdout:Apr 24 09:25:28 smithi114 ceph-mds[63289]: -- 172.21.15.114:0/2884556009 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ef86da1a40 con 0x55ef86d74c00 2024-04-24T09:25:28.677 INFO:journalctl@ceph.mds.h.smithi114.stdout:Apr 24 09:25:28 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-h[63285]: starting mds.h at 2024-04-24T09:25:28.678 INFO:journalctl@ceph.mds.e.smithi114.stdout:Apr 24 09:25:28 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-e[62977]: ignoring --setuser ceph since I am not root 2024-04-24T09:25:28.678 INFO:journalctl@ceph.mds.e.smithi114.stdout:Apr 24 09:25:28 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-e[62977]: ignoring --setgroup ceph since I am not root 2024-04-24T09:25:28.678 INFO:journalctl@ceph.mds.e.smithi114.stdout:Apr 24 09:25:28 smithi114 ceph-mds[62981]: -- 172.21.15.114:0/3819088858 <== mon.2 v2:172.21.15.114:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5586d0437a40 con 0x5586d040a800 2024-04-24T09:25:28.679 INFO:journalctl@ceph.mds.e.smithi114.stdout:Apr 24 09:25:28 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-e[62977]: starting mds.e at 2024-04-24T09:25:28.679 INFO:journalctl@ceph.mds.k.smithi114.stdout:Apr 24 09:25:28 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-k[62652]: ignoring --setuser ceph since I am not root 2024-04-24T09:25:28.679 INFO:journalctl@ceph.mds.k.smithi114.stdout:Apr 24 09:25:28 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-k[62652]: ignoring --setgroup ceph since I am not root 2024-04-24T09:25:28.679 INFO:journalctl@ceph.mds.k.smithi114.stdout:Apr 24 09:25:28 smithi114 ceph-mds[62656]: -- 172.21.15.114:0/3167492309 <== mon.2 v2:172.21.15.114:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cb56141a40 con 0x55cb56114c00 2024-04-24T09:25:28.680 INFO:journalctl@ceph.mds.k.smithi114.stdout:Apr 24 09:25:28 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-k[62652]: starting mds.k at 2024-04-24T09:25:28.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:28.689+0000 7faf0fa00640 1 -- 172.21.15.90:0/3884361509 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7faf08062c00 con 0x7faf08105470 2024-04-24T09:25:29.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:28 smithi090 ceph-mon[27825]: pgmap v513: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-24T09:25:29.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:28 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/2455607947' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-24T09:25:29.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:28 smithi090 ceph-mon[27825]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-24T09:25:29.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:28 smithi090 ceph-mon[27825]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-24T09:25:29.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:28 smithi090 ceph-mon[27825]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T09:25:29.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:28 smithi090 ceph-mon[27825]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-24T09:25:29.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:28 smithi090 ceph-mon[27825]: osdmap e89: 12 total, 12 up, 12 in 2024-04-24T09:25:29.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:28 smithi090 ceph-mon[27825]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-24T09:25:29.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:28 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/3884361509' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T09:25:29.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:28 smithi090 ceph-mon[27825]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T09:25:29.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:28 smithi149 ceph-mon[32592]: pgmap v513: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-24T09:25:29.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:28 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/2455607947' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-24T09:25:29.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:28 smithi149 ceph-mon[32592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-24T09:25:29.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:28 smithi149 ceph-mon[32592]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-24T09:25:29.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:28 smithi149 ceph-mon[32592]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T09:25:29.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:28 smithi149 ceph-mon[32592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-24T09:25:29.113 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:28 smithi149 ceph-mon[32592]: osdmap e89: 12 total, 12 up, 12 in 2024-04-24T09:25:29.113 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:28 smithi149 ceph-mon[32592]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-24T09:25:29.113 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:28 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/3884361509' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T09:25:29.113 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:28 smithi149 ceph-mon[32592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T09:25:29.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:28 smithi114 ceph-mon[32501]: pgmap v513: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-24T09:25:29.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:28 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/2455607947' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-24T09:25:29.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:28 smithi114 ceph-mon[32501]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-24T09:25:29.177 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:28 smithi114 ceph-mon[32501]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-24T09:25:29.177 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:28 smithi114 ceph-mon[32501]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T09:25:29.177 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:28 smithi114 ceph-mon[32501]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-24T09:25:29.177 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:28 smithi114 ceph-mon[32501]: osdmap e89: 12 total, 12 up, 12 in 2024-04-24T09:25:29.177 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:28 smithi114 ceph-mon[32501]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-24T09:25:29.177 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:28 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/3884361509' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T09:25:29.178 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:28 smithi114 ceph-mon[32501]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T09:25:29.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.219+0000 7faf05600640 1 -- 172.21.15.90:0/3884361509 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v33) ==== 104+0+0 (secure 0 0 0) 0x7faef009cae0 con 0x7faf08105470 2024-04-24T09:25:29.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.223+0000 7faf0fa00640 1 -- 172.21.15.90:0/3884361509 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7faeec0a3c00 msgr2=0x7faeec0a60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:29.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.223+0000 7faf0fa00640 1 --2- 172.21.15.90:0/3884361509 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7faeec0a3c00 0x7faeec0a60c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7faef802a9e0 tx=0x7faef8000bc0 comp rx=0 tx=0).stop 2024-04-24T09:25:29.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.223+0000 7faf0fa00640 1 -- 172.21.15.90:0/3884361509 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf08105470 msgr2=0x7faf0819aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:29.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.223+0000 7faf0fa00640 1 --2- 172.21.15.90:0/3884361509 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf08105470 0x7faf0819aaf0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7faef000e9f0 tx=0x7faef000edf0 comp rx=0 tx=0).stop 2024-04-24T09:25:29.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.224+0000 7faf0fa00640 1 -- 172.21.15.90:0/3884361509 shutdown_connections 2024-04-24T09:25:29.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.224+0000 7faf0fa00640 1 --2- 172.21.15.90:0/3884361509 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7faeec0a3c00 0x7faeec0a60c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:29.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.224+0000 7faf0fa00640 1 --2- 172.21.15.90:0/3884361509 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faf08105e30 0x7faf081a1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:29.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.224+0000 7faf0fa00640 1 --2- 172.21.15.90:0/3884361509 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faf08105470 0x7faf0819aaf0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:29.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.224+0000 7faf0fa00640 1 --2- 172.21.15.90:0/3884361509 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf08104280 0x7faf0819a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:29.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.224+0000 7faf0fa00640 1 -- 172.21.15.90:0/3884361509 >> 172.21.15.90:0/3884361509 conn(0x7faf080ff7f0 msgr2=0x7faf08108770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:25:29.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.225+0000 7faf0fa00640 1 -- 172.21.15.90:0/3884361509 shutdown_connections 2024-04-24T09:25:29.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.225+0000 7faf0fa00640 1 -- 172.21.15.90:0/3884361509 wait complete. 2024-04-24T09:25:29.249 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-24T09:25:29.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.451+0000 7f2267e00640 1 -- 172.21.15.90:0/30032615 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2268109650 msgr2=0x7f2268109ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:29.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.451+0000 7f2267e00640 1 --2- 172.21.15.90:0/30032615 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2268109650 0x7f2268109ad0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2250009900 tx=0x7f225002a410 comp rx=0 tx=0).stop 2024-04-24T09:25:29.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.451+0000 7f2267e00640 1 -- 172.21.15.90:0/30032615 shutdown_connections 2024-04-24T09:25:29.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.451+0000 7f2267e00640 1 --2- 172.21.15.90:0/30032615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f226810a010 0x7f22681108a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:29.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.451+0000 7f2267e00640 1 --2- 172.21.15.90:0/30032615 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2268109650 0x7f2268109ad0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:29.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.451+0000 7f2267e00640 1 --2- 172.21.15.90:0/30032615 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2268108460 0x7f2268108880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:29.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.451+0000 7f2267e00640 1 -- 172.21.15.90:0/30032615 >> 172.21.15.90:0/30032615 conn(0x7f2268077400 msgr2=0x7f2268079840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:25:29.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.451+0000 7f2267e00640 1 -- 172.21.15.90:0/30032615 shutdown_connections 2024-04-24T09:25:29.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.451+0000 7f2267e00640 1 -- 172.21.15.90:0/30032615 wait complete. 2024-04-24T09:25:29.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.452+0000 7f2267e00640 1 Processor -- start 2024-04-24T09:25:29.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.452+0000 7f2267e00640 1 -- start start 2024-04-24T09:25:29.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.453+0000 7f2267e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2268108460 0x7f226819e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:25:29.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.453+0000 7f2267e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2268109650 0x7f226819ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:25:29.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.453+0000 7f2267e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f226810a010 0x7f22681a5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:25:29.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.453+0000 7f2267e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2268114140 con 0x7f226810a010 2024-04-24T09:25:29.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.453+0000 7f2267e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2268113fc0 con 0x7f2268109650 2024-04-24T09:25:29.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.453+0000 7f2267e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f22681142c0 con 0x7f2268108460 2024-04-24T09:25:29.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.453+0000 7f2266a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2268108460 0x7f226819e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:25:29.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.453+0000 7f2267400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f226810a010 0x7f22681a5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:25:29.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.453+0000 7f2266000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2268109650 0x7f226819ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:25:29.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.453+0000 7f2266a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2268108460 0x7f226819e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.90:51576/0 (socket says 172.21.15.90:51576) 2024-04-24T09:25:29.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.453+0000 7f2266a00640 1 -- 172.21.15.90:0/4199206227 learned_addr learned my addr 172.21.15.90:0/4199206227 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:25:29.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.453+0000 7f2266000640 1 -- 172.21.15.90:0/4199206227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2268108460 msgr2=0x7f226819e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:29.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.453+0000 7f2266000640 1 --2- 172.21.15.90:0/4199206227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2268108460 0x7f226819e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:29.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.454+0000 7f2266000640 1 -- 172.21.15.90:0/4199206227 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f226810a010 msgr2=0x7f22681a5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:29.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.454+0000 7f2266000640 1 --2- 172.21.15.90:0/4199206227 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f226810a010 0x7f22681a5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:29.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.454+0000 7f2266000640 1 -- 172.21.15.90:0/4199206227 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22681a6320 con 0x7f2268109650 2024-04-24T09:25:29.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.454+0000 7f2266a00640 1 --2- 172.21.15.90:0/4199206227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2268108460 0x7f226819e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:25:29.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.454+0000 7f2267400640 1 --2- 172.21.15.90:0/4199206227 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f226810a010 0x7f22681a5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:25:29.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.454+0000 7f2266000640 1 --2- 172.21.15.90:0/4199206227 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2268109650 0x7f226819ec40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f225002a930 tx=0x7f2250032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:25:29.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.454+0000 7f225f400640 1 -- 172.21.15.90:0/4199206227 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225003f030 con 0x7f2268109650 2024-04-24T09:25:29.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.454+0000 7f2267e00640 1 -- 172.21.15.90:0/4199206227 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22681a65b0 con 0x7f2268109650 2024-04-24T09:25:29.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.454+0000 7f225f400640 1 -- 172.21.15.90:0/4199206227 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2250003ce0 con 0x7f2268109650 2024-04-24T09:25:29.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.454+0000 7f225f400640 1 -- 172.21.15.90:0/4199206227 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2250035070 con 0x7f2268109650 2024-04-24T09:25:29.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.454+0000 7f2267e00640 1 -- 172.21.15.90:0/4199206227 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22681a6ac0 con 0x7f2268109650 2024-04-24T09:25:29.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.455+0000 7f2267e00640 1 -- 172.21.15.90:0/4199206227 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f226810f720 con 0x7f2268109650 2024-04-24T09:25:29.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.457+0000 7f225f400640 1 -- 172.21.15.90:0/4199206227 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f2250042030 con 0x7f2268109650 2024-04-24T09:25:29.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.459+0000 7f225f400640 1 --2- 172.21.15.90:0/4199206227 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f224c0a3bb0 0x7f224c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:25:29.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.459+0000 7f225f400640 1 -- 172.21.15.90:0/4199206227 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8670+0+0 (secure 0 0 0) 0x7f22500f0a70 con 0x7f2268109650 2024-04-24T09:25:29.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.459+0000 7f2266a00640 1 --2- 172.21.15.90:0/4199206227 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f224c0a3bb0 0x7f224c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:25:29.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.459+0000 7f2266a00640 1 --2- 172.21.15.90:0/4199206227 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f224c0a3bb0 0x7f224c0a6070 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f22580104e0 tx=0x7f225800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:25:29.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.463+0000 7f225f400640 1 -- 172.21.15.90:0/4199206227 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f22500bcbf0 con 0x7f2268109650 2024-04-24T09:25:29.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:29.670+0000 7f2267e00640 1 -- 172.21.15.90:0/4199206227 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f226802d620 con 0x7f2268109650 2024-04-24T09:25:30.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:29 smithi090 ceph-mon[27825]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-24T09:25:30.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:29 smithi090 ceph-mon[27825]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-24T09:25:30.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:29 smithi090 ceph-mon[27825]: Cluster is now healthy 2024-04-24T09:25:30.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:29 smithi090 ceph-mon[27825]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-24T09:25:30.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:29 smithi090 ceph-mon[27825]: mds.? [v2:172.21.15.114:6833/2041969942,v1:172.21.15.114:6834/2041969942] up:boot 2024-04-24T09:25:30.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:29 smithi090 ceph-mon[27825]: mds.? [v2:172.21.15.114:6832/2702405496,v1:172.21.15.114:6835/2702405496] up:boot 2024-04-24T09:25:30.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:29 smithi090 ceph-mon[27825]: mds.? [v2:172.21.15.149:6832/3779633848,v1:172.21.15.149:6834/3779633848] up:boot 2024-04-24T09:25:30.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:29 smithi090 ceph-mon[27825]: mds.? [v2:172.21.15.149:6833/1965330065,v1:172.21.15.149:6835/1965330065] up:boot 2024-04-24T09:25:30.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:29 smithi090 ceph-mon[27825]: mds.? [v2:172.21.15.114:6836/1141348371,v1:172.21.15.114:6837/1141348371] up:boot 2024-04-24T09:25:30.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:29 smithi090 ceph-mon[27825]: mds.? [v2:172.21.15.114:6838/2520154062,v1:172.21.15.114:6839/2520154062] up:boot 2024-04-24T09:25:30.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:29 smithi090 ceph-mon[27825]: mds.? [v2:172.21.15.149:6837/2777766079,v1:172.21.15.149:6839/2777766079] up:boot 2024-04-24T09:25:30.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:29 smithi090 ceph-mon[27825]: mds.? [v2:172.21.15.149:6836/1456325482,v1:172.21.15.149:6838/1456325482] up:boot 2024-04-24T09:25:30.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:29 smithi090 ceph-mon[27825]: mds.? [v2:172.21.15.90:6834/2106018942,v1:172.21.15.90:6835/2106018942] up:boot 2024-04-24T09:25:30.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:29 smithi090 ceph-mon[27825]: mds.? [v2:172.21.15.90:6840/2890395646,v1:172.21.15.90:6841/2890395646] up:boot 2024-04-24T09:25:30.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:29 smithi090 ceph-mon[27825]: fsmap 12 up:standby 2024-04-24T09:25:30.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:29 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-24T09:25:30.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:29 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T09:25:30.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:29 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T09:25:30.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:29 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T09:25:30.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:29 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T09:25:30.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:29 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T09:25:30.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:29 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T09:25:30.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:29 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T09:25:30.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:29 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T09:25:30.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:29 smithi090 ceph-mon[27825]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T09:25:30.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:29 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/4199206227' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T09:25:30.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:29 smithi090 ceph-mon[27825]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T09:25:30.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:29 smithi149 ceph-mon[32592]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-24T09:25:30.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:29 smithi149 ceph-mon[32592]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-24T09:25:30.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:29 smithi149 ceph-mon[32592]: Cluster is now healthy 2024-04-24T09:25:30.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:29 smithi149 ceph-mon[32592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-24T09:25:30.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:29 smithi149 ceph-mon[32592]: mds.? [v2:172.21.15.114:6833/2041969942,v1:172.21.15.114:6834/2041969942] up:boot 2024-04-24T09:25:30.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:29 smithi149 ceph-mon[32592]: mds.? [v2:172.21.15.114:6832/2702405496,v1:172.21.15.114:6835/2702405496] up:boot 2024-04-24T09:25:30.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:29 smithi149 ceph-mon[32592]: mds.? [v2:172.21.15.149:6832/3779633848,v1:172.21.15.149:6834/3779633848] up:boot 2024-04-24T09:25:30.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:29 smithi149 ceph-mon[32592]: mds.? [v2:172.21.15.149:6833/1965330065,v1:172.21.15.149:6835/1965330065] up:boot 2024-04-24T09:25:30.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:29 smithi149 ceph-mon[32592]: mds.? [v2:172.21.15.114:6836/1141348371,v1:172.21.15.114:6837/1141348371] up:boot 2024-04-24T09:25:30.113 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:29 smithi149 ceph-mon[32592]: mds.? [v2:172.21.15.114:6838/2520154062,v1:172.21.15.114:6839/2520154062] up:boot 2024-04-24T09:25:30.113 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:29 smithi149 ceph-mon[32592]: mds.? [v2:172.21.15.149:6837/2777766079,v1:172.21.15.149:6839/2777766079] up:boot 2024-04-24T09:25:30.113 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:29 smithi149 ceph-mon[32592]: mds.? [v2:172.21.15.149:6836/1456325482,v1:172.21.15.149:6838/1456325482] up:boot 2024-04-24T09:25:30.113 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:29 smithi149 ceph-mon[32592]: mds.? [v2:172.21.15.90:6834/2106018942,v1:172.21.15.90:6835/2106018942] up:boot 2024-04-24T09:25:30.113 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:29 smithi149 ceph-mon[32592]: mds.? [v2:172.21.15.90:6840/2890395646,v1:172.21.15.90:6841/2890395646] up:boot 2024-04-24T09:25:30.113 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:29 smithi149 ceph-mon[32592]: fsmap 12 up:standby 2024-04-24T09:25:30.113 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:29 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-24T09:25:30.113 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:29 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T09:25:30.113 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:29 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T09:25:30.114 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:29 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T09:25:30.114 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:29 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T09:25:30.114 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:29 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T09:25:30.114 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:29 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T09:25:30.114 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:29 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T09:25:30.114 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:29 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T09:25:30.114 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:29 smithi149 ceph-mon[32592]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T09:25:30.114 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:29 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/4199206227' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T09:25:30.115 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:29 smithi149 ceph-mon[32592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T09:25:30.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:29 smithi114 ceph-mon[32501]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-24T09:25:30.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:29 smithi114 ceph-mon[32501]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-24T09:25:30.177 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:29 smithi114 ceph-mon[32501]: Cluster is now healthy 2024-04-24T09:25:30.177 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:29 smithi114 ceph-mon[32501]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-24T09:25:30.177 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:29 smithi114 ceph-mon[32501]: mds.? [v2:172.21.15.114:6833/2041969942,v1:172.21.15.114:6834/2041969942] up:boot 2024-04-24T09:25:30.177 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:29 smithi114 ceph-mon[32501]: mds.? [v2:172.21.15.114:6832/2702405496,v1:172.21.15.114:6835/2702405496] up:boot 2024-04-24T09:25:30.177 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:29 smithi114 ceph-mon[32501]: mds.? [v2:172.21.15.149:6832/3779633848,v1:172.21.15.149:6834/3779633848] up:boot 2024-04-24T09:25:30.177 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:29 smithi114 ceph-mon[32501]: mds.? [v2:172.21.15.149:6833/1965330065,v1:172.21.15.149:6835/1965330065] up:boot 2024-04-24T09:25:30.177 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:29 smithi114 ceph-mon[32501]: mds.? [v2:172.21.15.114:6836/1141348371,v1:172.21.15.114:6837/1141348371] up:boot 2024-04-24T09:25:30.177 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:29 smithi114 ceph-mon[32501]: mds.? [v2:172.21.15.114:6838/2520154062,v1:172.21.15.114:6839/2520154062] up:boot 2024-04-24T09:25:30.178 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:29 smithi114 ceph-mon[32501]: mds.? [v2:172.21.15.149:6837/2777766079,v1:172.21.15.149:6839/2777766079] up:boot 2024-04-24T09:25:30.178 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:29 smithi114 ceph-mon[32501]: mds.? [v2:172.21.15.149:6836/1456325482,v1:172.21.15.149:6838/1456325482] up:boot 2024-04-24T09:25:30.178 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:29 smithi114 ceph-mon[32501]: mds.? [v2:172.21.15.90:6834/2106018942,v1:172.21.15.90:6835/2106018942] up:boot 2024-04-24T09:25:30.178 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:29 smithi114 ceph-mon[32501]: mds.? [v2:172.21.15.90:6840/2890395646,v1:172.21.15.90:6841/2890395646] up:boot 2024-04-24T09:25:30.178 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:29 smithi114 ceph-mon[32501]: fsmap 12 up:standby 2024-04-24T09:25:30.178 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:29 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-24T09:25:30.178 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:29 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T09:25:30.178 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:29 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T09:25:30.179 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:29 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T09:25:30.179 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:29 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T09:25:30.179 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:29 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T09:25:30.179 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:29 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T09:25:30.179 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:29 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T09:25:30.179 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:29 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T09:25:30.179 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:29 smithi114 ceph-mon[32501]: from='mgr.14152 172.21.15.90:0/2274991545' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T09:25:30.179 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:29 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/4199206227' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T09:25:30.179 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:29 smithi114 ceph-mon[32501]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T09:25:30.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.233+0000 7f225f400640 1 -- 172.21.15.90:0/4199206227 <== mon.2 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v90) ==== 181+0+0 (secure 0 0 0) 0x7f22500c0ef0 con 0x7f2268109650 2024-04-24T09:25:30.234 INFO:teuthology.orchestra.run.smithi090.stderr:pool 'cephfs_metadata' removed 2024-04-24T09:25:30.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.238+0000 7f2267e00640 1 -- 172.21.15.90:0/4199206227 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f224c0a3bb0 msgr2=0x7f224c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:30.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.238+0000 7f2267e00640 1 --2- 172.21.15.90:0/4199206227 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f224c0a3bb0 0x7f224c0a6070 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f22580104e0 tx=0x7f225800b410 comp rx=0 tx=0).stop 2024-04-24T09:25:30.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.238+0000 7f2267e00640 1 -- 172.21.15.90:0/4199206227 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2268109650 msgr2=0x7f226819ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:30.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.238+0000 7f2267e00640 1 --2- 172.21.15.90:0/4199206227 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2268109650 0x7f226819ec40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f225002a930 tx=0x7f2250032ce0 comp rx=0 tx=0).stop 2024-04-24T09:25:30.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.239+0000 7f2267e00640 1 -- 172.21.15.90:0/4199206227 shutdown_connections 2024-04-24T09:25:30.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.239+0000 7f2267e00640 1 --2- 172.21.15.90:0/4199206227 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f224c0a3bb0 0x7f224c0a6070 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:30.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.239+0000 7f2267e00640 1 --2- 172.21.15.90:0/4199206227 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f226810a010 0x7f22681a5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:30.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.239+0000 7f2267e00640 1 --2- 172.21.15.90:0/4199206227 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2268109650 0x7f226819ec40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:30.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.239+0000 7f2267e00640 1 --2- 172.21.15.90:0/4199206227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2268108460 0x7f226819e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:30.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.239+0000 7f2267e00640 1 -- 172.21.15.90:0/4199206227 >> 172.21.15.90:0/4199206227 conn(0x7f2268077400 msgr2=0x7f226810c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:25:30.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.239+0000 7f2267e00640 1 -- 172.21.15.90:0/4199206227 shutdown_connections 2024-04-24T09:25:30.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.239+0000 7f2267e00640 1 -- 172.21.15.90:0/4199206227 wait complete. 2024-04-24T09:25:30.265 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-24T09:25:30.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.452+0000 7f6652600640 1 -- 172.21.15.90:0/861431366 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f664c104060 msgr2=0x7f664c104480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:30.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.452+0000 7f6652600640 1 --2- 172.21.15.90:0/861431366 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f664c104060 0x7f664c104480 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6640009860 tx=0x7f664002a340 comp rx=0 tx=0).stop 2024-04-24T09:25:30.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.453+0000 7f6652600640 1 -- 172.21.15.90:0/861431366 shutdown_connections 2024-04-24T09:25:30.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.453+0000 7f6652600640 1 --2- 172.21.15.90:0/861431366 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f664c105c10 0x7f664c10c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:30.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.453+0000 7f6652600640 1 --2- 172.21.15.90:0/861431366 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f664c105250 0x7f664c1056d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:30.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.453+0000 7f6652600640 1 --2- 172.21.15.90:0/861431366 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f664c104060 0x7f664c104480 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:30.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.453+0000 7f6652600640 1 -- 172.21.15.90:0/861431366 >> 172.21.15.90:0/861431366 conn(0x7f664c0ff5d0 msgr2=0x7f664c101a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:25:30.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.453+0000 7f6652600640 1 -- 172.21.15.90:0/861431366 shutdown_connections 2024-04-24T09:25:30.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.453+0000 7f6652600640 1 -- 172.21.15.90:0/861431366 wait complete. 2024-04-24T09:25:30.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.454+0000 7f6652600640 1 Processor -- start 2024-04-24T09:25:30.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.454+0000 7f6652600640 1 -- start start 2024-04-24T09:25:30.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.454+0000 7f6652600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f664c104060 0x7f664c19a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:25:30.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.454+0000 7f6652600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f664c105250 0x7f664c19a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:25:30.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.454+0000 7f6652600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f664c105c10 0x7f664c1a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:25:30.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.454+0000 7f6652600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f664c10fcf0 con 0x7f664c105250 2024-04-24T09:25:30.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.454+0000 7f6652600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f664c10fb70 con 0x7f664c104060 2024-04-24T09:25:30.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.454+0000 7f6652600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f664c10fe70 con 0x7f664c105c10 2024-04-24T09:25:30.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.455+0000 7f6651200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f664c104060 0x7f664c19a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:25:30.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.455+0000 7f664be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f664c105250 0x7f664c19a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:25:30.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.455+0000 7f6651200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f664c104060 0x7f664c19a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.90:53748/0 (socket says 172.21.15.90:53748) 2024-04-24T09:25:30.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.455+0000 7f6651200640 1 -- 172.21.15.90:0/2361657074 learned_addr learned my addr 172.21.15.90:0/2361657074 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-24T09:25:30.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.455+0000 7f6651c00640 1 --2- 172.21.15.90:0/2361657074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f664c105c10 0x7f664c1a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:25:30.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.455+0000 7f6651200640 1 -- 172.21.15.90:0/2361657074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f664c105c10 msgr2=0x7f664c1a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:30.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.455+0000 7f6651200640 1 --2- 172.21.15.90:0/2361657074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f664c105c10 0x7f664c1a1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:30.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.455+0000 7f6651200640 1 -- 172.21.15.90:0/2361657074 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f664c105250 msgr2=0x7f664c19a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:30.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.455+0000 7f6651200640 1 --2- 172.21.15.90:0/2361657074 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f664c105250 0x7f664c19a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:30.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.455+0000 7f6651200640 1 -- 172.21.15.90:0/2361657074 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f664c1a1fa0 con 0x7f664c104060 2024-04-24T09:25:30.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.455+0000 7f6651200640 1 --2- 172.21.15.90:0/2361657074 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f664c104060 0x7f664c19a380 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6640009780 tx=0x7f6640033b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:25:30.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.456+0000 7f6649600640 1 -- 172.21.15.90:0/2361657074 <== mon.2 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6640037040 con 0x7f664c104060 2024-04-24T09:25:30.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.456+0000 7f6649600640 1 -- 172.21.15.90:0/2361657074 <== mon.2 v2:172.21.15.114:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6640033e40 con 0x7f664c104060 2024-04-24T09:25:30.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.456+0000 7f6649600640 1 -- 172.21.15.90:0/2361657074 <== mon.2 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6640004920 con 0x7f664c104060 2024-04-24T09:25:30.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.456+0000 7f6652600640 1 -- 172.21.15.90:0/2361657074 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f664c1a2230 con 0x7f664c104060 2024-04-24T09:25:30.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.456+0000 7f6652600640 1 -- 172.21.15.90:0/2361657074 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f664c1a26d0 con 0x7f664c104060 2024-04-24T09:25:30.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.457+0000 7f6652600640 1 -- 172.21.15.90:0/2361657074 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f664c10b320 con 0x7f664c104060 2024-04-24T09:25:30.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.459+0000 7f6649600640 1 -- 172.21.15.90:0/2361657074 <== mon.2 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f6640004050 con 0x7f664c104060 2024-04-24T09:25:30.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.461+0000 7f6649600640 1 --2- 172.21.15.90:0/2361657074 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f66340a3bb0 0x7f66340a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:25:30.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.461+0000 7f6649600640 1 -- 172.21.15.90:0/2361657074 <== mon.2 v2:172.21.15.114:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8116+0+0 (secure 0 0 0) 0x7f66400f04e0 con 0x7f664c104060 2024-04-24T09:25:30.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.461+0000 7f664be00640 1 --2- 172.21.15.90:0/2361657074 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f66340a3bb0 0x7f66340a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:25:30.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.462+0000 7f664be00640 1 --2- 172.21.15.90:0/2361657074 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f66340a3bb0 0x7f66340a6070 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f664c19b890 tx=0x7f6638009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:25:30.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.463+0000 7f6649600640 1 -- 172.21.15.90:0/2361657074 <== mon.2 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f66400bcee0 con 0x7f664c104060 2024-04-24T09:25:30.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:30.668+0000 7f6652600640 1 -- 172.21.15.90:0/2361657074 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f664c02d620 con 0x7f664c104060 2024-04-24T09:25:31.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:30 smithi090 ceph-mon[27825]: pgmap v515: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.1 KiB/s wr, 11 op/s 2024-04-24T09:25:31.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:30 smithi090 ceph-mon[27825]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-24T09:25:31.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:30 smithi090 ceph-mon[27825]: osdmap e90: 12 total, 12 up, 12 in 2024-04-24T09:25:31.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:30 smithi090 ceph-mon[27825]: from='client.? 172.21.15.90:0/2361657074' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T09:25:31.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:30 smithi090 ceph-mon[27825]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T09:25:31.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:30 smithi149 ceph-mon[32592]: pgmap v515: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.1 KiB/s wr, 11 op/s 2024-04-24T09:25:31.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:30 smithi149 ceph-mon[32592]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-24T09:25:31.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:30 smithi149 ceph-mon[32592]: osdmap e90: 12 total, 12 up, 12 in 2024-04-24T09:25:31.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:30 smithi149 ceph-mon[32592]: from='client.? 172.21.15.90:0/2361657074' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T09:25:31.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:30 smithi149 ceph-mon[32592]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T09:25:31.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:30 smithi114 ceph-mon[32501]: pgmap v515: 97 pgs: 97 active+clean; 71 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.1 KiB/s wr, 11 op/s 2024-04-24T09:25:31.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:30 smithi114 ceph-mon[32501]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-24T09:25:31.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:30 smithi114 ceph-mon[32501]: osdmap e90: 12 total, 12 up, 12 in 2024-04-24T09:25:31.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:30 smithi114 ceph-mon[32501]: from='client.? 172.21.15.90:0/2361657074' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T09:25:31.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:30 smithi114 ceph-mon[32501]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T09:25:31.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:31.241+0000 7f6649600640 1 -- 172.21.15.90:0/2361657074 <== mon.2 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v91) ==== 169+0+0 (secure 0 0 0) 0x7f66400bc7e0 con 0x7f664c104060 2024-04-24T09:25:31.243 INFO:teuthology.orchestra.run.smithi090.stderr:pool 'cephfs_data' removed 2024-04-24T09:25:31.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:31.246+0000 7f6652600640 1 -- 172.21.15.90:0/2361657074 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f66340a3bb0 msgr2=0x7f66340a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:31.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:31.246+0000 7f6652600640 1 --2- 172.21.15.90:0/2361657074 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f66340a3bb0 0x7f66340a6070 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f664c19b890 tx=0x7f6638009260 comp rx=0 tx=0).stop 2024-04-24T09:25:31.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:31.246+0000 7f6652600640 1 -- 172.21.15.90:0/2361657074 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f664c104060 msgr2=0x7f664c19a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:25:31.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:31.246+0000 7f6652600640 1 --2- 172.21.15.90:0/2361657074 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f664c104060 0x7f664c19a380 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6640009780 tx=0x7f6640033b60 comp rx=0 tx=0).stop 2024-04-24T09:25:31.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:31.246+0000 7f6652600640 1 -- 172.21.15.90:0/2361657074 shutdown_connections 2024-04-24T09:25:31.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:31.246+0000 7f6652600640 1 --2- 172.21.15.90:0/2361657074 >> [v2:172.21.15.90:6800/4261662956,v1:172.21.15.90:6801/4261662956] conn(0x7f66340a3bb0 0x7f66340a6070 secure :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f664c19b890 tx=0x7f6638009260 comp rx=0 tx=0).stop 2024-04-24T09:25:31.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:31.246+0000 7f6652600640 1 --2- 172.21.15.90:0/2361657074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f664c105c10 0x7f664c1a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:31.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:31.246+0000 7f6652600640 1 --2- 172.21.15.90:0/2361657074 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f664c105250 0x7f664c19a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:31.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:31.246+0000 7f6652600640 1 --2- 172.21.15.90:0/2361657074 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f664c104060 0x7f664c19a380 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:25:31.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:31.246+0000 7f6652600640 1 -- 172.21.15.90:0/2361657074 >> 172.21.15.90:0/2361657074 conn(0x7f664c0ff5d0 msgr2=0x7f664c1084f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:25:31.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:31.247+0000 7f6652600640 1 -- 172.21.15.90:0/2361657074 shutdown_connections 2024-04-24T09:25:31.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-24T09:25:31.247+0000 7f6652600640 1 -- 172.21.15.90:0/2361657074 wait complete. 2024-04-24T09:25:31.274 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T09:25:31.346 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T09:25:31.378 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T09:25:31.407 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-24T09:25:31.408 DEBUG:teuthology.orchestra.run.smithi090:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-24T09:25:31.461 DEBUG:teuthology.orchestra.run.smithi114:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-24T09:25:31.478 DEBUG:teuthology.orchestra.run.smithi149:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-24T09:25:31.493 INFO:tasks.cephadm:Stopping all daemons... 2024-04-24T09:25:31.494 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-24T09:25:31.494 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mon.a 2024-04-24T09:25:31.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:31 smithi090 systemd[1]: Stopping Ceph mon.a for 30ee9be2-021a-11ef-bc93-c7b262605968... 2024-04-24T09:25:32.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:32 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mon-a[27821]: 2024-04-24T09:25:32.055+0000 7f6f43a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:25:32.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:32 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mon-a[27821]: 2024-04-24T09:25:32.055+0000 7f6f43a00700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-24T09:25:32.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:32 smithi090 podman[88828]: 2024-04-24 09:25:32.107990007 +0000 UTC m=+0.207699326 container died b8ed5a8fba8a52eb8bc0a64687ab90e04a2d6b7eac98cd7bf5b1f54038c4736b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0) 2024-04-24T09:25:32.703 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:32 smithi090 podman[88828]: 2024-04-24 09:25:32.691956648 +0000 UTC m=+0.791666000 container cleanup b8ed5a8fba8a52eb8bc0a64687ab90e04a2d6b7eac98cd7bf5b1f54038c4736b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T09:25:32.703 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:32 smithi090 bash[88828]: ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mon-a 2024-04-24T09:25:33.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 24 09:25:32 smithi090 podman[88842]: 2024-04-24 09:25:32.960963929 +0000 UTC m=+0.843266548 container remove b8ed5a8fba8a52eb8bc0a64687ab90e04a2d6b7eac98cd7bf5b1f54038c4736b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-24T09:25:33.460 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mon.a.service' 2024-04-24T09:25:33.593 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:25:33.593 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-24T09:25:33.594 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-24T09:25:33.594 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mon.b 2024-04-24T09:25:33.919 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:33 smithi114 systemd[1]: Stopping Ceph mon.b for 30ee9be2-021a-11ef-bc93-c7b262605968... 2024-04-24T09:25:34.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:33 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mon-b[32497]: 2024-04-24T09:25:33.919+0000 7fb20f427700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:25:34.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:33 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mon-b[32497]: 2024-04-24T09:25:33.919+0000 7fb20f427700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-24T09:25:34.176 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:34 smithi114 podman[66771]: 2024-04-24 09:25:34.021578083 +0000 UTC m=+0.228792766 container died 5d44667e66fbb1239da3e5ddc4a304f70e5efd0468274673dd9e281550c82713 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mon-b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-24T09:25:34.508 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:34 smithi114 podman[66771]: 2024-04-24 09:25:34.445124119 +0000 UTC m=+0.652338811 container cleanup 5d44667e66fbb1239da3e5ddc4a304f70e5efd0468274673dd9e281550c82713 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mon-b, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True) 2024-04-24T09:25:34.509 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:34 smithi114 bash[66771]: ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mon-b 2024-04-24T09:25:34.910 INFO:journalctl@ceph.mon.b.smithi114.stdout:Apr 24 09:25:34 smithi114 podman[66785]: 2024-04-24 09:25:34.680665267 +0000 UTC m=+0.655970298 container remove 5d44667e66fbb1239da3e5ddc4a304f70e5efd0468274673dd9e281550c82713 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mon-b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-24T09:25:34.932 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mon.b.service' 2024-04-24T09:25:34.976 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:25:34.977 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-24T09:25:34.977 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-24T09:25:34.977 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mon.c 2024-04-24T09:25:35.301 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:35 smithi149 systemd[1]: Stopping Ceph mon.c for 30ee9be2-021a-11ef-bc93-c7b262605968... 2024-04-24T09:25:35.611 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:35 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mon-c[32588]: 2024-04-24T09:25:35.301+0000 7fe22a3ae700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:25:35.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:35 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mon-c[32588]: 2024-04-24T09:25:35.301+0000 7fe22a3ae700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-24T09:25:35.612 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:35 smithi149 podman[67572]: 2024-04-24 09:25:35.495227513 +0000 UTC m=+0.321528786 container died da0fa284ae48dd2af3d150795460f8461a440656db5f563a7e576ea9614c05a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mon-c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424) 2024-04-24T09:25:36.111 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:35 smithi149 podman[67572]: 2024-04-24 09:25:35.88493178 +0000 UTC m=+0.711233051 container cleanup da0fa284ae48dd2af3d150795460f8461a440656db5f563a7e576ea9614c05a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mon-c, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-24T09:25:36.112 INFO:journalctl@ceph.mon.c.smithi149.stdout:Apr 24 09:25:35 smithi149 bash[67572]: ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mon-c 2024-04-24T09:25:36.306 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mon.c.service' 2024-04-24T09:25:36.376 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:25:36.376 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-24T09:25:36.376 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-24T09:25:36.377 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mgr.x 2024-04-24T09:25:36.852 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:25:36 smithi090 systemd[1]: Stopping Ceph mgr.x for 30ee9be2-021a-11ef-bc93-c7b262605968... 2024-04-24T09:25:37.352 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:25:37 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:25:37.045+0000 7f2dede00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:25:37.352 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:25:37 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x[28053]: 2024-04-24T09:25:37.045+0000 7f2dede00700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-24T09:25:37.353 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:25:37 smithi090 podman[88948]: 2024-04-24 09:25:37.139304809 +0000 UTC m=+0.369228866 container died 71027b126301ce439ed26339b3e252f56e5b3de33c2fec7549d1e6f82db4859e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424) 2024-04-24T09:25:38.078 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:25:37 smithi090 podman[88948]: 2024-04-24 09:25:37.79293649 +0000 UTC m=+1.022860537 container cleanup 71027b126301ce439ed26339b3e252f56e5b3de33c2fec7549d1e6f82db4859e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_CLEAN=True) 2024-04-24T09:25:38.079 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:25:37 smithi090 bash[88948]: ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x 2024-04-24T09:25:38.352 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:25:38 smithi090 podman[88962]: 2024-04-24 09:25:38.078579318 +0000 UTC m=+0.931261276 container remove 71027b126301ce439ed26339b3e252f56e5b3de33c2fec7549d1e6f82db4859e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-x, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-24T09:25:38.542 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-24T09:25:38.618 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:25:38 smithi090 systemd[1]: ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mgr.x.service: Deactivated successfully. 2024-04-24T09:25:38.618 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:25:38 smithi090 systemd[1]: Stopped Ceph mgr.x for 30ee9be2-021a-11ef-bc93-c7b262605968. 2024-04-24T09:25:38.618 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 24 09:25:38 smithi090 systemd[1]: ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mgr.x.service: Consumed 1min 44.758s CPU time. 2024-04-24T09:25:38.653 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:25:38.653 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-24T09:25:38.654 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-24T09:25:38.654 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mgr.y 2024-04-24T09:25:38.995 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Apr 24 09:25:38 smithi114 systemd[1]: Stopping Ceph mgr.y for 30ee9be2-021a-11ef-bc93-c7b262605968... 2024-04-24T09:25:39.292 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Apr 24 09:25:38 smithi114 podman[66906]: 2024-04-24 09:25:38.996106659 +0000 UTC m=+0.146339374 container died 5b4d3ed09dc7e77b99b254d6260ce6b0dc14eef6a46dc05c9cbb9ede36bc6b08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-y, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T09:25:39.643 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Apr 24 09:25:39 smithi114 podman[66906]: 2024-04-24 09:25:39.384600532 +0000 UTC m=+0.534833255 container cleanup 5b4d3ed09dc7e77b99b254d6260ce6b0dc14eef6a46dc05c9cbb9ede36bc6b08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-y, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-24T09:25:39.643 INFO:journalctl@ceph.mgr.y.smithi114.stdout:Apr 24 09:25:39 smithi114 bash[66906]: ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-y 2024-04-24T09:25:39.802 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-24T09:25:39.838 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:25:39.839 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-24T09:25:39.839 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-24T09:25:39.840 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mgr.z 2024-04-24T09:25:40.184 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Apr 24 09:25:39 smithi149 systemd[1]: Stopping Ceph mgr.z for 30ee9be2-021a-11ef-bc93-c7b262605968... 2024-04-24T09:25:40.481 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Apr 24 09:25:40 smithi149 podman[67693]: 2024-04-24 09:25:40.185388738 +0000 UTC m=+0.140948212 container died b35998b28eee3bd3be6e9b0c5aaa91998a64ef7de34f1af236eeb3a9f4744a82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-z, org.label-schema.build-date=20240424, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T09:25:40.861 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Apr 24 09:25:40 smithi149 podman[67693]: 2024-04-24 09:25:40.690183805 +0000 UTC m=+0.645743286 container cleanup b35998b28eee3bd3be6e9b0c5aaa91998a64ef7de34f1af236eeb3a9f4744a82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T09:25:40.862 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Apr 24 09:25:40 smithi149 bash[67693]: ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mgr-z 2024-04-24T09:25:41.157 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-24T09:25:41.192 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:25:41.192 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-24T09:25:41.193 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-24T09:25:41.193 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-30ee9be2-021a-11ef-bc93-c7b262605968@osd.0 2024-04-24T09:25:41.602 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 09:25:41 smithi090 systemd[1]: Stopping Ceph osd.0 for 30ee9be2-021a-11ef-bc93-c7b262605968... 2024-04-24T09:25:42.102 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 09:25:41 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-0[42170]: 2024-04-24T09:25:41.739+0000 7fa02dc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:25:42.103 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 09:25:41 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-0[42170]: 2024-04-24T09:25:41.739+0000 7fa02dc00700 -1 osd.0 91 *** Got signal Terminated *** 2024-04-24T09:25:42.103 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 09:25:41 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-0[42170]: 2024-04-24T09:25:41.739+0000 7fa02dc00700 -1 osd.0 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T09:25:47.603 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 09:25:47 smithi090 podman[89079]: 2024-04-24 09:25:47.287569182 +0000 UTC m=+5.698338064 container died f0400b31527dc5ab01df117d5ab89d0b082779bc9e4985e17f9d60185a6b7b5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-24T09:25:47.926 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 09:25:47 smithi090 podman[89079]: 2024-04-24 09:25:47.893799517 +0000 UTC m=+6.304568433 container cleanup f0400b31527dc5ab01df117d5ab89d0b082779bc9e4985e17f9d60185a6b7b5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T09:25:47.926 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 09:25:47 smithi090 bash[89079]: ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-0 2024-04-24T09:25:48.602 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 09:25:48 smithi090 podman[89099]: 2024-04-24 09:25:48.212650143 +0000 UTC m=+0.909789820 container remove f0400b31527dc5ab01df117d5ab89d0b082779bc9e4985e17f9d60185a6b7b5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-24T09:25:49.471 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 09:25:49 smithi090 podman[89193]: 2024-04-24 09:25:49.011898672 +0000 UTC m=+0.033235714 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:25:49.472 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 09:25:49 smithi090 podman[89193]: 2024-04-24 09:25:49.147272045 +0000 UTC m=+0.168609122 container create 1c2d376d66c359034697f411577d318fc9c2044395efe16e58435d817be5fb33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-0-deactivate, ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-24T09:25:49.852 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 09:25:49 smithi090 podman[89193]: 2024-04-24 09:25:49.470822766 +0000 UTC m=+0.492159900 container init 1c2d376d66c359034697f411577d318fc9c2044395efe16e58435d817be5fb33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD) 2024-04-24T09:25:49.853 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 09:25:49 smithi090 podman[89193]: 2024-04-24 09:25:49.483112997 +0000 UTC m=+0.504450043 container start 1c2d376d66c359034697f411577d318fc9c2044395efe16e58435d817be5fb33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-0-deactivate, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-24T09:25:49.853 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 09:25:49 smithi090 podman[89193]: 2024-04-24 09:25:49.568454574 +0000 UTC m=+0.589791697 container attach 1c2d376d66c359034697f411577d318fc9c2044395efe16e58435d817be5fb33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T09:25:50.335 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 09:25:49 smithi090 podman[89193]: 2024-04-24 09:25:49.941799398 +0000 UTC m=+0.963136505 container died 1c2d376d66c359034697f411577d318fc9c2044395efe16e58435d817be5fb33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T09:25:50.896 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@osd.0.service' 2024-04-24T09:25:50.973 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 09:25:50 smithi090 podman[89218]: 2024-04-24 09:25:50.841124367 +0000 UTC m=+0.880486348 container remove 1c2d376d66c359034697f411577d318fc9c2044395efe16e58435d817be5fb33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-24T09:25:50.973 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 09:25:50 smithi090 systemd[1]: ceph-30ee9be2-021a-11ef-bc93-c7b262605968@osd.0.service: Deactivated successfully. 2024-04-24T09:25:50.974 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 09:25:50 smithi090 systemd[1]: Stopped Ceph osd.0 for 30ee9be2-021a-11ef-bc93-c7b262605968. 2024-04-24T09:25:50.974 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 24 09:25:50 smithi090 systemd[1]: ceph-30ee9be2-021a-11ef-bc93-c7b262605968@osd.0.service: Consumed 19.925s CPU time. 2024-04-24T09:25:51.012 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:25:51.012 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-24T09:25:51.012 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-24T09:25:51.012 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-30ee9be2-021a-11ef-bc93-c7b262605968@osd.1 2024-04-24T09:25:51.331 INFO:journalctl@ceph.osd.1.smithi114.stdout:Apr 24 09:25:51 smithi114 systemd[1]: Stopping Ceph osd.1 for 30ee9be2-021a-11ef-bc93-c7b262605968... 2024-04-24T09:25:51.676 INFO:journalctl@ceph.osd.1.smithi114.stdout:Apr 24 09:25:51 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-1[37080]: 2024-04-24T09:25:51.330+0000 7f3a52f28700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:25:51.676 INFO:journalctl@ceph.osd.1.smithi114.stdout:Apr 24 09:25:51 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-1[37080]: 2024-04-24T09:25:51.330+0000 7f3a52f28700 -1 osd.1 91 *** Got signal Terminated *** 2024-04-24T09:25:51.676 INFO:journalctl@ceph.osd.1.smithi114.stdout:Apr 24 09:25:51 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-1[37080]: 2024-04-24T09:25:51.330+0000 7f3a52f28700 -1 osd.1 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T09:25:57.114 INFO:journalctl@ceph.osd.1.smithi114.stdout:Apr 24 09:25:56 smithi114 podman[67030]: 2024-04-24 09:25:56.782647576 +0000 UTC m=+5.575345475 container died 8ec64866bb21f7e4958a378b6fd3ce89a0eac91605037cf51e4bf1c08a4f2de5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-24T09:25:57.426 INFO:journalctl@ceph.osd.1.smithi114.stdout:Apr 24 09:25:57 smithi114 podman[67030]: 2024-04-24 09:25:57.206158613 +0000 UTC m=+5.998856508 container cleanup 8ec64866bb21f7e4958a378b6fd3ce89a0eac91605037cf51e4bf1c08a4f2de5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-1, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-24T09:25:57.426 INFO:journalctl@ceph.osd.1.smithi114.stdout:Apr 24 09:25:57 smithi114 bash[67030]: ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-1 2024-04-24T09:25:57.871 INFO:journalctl@ceph.osd.1.smithi114.stdout:Apr 24 09:25:57 smithi114 podman[67047]: 2024-04-24 09:25:57.449271157 +0000 UTC m=+0.664886317 container remove 8ec64866bb21f7e4958a378b6fd3ce89a0eac91605037cf51e4bf1c08a4f2de5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-1, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T09:25:58.152 INFO:journalctl@ceph.osd.1.smithi114.stdout:Apr 24 09:25:57 smithi114 podman[67136]: 2024-04-24 09:25:57.774909735 +0000 UTC m=+0.017886453 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:25:58.152 INFO:journalctl@ceph.osd.1.smithi114.stdout:Apr 24 09:25:57 smithi114 podman[67136]: 2024-04-24 09:25:57.900382689 +0000 UTC m=+0.143359376 container create 5e066cc93dfd9a26b786ea636b1511c6f0d70992b0bbdb326b7e0f06e753ad37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-24T09:25:58.153 INFO:journalctl@ceph.osd.1.smithi114.stdout:Apr 24 09:25:58 smithi114 podman[67136]: 2024-04-24 09:25:58.153301637 +0000 UTC m=+0.396278327 container init 5e066cc93dfd9a26b786ea636b1511c6f0d70992b0bbdb326b7e0f06e753ad37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424) 2024-04-24T09:25:58.426 INFO:journalctl@ceph.osd.1.smithi114.stdout:Apr 24 09:25:58 smithi114 podman[67136]: 2024-04-24 09:25:58.156689331 +0000 UTC m=+0.399666018 container start 5e066cc93dfd9a26b786ea636b1511c6f0d70992b0bbdb326b7e0f06e753ad37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, ceph=True) 2024-04-24T09:25:58.426 INFO:journalctl@ceph.osd.1.smithi114.stdout:Apr 24 09:25:58 smithi114 podman[67136]: 2024-04-24 09:25:58.253459867 +0000 UTC m=+0.496436554 container attach 5e066cc93dfd9a26b786ea636b1511c6f0d70992b0bbdb326b7e0f06e753ad37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T09:25:58.755 INFO:journalctl@ceph.osd.1.smithi114.stdout:Apr 24 09:25:58 smithi114 podman[67136]: 2024-04-24 09:25:58.448142357 +0000 UTC m=+0.691119040 container died 5e066cc93dfd9a26b786ea636b1511c6f0d70992b0bbdb326b7e0f06e753ad37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T09:25:59.191 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@osd.1.service' 2024-04-24T09:25:59.232 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:25:59.233 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-24T09:25:59.233 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-24T09:25:59.233 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-30ee9be2-021a-11ef-bc93-c7b262605968@osd.2 2024-04-24T09:25:59.527 INFO:journalctl@ceph.osd.2.smithi149.stdout:Apr 24 09:25:59 smithi149 systemd[1]: Stopping Ceph osd.2 for 30ee9be2-021a-11ef-bc93-c7b262605968... 2024-04-24T09:25:59.861 INFO:journalctl@ceph.osd.2.smithi149.stdout:Apr 24 09:25:59 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-2[37755]: 2024-04-24T09:25:59.526+0000 7fb328824700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:25:59.862 INFO:journalctl@ceph.osd.2.smithi149.stdout:Apr 24 09:25:59 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-2[37755]: 2024-04-24T09:25:59.526+0000 7fb328824700 -1 osd.2 91 *** Got signal Terminated *** 2024-04-24T09:25:59.862 INFO:journalctl@ceph.osd.2.smithi149.stdout:Apr 24 09:25:59 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-2[37755]: 2024-04-24T09:25:59.526+0000 7fb328824700 -1 osd.2 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T09:26:05.352 INFO:journalctl@ceph.osd.2.smithi149.stdout:Apr 24 09:26:05 smithi149 podman[67831]: 2024-04-24 09:26:05.027351044 +0000 UTC m=+5.601686527 container died b4c745b0a4907e4ed4efed0394920d96e1d7e9f5d1e26bcb90aafbfd9142473f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-24T09:26:05.352 INFO:journalctl@ceph.osd.2.smithi149.stdout:Apr 24 09:26:05 smithi149 podman[67831]: 2024-04-24 09:26:05.352762591 +0000 UTC m=+5.927098082 container cleanup b4c745b0a4907e4ed4efed0394920d96e1d7e9f5d1e26bcb90aafbfd9142473f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-24T09:26:05.611 INFO:journalctl@ceph.osd.2.smithi149.stdout:Apr 24 09:26:05 smithi149 bash[67831]: ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-2 2024-04-24T09:26:05.611 INFO:journalctl@ceph.osd.2.smithi149.stdout:Apr 24 09:26:05 smithi149 podman[67849]: 2024-04-24 09:26:05.562450772 +0000 UTC m=+0.536407209 container remove b4c745b0a4907e4ed4efed0394920d96e1d7e9f5d1e26bcb90aafbfd9142473f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-2, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T09:26:06.248 INFO:journalctl@ceph.osd.2.smithi149.stdout:Apr 24 09:26:05 smithi149 podman[67943]: 2024-04-24 09:26:05.88106922 +0000 UTC m=+0.018300305 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:26:06.249 INFO:journalctl@ceph.osd.2.smithi149.stdout:Apr 24 09:26:06 smithi149 podman[67943]: 2024-04-24 09:26:06.021832761 +0000 UTC m=+0.159063885 container create 8edbc9908bfac73c1c1ee3795451fe87efcb0ebe2a79dce24f78c8f6b54e1607 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-24T09:26:06.249 INFO:journalctl@ceph.osd.2.smithi149.stdout:Apr 24 09:26:06 smithi149 podman[67943]: 2024-04-24 09:26:06.249749526 +0000 UTC m=+0.386980608 container init 8edbc9908bfac73c1c1ee3795451fe87efcb0ebe2a79dce24f78c8f6b54e1607 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-24T09:26:06.542 INFO:journalctl@ceph.osd.2.smithi149.stdout:Apr 24 09:26:06 smithi149 podman[67943]: 2024-04-24 09:26:06.252961301 +0000 UTC m=+0.390192380 container start 8edbc9908bfac73c1c1ee3795451fe87efcb0ebe2a79dce24f78c8f6b54e1607 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True) 2024-04-24T09:26:06.543 INFO:journalctl@ceph.osd.2.smithi149.stdout:Apr 24 09:26:06 smithi149 podman[67943]: 2024-04-24 09:26:06.341380556 +0000 UTC m=+0.478611651 container attach 8edbc9908bfac73c1c1ee3795451fe87efcb0ebe2a79dce24f78c8f6b54e1607 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T09:26:06.843 INFO:journalctl@ceph.osd.2.smithi149.stdout:Apr 24 09:26:06 smithi149 podman[67943]: 2024-04-24 09:26:06.543473787 +0000 UTC m=+0.680704870 container died 8edbc9908bfac73c1c1ee3795451fe87efcb0ebe2a79dce24f78c8f6b54e1607 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_CLEAN=True) 2024-04-24T09:26:07.182 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@osd.2.service' 2024-04-24T09:26:07.223 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:26:07.223 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-24T09:26:07.223 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-24T09:26:07.223 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-30ee9be2-021a-11ef-bc93-c7b262605968@osd.3 2024-04-24T09:26:07.602 INFO:journalctl@ceph.osd.3.smithi090.stdout:Apr 24 09:26:07 smithi090 systemd[1]: Stopping Ceph osd.3 for 30ee9be2-021a-11ef-bc93-c7b262605968... 2024-04-24T09:26:08.103 INFO:journalctl@ceph.osd.3.smithi090.stdout:Apr 24 09:26:07 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-3[49069]: 2024-04-24T09:26:07.837+0000 7fd791800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:26:08.103 INFO:journalctl@ceph.osd.3.smithi090.stdout:Apr 24 09:26:07 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-3[49069]: 2024-04-24T09:26:07.837+0000 7fd791800700 -1 osd.3 91 *** Got signal Terminated *** 2024-04-24T09:26:08.103 INFO:journalctl@ceph.osd.3.smithi090.stdout:Apr 24 09:26:07 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-3[49069]: 2024-04-24T09:26:07.837+0000 7fd791800700 -1 osd.3 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T09:26:08.361 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:07 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:07.976+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:09.102 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:08 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:08.854+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:42.619691+0000 front 2024-04-24T09:25:42.619948+0000 (oldest deadline 2024-04-24T09:26:07.920133+0000) 2024-04-24T09:26:09.361 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:09 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:09.016+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:10.089 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:09 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:09.887+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:42.619691+0000 front 2024-04-24T09:25:42.619948+0000 (oldest deadline 2024-04-24T09:26:07.920133+0000) 2024-04-24T09:26:10.352 INFO:journalctl@ceph.osd.3.smithi090.stdout:Apr 24 09:26:10 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-3[49069]: 2024-04-24T09:26:10.089+0000 7fd78d000700 -1 osd.3 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:44.241973+0000 front 2024-04-24T09:25:44.242253+0000 (oldest deadline 2024-04-24T09:26:09.542199+0000) 2024-04-24T09:26:10.361 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:10 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:10.018+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:10.710 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:10 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:10.450+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:11.352 INFO:journalctl@ceph.osd.3.smithi090.stdout:Apr 24 09:26:11 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-3[49069]: 2024-04-24T09:26:11.069+0000 7fd78d000700 -1 osd.3 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:44.241973+0000 front 2024-04-24T09:25:44.242253+0000 (oldest deadline 2024-04-24T09:26:09.542199+0000) 2024-04-24T09:26:11.352 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:10 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:10.886+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:42.619691+0000 front 2024-04-24T09:25:42.619948+0000 (oldest deadline 2024-04-24T09:26:07.920133+0000) 2024-04-24T09:26:11.361 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:10 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:10.977+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:11.697 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:11 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:11.441+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:11.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:11 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:11.573+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:26:12.176 INFO:journalctl@ceph.osd.4.smithi114.stdout:Apr 24 09:26:11 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-4[44018]: 2024-04-24T09:26:11.697+0000 7f7440c6a700 -1 osd.4 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:46.112577+0000 front 2024-04-24T09:25:46.112797+0000 (oldest deadline 2024-04-24T09:26:11.412338+0000) 2024-04-24T09:26:12.352 INFO:journalctl@ceph.osd.3.smithi090.stdout:Apr 24 09:26:12 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-3[49069]: 2024-04-24T09:26:12.102+0000 7fd78d000700 -1 osd.3 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:44.241973+0000 front 2024-04-24T09:25:44.242253+0000 (oldest deadline 2024-04-24T09:26:09.542199+0000) 2024-04-24T09:26:12.352 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:11 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:11.877+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:42.619691+0000 front 2024-04-24T09:25:42.619948+0000 (oldest deadline 2024-04-24T09:26:07.920133+0000) 2024-04-24T09:26:12.361 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:12 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:12.016+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:12.676 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:12 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:12.420+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:12.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:12 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:12.542+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:26:13.176 INFO:journalctl@ceph.osd.4.smithi114.stdout:Apr 24 09:26:12 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-4[44018]: 2024-04-24T09:26:12.714+0000 7f7440c6a700 -1 osd.4 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:46.112577+0000 front 2024-04-24T09:25:46.112797+0000 (oldest deadline 2024-04-24T09:26:11.412338+0000) 2024-04-24T09:26:13.352 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:12 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:12.855+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:42.619691+0000 front 2024-04-24T09:25:42.619948+0000 (oldest deadline 2024-04-24T09:26:07.920133+0000) 2024-04-24T09:26:13.361 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:13 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:13.025+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:13.361 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 09:26:13 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-5[44708]: 2024-04-24T09:26:13.042+0000 7f2d72fa2700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:44.676657+0000 front 2024-04-24T09:25:44.676835+0000 (oldest deadline 2024-04-24T09:26:12.877136+0000) 2024-04-24T09:26:13.732 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:13 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:13.466+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:13.813 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:13 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:13.579+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:26:13.813 INFO:journalctl@ceph.osd.3.smithi090.stdout:Apr 24 09:26:13 smithi090 podman[89298]: 2024-04-24 09:26:13.420366542 +0000 UTC m=+5.794496244 container died 71f3dd397ea8aae4a83db6dfaba94285a418e8f294ee4e6d4d5a52dc6daa08e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T09:26:14.102 INFO:journalctl@ceph.osd.3.smithi090.stdout:Apr 24 09:26:14 smithi090 podman[89298]: 2024-04-24 09:26:14.022409813 +0000 UTC m=+6.396539548 container cleanup 71f3dd397ea8aae4a83db6dfaba94285a418e8f294ee4e6d4d5a52dc6daa08e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T09:26:14.102 INFO:journalctl@ceph.osd.3.smithi090.stdout:Apr 24 09:26:14 smithi090 bash[89298]: ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-3 2024-04-24T09:26:14.102 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:13 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:13.887+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:42.619691+0000 front 2024-04-24T09:25:42.619948+0000 (oldest deadline 2024-04-24T09:26:07.920133+0000) 2024-04-24T09:26:14.176 INFO:journalctl@ceph.osd.4.smithi114.stdout:Apr 24 09:26:13 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-4[44018]: 2024-04-24T09:26:13.732+0000 7f7440c6a700 -1 osd.4 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:46.112577+0000 front 2024-04-24T09:25:46.112797+0000 (oldest deadline 2024-04-24T09:26:11.412338+0000) 2024-04-24T09:26:14.361 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:14 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:14.025+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:14.362 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 09:26:14 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-5[44708]: 2024-04-24T09:26:14.086+0000 7f2d72fa2700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:44.676657+0000 front 2024-04-24T09:25:44.676835+0000 (oldest deadline 2024-04-24T09:26:12.877136+0000) 2024-04-24T09:26:14.852 INFO:journalctl@ceph.osd.3.smithi090.stdout:Apr 24 09:26:14 smithi090 podman[89329]: 2024-04-24 09:26:14.38284097 +0000 UTC m=+0.952634970 container remove 71f3dd397ea8aae4a83db6dfaba94285a418e8f294ee4e6d4d5a52dc6daa08e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-3, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T09:26:14.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:14 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:14.579+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:26:14.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:14 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:14.485+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:14.926 INFO:journalctl@ceph.osd.4.smithi114.stdout:Apr 24 09:26:14 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-4[44018]: 2024-04-24T09:26:14.691+0000 7f7440c6a700 -1 osd.4 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:46.112577+0000 front 2024-04-24T09:25:46.112797+0000 (oldest deadline 2024-04-24T09:26:11.412338+0000) 2024-04-24T09:26:15.249 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:14 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:14.897+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:42.619691+0000 front 2024-04-24T09:25:42.619948+0000 (oldest deadline 2024-04-24T09:26:07.920133+0000) 2024-04-24T09:26:15.361 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 09:26:15 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-5[44708]: 2024-04-24T09:26:15.109+0000 7f2d72fa2700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:44.676657+0000 front 2024-04-24T09:25:44.676835+0000 (oldest deadline 2024-04-24T09:26:12.877136+0000) 2024-04-24T09:26:15.362 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:14 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:14.999+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:15.542 INFO:journalctl@ceph.osd.3.smithi090.stdout:Apr 24 09:26:15 smithi090 podman[89421]: 2024-04-24 09:26:15.153681754 +0000 UTC m=+0.033405148 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:26:15.542 INFO:journalctl@ceph.osd.3.smithi090.stdout:Apr 24 09:26:15 smithi090 podman[89421]: 2024-04-24 09:26:15.277055605 +0000 UTC m=+0.156779053 container create 0df3b8a4d29047a7e3f0ed5a3f66864d99cd30f8576bc8f3f97b8b698455953c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-24T09:26:15.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:15 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:15.598+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:26:15.852 INFO:journalctl@ceph.osd.3.smithi090.stdout:Apr 24 09:26:15 smithi090 podman[89421]: 2024-04-24 09:26:15.541799098 +0000 UTC m=+0.421522569 container init 0df3b8a4d29047a7e3f0ed5a3f66864d99cd30f8576bc8f3f97b8b698455953c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-24T09:26:15.853 INFO:journalctl@ceph.osd.3.smithi090.stdout:Apr 24 09:26:15 smithi090 podman[89421]: 2024-04-24 09:26:15.553532111 +0000 UTC m=+0.433255542 container start 0df3b8a4d29047a7e3f0ed5a3f66864d99cd30f8576bc8f3f97b8b698455953c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=) 2024-04-24T09:26:15.853 INFO:journalctl@ceph.osd.3.smithi090.stdout:Apr 24 09:26:15 smithi090 podman[89421]: 2024-04-24 09:26:15.671569401 +0000 UTC m=+0.551292818 container attach 0df3b8a4d29047a7e3f0ed5a3f66864d99cd30f8576bc8f3f97b8b698455953c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-24T09:26:15.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:15 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:15.470+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:15.926 INFO:journalctl@ceph.osd.4.smithi114.stdout:Apr 24 09:26:15 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-4[44018]: 2024-04-24T09:26:15.653+0000 7f7440c6a700 -1 osd.4 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:46.112577+0000 front 2024-04-24T09:25:46.112797+0000 (oldest deadline 2024-04-24T09:26:11.412338+0000) 2024-04-24T09:26:16.352 INFO:journalctl@ceph.osd.3.smithi090.stdout:Apr 24 09:26:16 smithi090 podman[89421]: 2024-04-24 09:26:16.001878342 +0000 UTC m=+0.881601771 container died 0df3b8a4d29047a7e3f0ed5a3f66864d99cd30f8576bc8f3f97b8b698455953c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T09:26:16.352 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:15 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:15.872+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:42.619691+0000 front 2024-04-24T09:25:42.619948+0000 (oldest deadline 2024-04-24T09:26:07.920133+0000) 2024-04-24T09:26:16.361 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 09:26:16 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-5[44708]: 2024-04-24T09:26:16.128+0000 7f2d72fa2700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:44.676657+0000 front 2024-04-24T09:25:44.676835+0000 (oldest deadline 2024-04-24T09:26:12.877136+0000) 2024-04-24T09:26:16.361 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:15 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:15.985+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:16.881 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@osd.3.service' 2024-04-24T09:26:16.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:16 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:16.499+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:16.926 INFO:journalctl@ceph.osd.4.smithi114.stdout:Apr 24 09:26:16 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-4[44018]: 2024-04-24T09:26:16.632+0000 7f7440c6a700 -1 osd.4 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:46.112577+0000 front 2024-04-24T09:25:46.112797+0000 (oldest deadline 2024-04-24T09:26:11.412338+0000) 2024-04-24T09:26:16.956 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:16 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:16.629+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:26:16.957 INFO:journalctl@ceph.osd.3.smithi090.stdout:Apr 24 09:26:16 smithi090 podman[89446]: 2024-04-24 09:26:16.836360428 +0000 UTC m=+0.814814047 container remove 0df3b8a4d29047a7e3f0ed5a3f66864d99cd30f8576bc8f3f97b8b698455953c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-24T09:26:16.957 INFO:journalctl@ceph.osd.3.smithi090.stdout:Apr 24 09:26:16 smithi090 systemd[1]: ceph-30ee9be2-021a-11ef-bc93-c7b262605968@osd.3.service: Deactivated successfully. 2024-04-24T09:26:16.957 INFO:journalctl@ceph.osd.3.smithi090.stdout:Apr 24 09:26:16 smithi090 systemd[1]: Stopped Ceph osd.3 for 30ee9be2-021a-11ef-bc93-c7b262605968. 2024-04-24T09:26:16.957 INFO:journalctl@ceph.osd.3.smithi090.stdout:Apr 24 09:26:16 smithi090 systemd[1]: ceph-30ee9be2-021a-11ef-bc93-c7b262605968@osd.3.service: Consumed 20.520s CPU time. 2024-04-24T09:26:16.958 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:16 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:16.869+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:42.619691+0000 front 2024-04-24T09:25:42.619948+0000 (oldest deadline 2024-04-24T09:26:07.920133+0000) 2024-04-24T09:26:16.996 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:26:16.996 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-24T09:26:16.996 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-24T09:26:16.997 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-30ee9be2-021a-11ef-bc93-c7b262605968@osd.4 2024-04-24T09:26:17.274 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 09:26:17 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-5[44708]: 2024-04-24T09:26:17.105+0000 7f2d72fa2700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:44.676657+0000 front 2024-04-24T09:25:44.676835+0000 (oldest deadline 2024-04-24T09:26:12.877136+0000) 2024-04-24T09:26:17.275 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:17 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:17.020+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:17.318 INFO:journalctl@ceph.osd.4.smithi114.stdout:Apr 24 09:26:17 smithi114 systemd[1]: Stopping Ceph osd.4 for 30ee9be2-021a-11ef-bc93-c7b262605968... 2024-04-24T09:26:17.611 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:17 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:17.327+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.547664+0000 front 2024-04-24T09:25:54.547684+0000 (oldest deadline 2024-04-24T09:26:16.847855+0000) 2024-04-24T09:26:17.668 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:17 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:17.532+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:17.783 INFO:journalctl@ceph.osd.4.smithi114.stdout:Apr 24 09:26:17 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-4[44018]: 2024-04-24T09:26:17.318+0000 7f7444676700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:26:17.783 INFO:journalctl@ceph.osd.4.smithi114.stdout:Apr 24 09:26:17 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-4[44018]: 2024-04-24T09:26:17.318+0000 7f7444676700 -1 osd.4 91 *** Got signal Terminated *** 2024-04-24T09:26:17.783 INFO:journalctl@ceph.osd.4.smithi114.stdout:Apr 24 09:26:17 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-4[44018]: 2024-04-24T09:26:17.318+0000 7f7444676700 -1 osd.4 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T09:26:17.783 INFO:journalctl@ceph.osd.4.smithi114.stdout:Apr 24 09:26:17 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-4[44018]: 2024-04-24T09:26:17.669+0000 7f7440c6a700 -1 osd.4 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:46.112577+0000 front 2024-04-24T09:25:46.112797+0000 (oldest deadline 2024-04-24T09:26:11.412338+0000) 2024-04-24T09:26:18.103 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:17 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:17.823+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:42.619691+0000 front 2024-04-24T09:25:42.619948+0000 (oldest deadline 2024-04-24T09:26:07.920133+0000) 2024-04-24T09:26:18.104 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:17 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:17.653+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:26:18.312 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 09:26:18 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-5[44708]: 2024-04-24T09:26:18.096+0000 7f2d72fa2700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:44.676657+0000 front 2024-04-24T09:25:44.676835+0000 (oldest deadline 2024-04-24T09:26:12.877136+0000) 2024-04-24T09:26:18.313 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:18 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:18.045+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:18.313 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:18 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:18.045+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:26:18.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:17 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:17.981+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:56.271466+0000 front 2024-04-24T09:25:56.271493+0000 (oldest deadline 2024-04-24T09:26:17.371645+0000) 2024-04-24T09:26:18.611 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:18 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:18.312+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.547664+0000 front 2024-04-24T09:25:54.547684+0000 (oldest deadline 2024-04-24T09:26:16.847855+0000) 2024-04-24T09:26:18.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:18 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:18.559+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:18.927 INFO:journalctl@ceph.osd.4.smithi114.stdout:Apr 24 09:26:18 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-4[44018]: 2024-04-24T09:26:18.671+0000 7f7440c6a700 -1 osd.4 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:46.112577+0000 front 2024-04-24T09:25:46.112797+0000 (oldest deadline 2024-04-24T09:26:11.412338+0000) 2024-04-24T09:26:19.102 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:18 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:18.810+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:42.619691+0000 front 2024-04-24T09:25:42.619948+0000 (oldest deadline 2024-04-24T09:26:07.920133+0000) 2024-04-24T09:26:19.103 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:18 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:18.613+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:26:19.103 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:18 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:18.613+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.160252+0000 front 2024-04-24T09:25:54.159838+0000 (oldest deadline 2024-04-24T09:26:18.259963+0000) 2024-04-24T09:26:19.280 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 09:26:19 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-5[44708]: 2024-04-24T09:26:19.073+0000 7f2d72fa2700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:44.676657+0000 front 2024-04-24T09:25:44.676835+0000 (oldest deadline 2024-04-24T09:26:12.877136+0000) 2024-04-24T09:26:19.280 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 09:26:19 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-5[44708]: 2024-04-24T09:26:19.073+0000 7f2d72fa2700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:56.278631+0000 front 2024-04-24T09:25:56.278647+0000 (oldest deadline 2024-04-24T09:26:18.578703+0000) 2024-04-24T09:26:19.280 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:19 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:19.025+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:19.281 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:19 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:19.025+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:26:19.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:18 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:18.999+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:56.271466+0000 front 2024-04-24T09:25:56.271493+0000 (oldest deadline 2024-04-24T09:26:17.371645+0000) 2024-04-24T09:26:19.611 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:19 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:19.279+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.547664+0000 front 2024-04-24T09:25:54.547684+0000 (oldest deadline 2024-04-24T09:26:16.847855+0000) 2024-04-24T09:26:19.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:19 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:19.582+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:26:19.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:19 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:19.582+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.160252+0000 front 2024-04-24T09:25:54.159838+0000 (oldest deadline 2024-04-24T09:26:18.259963+0000) 2024-04-24T09:26:19.926 INFO:journalctl@ceph.osd.4.smithi114.stdout:Apr 24 09:26:19 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-4[44018]: 2024-04-24T09:26:19.663+0000 7f7440c6a700 -1 osd.4 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:46.112577+0000 front 2024-04-24T09:25:46.112797+0000 (oldest deadline 2024-04-24T09:26:11.412338+0000) 2024-04-24T09:26:19.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:19 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:19.589+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:19.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:19 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:19.589+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:26:20.288 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 09:26:20 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-5[44708]: 2024-04-24T09:26:20.120+0000 7f2d72fa2700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:44.676657+0000 front 2024-04-24T09:25:44.676835+0000 (oldest deadline 2024-04-24T09:26:12.877136+0000) 2024-04-24T09:26:20.289 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 09:26:20 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-5[44708]: 2024-04-24T09:26:20.120+0000 7f2d72fa2700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:56.278631+0000 front 2024-04-24T09:25:56.278647+0000 (oldest deadline 2024-04-24T09:26:18.578703+0000) 2024-04-24T09:26:20.289 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:20 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:20.029+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:20.289 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:20 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:20.029+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:26:20.352 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:19 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:19.851+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:42.619691+0000 front 2024-04-24T09:25:42.619948+0000 (oldest deadline 2024-04-24T09:26:07.920133+0000) 2024-04-24T09:26:20.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:19 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:19.985+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:56.271466+0000 front 2024-04-24T09:25:56.271493+0000 (oldest deadline 2024-04-24T09:26:17.371645+0000) 2024-04-24T09:26:20.611 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:20 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:20.289+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.547664+0000 front 2024-04-24T09:25:54.547684+0000 (oldest deadline 2024-04-24T09:26:16.847855+0000) 2024-04-24T09:26:20.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:20 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:20.592+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:26:20.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:20 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:20.592+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.160252+0000 front 2024-04-24T09:25:54.159838+0000 (oldest deadline 2024-04-24T09:26:18.259963+0000) 2024-04-24T09:26:20.926 INFO:journalctl@ceph.osd.4.smithi114.stdout:Apr 24 09:26:20 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-4[44018]: 2024-04-24T09:26:20.667+0000 7f7440c6a700 -1 osd.4 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:46.112577+0000 front 2024-04-24T09:25:46.112797+0000 (oldest deadline 2024-04-24T09:26:11.412338+0000) 2024-04-24T09:26:20.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:20 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:20.591+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:20.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:20 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:20.591+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:26:21.287 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 09:26:21 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-5[44708]: 2024-04-24T09:26:21.075+0000 7f2d72fa2700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:44.676657+0000 front 2024-04-24T09:25:44.676835+0000 (oldest deadline 2024-04-24T09:26:12.877136+0000) 2024-04-24T09:26:21.287 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 09:26:21 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-5[44708]: 2024-04-24T09:26:21.075+0000 7f2d72fa2700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:56.278631+0000 front 2024-04-24T09:25:56.278647+0000 (oldest deadline 2024-04-24T09:26:18.578703+0000) 2024-04-24T09:26:21.288 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:21 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:21.027+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:21.288 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:21 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:21.027+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:26:21.352 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:20 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:20.862+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:42.619691+0000 front 2024-04-24T09:25:42.619948+0000 (oldest deadline 2024-04-24T09:26:07.920133+0000) 2024-04-24T09:26:21.353 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:20 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:20.862+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:55.523989+0000 front 2024-04-24T09:25:55.524168+0000 (oldest deadline 2024-04-24T09:26:20.824236+0000) 2024-04-24T09:26:21.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:20 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:20.946+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:56.271466+0000 front 2024-04-24T09:25:56.271493+0000 (oldest deadline 2024-04-24T09:26:17.371645+0000) 2024-04-24T09:26:21.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:21 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:21.288+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.547664+0000 front 2024-04-24T09:25:54.547684+0000 (oldest deadline 2024-04-24T09:26:16.847855+0000) 2024-04-24T09:26:21.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:21 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:21.577+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:26:21.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:21 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:21.577+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.160252+0000 front 2024-04-24T09:25:54.159838+0000 (oldest deadline 2024-04-24T09:26:18.259963+0000) 2024-04-24T09:26:21.926 INFO:journalctl@ceph.osd.4.smithi114.stdout:Apr 24 09:26:21 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-4[44018]: 2024-04-24T09:26:21.636+0000 7f7440c6a700 -1 osd.4 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:46.112577+0000 front 2024-04-24T09:25:46.112797+0000 (oldest deadline 2024-04-24T09:26:11.412338+0000) 2024-04-24T09:26:21.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:21 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:21.627+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:21.927 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:21 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:21.627+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:26:22.352 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:21 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:21.855+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:42.619691+0000 front 2024-04-24T09:25:42.619948+0000 (oldest deadline 2024-04-24T09:26:07.920133+0000) 2024-04-24T09:26:22.352 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:21 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:21.855+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:55.523989+0000 front 2024-04-24T09:25:55.524168+0000 (oldest deadline 2024-04-24T09:26:20.824236+0000) 2024-04-24T09:26:22.361 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 09:26:22 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-5[44708]: 2024-04-24T09:26:22.085+0000 7f2d72fa2700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:44.676657+0000 front 2024-04-24T09:25:44.676835+0000 (oldest deadline 2024-04-24T09:26:12.877136+0000) 2024-04-24T09:26:22.361 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 09:26:22 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-5[44708]: 2024-04-24T09:26:22.085+0000 7f2d72fa2700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:56.278631+0000 front 2024-04-24T09:25:56.278647+0000 (oldest deadline 2024-04-24T09:26:18.578703+0000) 2024-04-24T09:26:22.362 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:22 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:22.288+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.547664+0000 front 2024-04-24T09:25:54.547684+0000 (oldest deadline 2024-04-24T09:26:16.847855+0000) 2024-04-24T09:26:22.362 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:22 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:22.070+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:22.362 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:22 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:22.070+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:26:22.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:21 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:21.926+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:56.271466+0000 front 2024-04-24T09:25:56.271493+0000 (oldest deadline 2024-04-24T09:26:17.371645+0000) 2024-04-24T09:26:22.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:22 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:22.545+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:26:22.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:22 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:22.545+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.160252+0000 front 2024-04-24T09:25:54.159838+0000 (oldest deadline 2024-04-24T09:26:18.259963+0000) 2024-04-24T09:26:22.925 INFO:journalctl@ceph.osd.4.smithi114.stdout:Apr 24 09:26:22 smithi114 podman[67269]: 2024-04-24 09:26:22.829244358 +0000 UTC m=+5.630787832 container died 31e36e901666f79417b735168f80f20f872792b25e94553cdb4d6c4e12602c3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-4, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424) 2024-04-24T09:26:22.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:22 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:22.633+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:22.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:22 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:22.633+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:26:23.176 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:22 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:22.925+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:56.271466+0000 front 2024-04-24T09:25:56.271493+0000 (oldest deadline 2024-04-24T09:26:17.371645+0000) 2024-04-24T09:26:23.352 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:22 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:22.882+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:42.619691+0000 front 2024-04-24T09:25:42.619948+0000 (oldest deadline 2024-04-24T09:26:07.920133+0000) 2024-04-24T09:26:23.352 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:22 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:22.882+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:55.523989+0000 front 2024-04-24T09:25:55.524168+0000 (oldest deadline 2024-04-24T09:26:20.824236+0000) 2024-04-24T09:26:23.545 INFO:journalctl@ceph.osd.4.smithi114.stdout:Apr 24 09:26:23 smithi114 podman[67269]: 2024-04-24 09:26:23.227348898 +0000 UTC m=+6.028892383 container cleanup 31e36e901666f79417b735168f80f20f872792b25e94553cdb4d6c4e12602c3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-4, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-24T09:26:23.545 INFO:journalctl@ceph.osd.4.smithi114.stdout:Apr 24 09:26:23 smithi114 bash[67269]: ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-4 2024-04-24T09:26:23.611 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:23 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:23.242+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.547664+0000 front 2024-04-24T09:25:54.547684+0000 (oldest deadline 2024-04-24T09:26:16.847855+0000) 2024-04-24T09:26:23.612 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 09:26:23 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-5[44708]: 2024-04-24T09:26:23.114+0000 7f2d72fa2700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:44.676657+0000 front 2024-04-24T09:25:44.676835+0000 (oldest deadline 2024-04-24T09:26:12.877136+0000) 2024-04-24T09:26:23.612 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 09:26:23 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-5[44708]: 2024-04-24T09:26:23.114+0000 7f2d72fa2700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:56.278631+0000 front 2024-04-24T09:25:56.278647+0000 (oldest deadline 2024-04-24T09:26:18.578703+0000) 2024-04-24T09:26:23.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:23 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:23.119+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:23.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:23 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:23.119+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:26:23.833 INFO:journalctl@ceph.osd.4.smithi114.stdout:Apr 24 09:26:23 smithi114 podman[67286]: 2024-04-24 09:26:23.545566839 +0000 UTC m=+0.718978855 container remove 31e36e901666f79417b735168f80f20f872792b25e94553cdb4d6c4e12602c3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-24T09:26:23.834 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:23 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:23.588+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:23.834 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:23 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:23.588+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:26:23.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:23 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:23.543+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:26:23.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:23 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:23.543+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.160252+0000 front 2024-04-24T09:25:54.159838+0000 (oldest deadline 2024-04-24T09:26:18.259963+0000) 2024-04-24T09:26:24.097 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:23 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:23.953+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:56.271466+0000 front 2024-04-24T09:25:56.271493+0000 (oldest deadline 2024-04-24T09:26:17.371645+0000) 2024-04-24T09:26:24.098 INFO:journalctl@ceph.osd.4.smithi114.stdout:Apr 24 09:26:23 smithi114 podman[67377]: 2024-04-24 09:26:23.737736435 +0000 UTC m=+0.017835948 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:26:24.098 INFO:journalctl@ceph.osd.4.smithi114.stdout:Apr 24 09:26:23 smithi114 podman[67377]: 2024-04-24 09:26:23.863202606 +0000 UTC m=+0.143302145 container create 0e65aab73adf29abe25c74bf101a02cb79da1ab93723f1c5378af35311a6f911 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-24T09:26:24.352 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:23 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:23.906+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:42.619691+0000 front 2024-04-24T09:25:42.619948+0000 (oldest deadline 2024-04-24T09:26:07.920133+0000) 2024-04-24T09:26:24.352 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:23 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:23.906+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:55.523989+0000 front 2024-04-24T09:25:55.524168+0000 (oldest deadline 2024-04-24T09:26:20.824236+0000) 2024-04-24T09:26:24.361 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:24 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:24.292+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.547664+0000 front 2024-04-24T09:25:54.547684+0000 (oldest deadline 2024-04-24T09:26:16.847855+0000) 2024-04-24T09:26:24.362 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 09:26:24 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-5[44708]: 2024-04-24T09:26:24.066+0000 7f2d72fa2700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:44.676657+0000 front 2024-04-24T09:25:44.676835+0000 (oldest deadline 2024-04-24T09:26:12.877136+0000) 2024-04-24T09:26:24.362 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 09:26:24 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-5[44708]: 2024-04-24T09:26:24.066+0000 7f2d72fa2700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:56.278631+0000 front 2024-04-24T09:25:56.278647+0000 (oldest deadline 2024-04-24T09:26:18.578703+0000) 2024-04-24T09:26:24.362 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:24 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:24.129+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:24.362 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:24 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:24.129+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:26:24.388 INFO:journalctl@ceph.osd.4.smithi114.stdout:Apr 24 09:26:24 smithi114 podman[67377]: 2024-04-24 09:26:24.098944289 +0000 UTC m=+0.379043789 container init 0e65aab73adf29abe25c74bf101a02cb79da1ab93723f1c5378af35311a6f911 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-24T09:26:24.389 INFO:journalctl@ceph.osd.4.smithi114.stdout:Apr 24 09:26:24 smithi114 podman[67377]: 2024-04-24 09:26:24.102129142 +0000 UTC m=+0.382228640 container start 0e65aab73adf29abe25c74bf101a02cb79da1ab93723f1c5378af35311a6f911 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T09:26:24.389 INFO:journalctl@ceph.osd.4.smithi114.stdout:Apr 24 09:26:24 smithi114 podman[67377]: 2024-04-24 09:26:24.199132693 +0000 UTC m=+0.479232199 container attach 0e65aab73adf29abe25c74bf101a02cb79da1ab93723f1c5378af35311a6f911 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-24T09:26:24.643 INFO:journalctl@ceph.osd.4.smithi114.stdout:Apr 24 09:26:24 smithi114 podman[67377]: 2024-04-24 09:26:24.389790842 +0000 UTC m=+0.669890340 container died 0e65aab73adf29abe25c74bf101a02cb79da1ab93723f1c5378af35311a6f911 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424) 2024-04-24T09:26:24.643 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:24 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:24.566+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:24.643 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:24 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:24.566+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:26:24.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:24 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:24.510+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:26:24.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:24 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:24.510+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.160252+0000 front 2024-04-24T09:25:54.159838+0000 (oldest deadline 2024-04-24T09:26:18.259963+0000) 2024-04-24T09:26:24.971 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@osd.4.service' 2024-04-24T09:26:25.009 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:26:25.009 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-24T09:26:25.010 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-24T09:26:25.010 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-30ee9be2-021a-11ef-bc93-c7b262605968@osd.5 2024-04-24T09:26:25.304 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:25 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:25.092+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:25.304 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:25 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:25.092+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:26:25.305 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 09:26:25 smithi149 systemd[1]: Stopping Ceph osd.5 for 30ee9be2-021a-11ef-bc93-c7b262605968... 2024-04-24T09:26:25.305 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 09:26:25 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-5[44708]: 2024-04-24T09:26:25.105+0000 7f2d72fa2700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:44.676657+0000 front 2024-04-24T09:25:44.676835+0000 (oldest deadline 2024-04-24T09:26:12.877136+0000) 2024-04-24T09:26:25.305 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 09:26:25 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-5[44708]: 2024-04-24T09:26:25.105+0000 7f2d72fa2700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:56.278631+0000 front 2024-04-24T09:25:56.278647+0000 (oldest deadline 2024-04-24T09:26:18.578703+0000) 2024-04-24T09:26:25.352 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:24 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:24.857+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:42.619691+0000 front 2024-04-24T09:25:42.619948+0000 (oldest deadline 2024-04-24T09:26:07.920133+0000) 2024-04-24T09:26:25.352 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:24 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:24.857+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:55.523989+0000 front 2024-04-24T09:25:55.524168+0000 (oldest deadline 2024-04-24T09:26:20.824236+0000) 2024-04-24T09:26:25.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:25 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:25.002+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:56.271466+0000 front 2024-04-24T09:25:56.271493+0000 (oldest deadline 2024-04-24T09:26:17.371645+0000) 2024-04-24T09:26:25.611 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:25 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:25.305+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.547664+0000 front 2024-04-24T09:25:54.547684+0000 (oldest deadline 2024-04-24T09:26:16.847855+0000) 2024-04-24T09:26:25.612 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 09:26:25 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-5[44708]: 2024-04-24T09:26:25.332+0000 7f2d7a9b6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:26:25.612 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 09:26:25 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-5[44708]: 2024-04-24T09:26:25.332+0000 7f2d7a9b6700 -1 osd.5 91 *** Got signal Terminated *** 2024-04-24T09:26:25.612 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 09:26:25 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-5[44708]: 2024-04-24T09:26:25.332+0000 7f2d7a9b6700 -1 osd.5 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T09:26:25.851 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:25 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:25.522+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:26:25.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:25 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:25.522+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.160252+0000 front 2024-04-24T09:25:54.159838+0000 (oldest deadline 2024-04-24T09:26:18.259963+0000) 2024-04-24T09:26:25.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:25 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:25.520+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:25.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:25 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:25.520+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:26:26.102 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:25 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:25.851+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:42.619691+0000 front 2024-04-24T09:25:42.619948+0000 (oldest deadline 2024-04-24T09:26:07.920133+0000) 2024-04-24T09:26:26.102 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:25 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:25.851+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:55.523989+0000 front 2024-04-24T09:25:55.524168+0000 (oldest deadline 2024-04-24T09:26:20.824236+0000) 2024-04-24T09:26:26.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:26 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:26.052+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:56.271466+0000 front 2024-04-24T09:25:56.271493+0000 (oldest deadline 2024-04-24T09:26:17.371645+0000) 2024-04-24T09:26:26.611 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:26 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:26.340+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.547664+0000 front 2024-04-24T09:25:54.547684+0000 (oldest deadline 2024-04-24T09:26:16.847855+0000) 2024-04-24T09:26:26.612 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 09:26:26 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-5[44708]: 2024-04-24T09:26:26.128+0000 7f2d72fa2700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:44.676657+0000 front 2024-04-24T09:25:44.676835+0000 (oldest deadline 2024-04-24T09:26:12.877136+0000) 2024-04-24T09:26:26.612 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 09:26:26 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-5[44708]: 2024-04-24T09:26:26.128+0000 7f2d72fa2700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:56.278631+0000 front 2024-04-24T09:25:56.278647+0000 (oldest deadline 2024-04-24T09:26:18.578703+0000) 2024-04-24T09:26:26.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:26 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:26.128+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:26.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:26 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:26.128+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:26:26.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:26 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:26.508+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:26:26.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:26 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:26.508+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.160252+0000 front 2024-04-24T09:25:54.159838+0000 (oldest deadline 2024-04-24T09:26:18.259963+0000) 2024-04-24T09:26:26.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:26 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:26.508+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.764867+0000 front 2024-04-24T09:26:02.764831+0000 (oldest deadline 2024-04-24T09:26:26.264981+0000) 2024-04-24T09:26:26.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:26 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:26.504+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:26.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:26 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:26.504+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:26:27.352 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:26 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:26.890+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:42.619691+0000 front 2024-04-24T09:25:42.619948+0000 (oldest deadline 2024-04-24T09:26:07.920133+0000) 2024-04-24T09:26:27.352 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:26 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:26.890+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:55.523989+0000 front 2024-04-24T09:25:55.524168+0000 (oldest deadline 2024-04-24T09:26:20.824236+0000) 2024-04-24T09:26:27.361 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:27 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:27.336+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.547664+0000 front 2024-04-24T09:25:54.547684+0000 (oldest deadline 2024-04-24T09:26:16.847855+0000) 2024-04-24T09:26:27.362 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 09:26:27 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-5[44708]: 2024-04-24T09:26:27.109+0000 7f2d72fa2700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:44.676657+0000 front 2024-04-24T09:25:44.676835+0000 (oldest deadline 2024-04-24T09:26:12.877136+0000) 2024-04-24T09:26:27.362 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 09:26:27 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-5[44708]: 2024-04-24T09:26:27.109+0000 7f2d72fa2700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:56.278631+0000 front 2024-04-24T09:25:56.278647+0000 (oldest deadline 2024-04-24T09:26:18.578703+0000) 2024-04-24T09:26:27.362 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:27 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:27.139+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:27.362 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:27 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:27.139+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:26:27.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:27 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:27.048+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:56.271466+0000 front 2024-04-24T09:25:56.271493+0000 (oldest deadline 2024-04-24T09:26:17.371645+0000) 2024-04-24T09:26:27.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:27 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:27.048+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.573220+0000 front 2024-04-24T09:26:02.573169+0000 (oldest deadline 2024-04-24T09:26:26.073363+0000) 2024-04-24T09:26:27.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:27 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:27.481+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:26:27.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:27 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:27.481+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.160252+0000 front 2024-04-24T09:25:54.159838+0000 (oldest deadline 2024-04-24T09:26:18.259963+0000) 2024-04-24T09:26:27.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:27 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:27.481+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.764867+0000 front 2024-04-24T09:26:02.764831+0000 (oldest deadline 2024-04-24T09:26:26.264981+0000) 2024-04-24T09:26:27.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:27 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:27.551+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:27.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:27 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:27.551+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:26:27.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:27 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:27.551+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:03.168384+0000 front 2024-04-24T09:26:03.168290+0000 (oldest deadline 2024-04-24T09:26:27.268338+0000) 2024-04-24T09:26:28.352 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:27 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:27.858+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:42.619691+0000 front 2024-04-24T09:25:42.619948+0000 (oldest deadline 2024-04-24T09:26:07.920133+0000) 2024-04-24T09:26:28.352 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:27 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:27.858+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:55.523989+0000 front 2024-04-24T09:25:55.524168+0000 (oldest deadline 2024-04-24T09:26:20.824236+0000) 2024-04-24T09:26:28.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:28 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:28.084+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:56.271466+0000 front 2024-04-24T09:25:56.271493+0000 (oldest deadline 2024-04-24T09:26:17.371645+0000) 2024-04-24T09:26:28.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:28 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:28.084+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.573220+0000 front 2024-04-24T09:26:02.573169+0000 (oldest deadline 2024-04-24T09:26:26.073363+0000) 2024-04-24T09:26:28.611 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:28 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:28.293+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.547664+0000 front 2024-04-24T09:25:54.547684+0000 (oldest deadline 2024-04-24T09:26:16.847855+0000) 2024-04-24T09:26:28.612 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 09:26:28 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-5[44708]: 2024-04-24T09:26:28.154+0000 7f2d72fa2700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:44.676657+0000 front 2024-04-24T09:25:44.676835+0000 (oldest deadline 2024-04-24T09:26:12.877136+0000) 2024-04-24T09:26:28.612 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 09:26:28 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-5[44708]: 2024-04-24T09:26:28.154+0000 7f2d72fa2700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:56.278631+0000 front 2024-04-24T09:25:56.278647+0000 (oldest deadline 2024-04-24T09:26:18.578703+0000) 2024-04-24T09:26:28.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:28 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:28.175+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:28.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:28 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:28.175+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:26:28.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:28 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:28.449+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:26:28.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:28 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:28.449+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.160252+0000 front 2024-04-24T09:25:54.159838+0000 (oldest deadline 2024-04-24T09:26:18.259963+0000) 2024-04-24T09:26:28.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:28 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:28.449+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.764867+0000 front 2024-04-24T09:26:02.764831+0000 (oldest deadline 2024-04-24T09:26:26.264981+0000) 2024-04-24T09:26:28.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:28 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:28.591+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:28.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:28 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:28.591+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:26:28.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:28 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:28.591+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:03.168384+0000 front 2024-04-24T09:26:03.168290+0000 (oldest deadline 2024-04-24T09:26:27.268338+0000) 2024-04-24T09:26:29.352 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:28 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:28.901+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:42.619691+0000 front 2024-04-24T09:25:42.619948+0000 (oldest deadline 2024-04-24T09:26:07.920133+0000) 2024-04-24T09:26:29.352 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:28 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:28.901+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:55.523989+0000 front 2024-04-24T09:25:55.524168+0000 (oldest deadline 2024-04-24T09:26:20.824236+0000) 2024-04-24T09:26:29.352 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:28 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:28.901+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.526723+0000 front 2024-04-24T09:26:02.526595+0000 (oldest deadline 2024-04-24T09:26:28.426856+0000) 2024-04-24T09:26:29.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:29 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:29.110+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:56.271466+0000 front 2024-04-24T09:25:56.271493+0000 (oldest deadline 2024-04-24T09:26:17.371645+0000) 2024-04-24T09:26:29.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:29 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:29.110+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.573220+0000 front 2024-04-24T09:26:02.573169+0000 (oldest deadline 2024-04-24T09:26:26.073363+0000) 2024-04-24T09:26:29.611 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 09:26:29 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-5[44708]: 2024-04-24T09:26:29.152+0000 7f2d72fa2700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:44.676657+0000 front 2024-04-24T09:25:44.676835+0000 (oldest deadline 2024-04-24T09:26:12.877136+0000) 2024-04-24T09:26:29.612 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 09:26:29 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-5[44708]: 2024-04-24T09:26:29.152+0000 7f2d72fa2700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:56.278631+0000 front 2024-04-24T09:25:56.278647+0000 (oldest deadline 2024-04-24T09:26:18.578703+0000) 2024-04-24T09:26:29.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:29 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:29.282+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.547664+0000 front 2024-04-24T09:25:54.547684+0000 (oldest deadline 2024-04-24T09:26:16.847855+0000) 2024-04-24T09:26:29.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:29 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:29.282+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:04.350005+0000 front 2024-04-24T09:26:04.349959+0000 (oldest deadline 2024-04-24T09:26:28.450238+0000) 2024-04-24T09:26:29.613 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:29 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:29.200+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:29.613 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:29 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:29.200+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:26:29.613 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:29 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:29.200+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:26:29.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:29 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:29.493+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:26:29.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:29 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:29.493+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.160252+0000 front 2024-04-24T09:25:54.159838+0000 (oldest deadline 2024-04-24T09:26:18.259963+0000) 2024-04-24T09:26:29.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:29 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:29.493+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.764867+0000 front 2024-04-24T09:26:02.764831+0000 (oldest deadline 2024-04-24T09:26:26.264981+0000) 2024-04-24T09:26:29.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:29 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:29.596+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:29.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:29 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:29.596+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:26:29.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:29 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:29.596+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:03.168384+0000 front 2024-04-24T09:26:03.168290+0000 (oldest deadline 2024-04-24T09:26:27.268338+0000) 2024-04-24T09:26:30.352 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:29 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:29.864+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:42.619691+0000 front 2024-04-24T09:25:42.619948+0000 (oldest deadline 2024-04-24T09:26:07.920133+0000) 2024-04-24T09:26:30.353 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:29 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:29.864+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:55.523989+0000 front 2024-04-24T09:25:55.524168+0000 (oldest deadline 2024-04-24T09:26:20.824236+0000) 2024-04-24T09:26:30.353 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:29 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:29.864+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.526723+0000 front 2024-04-24T09:26:02.526595+0000 (oldest deadline 2024-04-24T09:26:28.426856+0000) 2024-04-24T09:26:30.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:30 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:30.074+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:56.271466+0000 front 2024-04-24T09:25:56.271493+0000 (oldest deadline 2024-04-24T09:26:17.371645+0000) 2024-04-24T09:26:30.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:30 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:30.074+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.573220+0000 front 2024-04-24T09:26:02.573169+0000 (oldest deadline 2024-04-24T09:26:26.073363+0000) 2024-04-24T09:26:30.611 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:30 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:30.242+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.547664+0000 front 2024-04-24T09:25:54.547684+0000 (oldest deadline 2024-04-24T09:26:16.847855+0000) 2024-04-24T09:26:30.611 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:30 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:30.242+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:04.350005+0000 front 2024-04-24T09:26:04.349959+0000 (oldest deadline 2024-04-24T09:26:28.450238+0000) 2024-04-24T09:26:30.612 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 09:26:30 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-5[44708]: 2024-04-24T09:26:30.198+0000 7f2d72fa2700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:44.676657+0000 front 2024-04-24T09:25:44.676835+0000 (oldest deadline 2024-04-24T09:26:12.877136+0000) 2024-04-24T09:26:30.612 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 09:26:30 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-5[44708]: 2024-04-24T09:26:30.198+0000 7f2d72fa2700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:56.278631+0000 front 2024-04-24T09:25:56.278647+0000 (oldest deadline 2024-04-24T09:26:18.578703+0000) 2024-04-24T09:26:30.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:30 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:30.237+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:30.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:30 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:30.237+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:26:30.613 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:30 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:30.237+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:26:30.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:30 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:30.481+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:26:30.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:30 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:30.481+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.160252+0000 front 2024-04-24T09:25:54.159838+0000 (oldest deadline 2024-04-24T09:26:18.259963+0000) 2024-04-24T09:26:30.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:30 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:30.481+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.764867+0000 front 2024-04-24T09:26:02.764831+0000 (oldest deadline 2024-04-24T09:26:26.264981+0000) 2024-04-24T09:26:30.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:30 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:30.589+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:30.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:30 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:30.589+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:26:30.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:30 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:30.589+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:03.168384+0000 front 2024-04-24T09:26:03.168290+0000 (oldest deadline 2024-04-24T09:26:27.268338+0000) 2024-04-24T09:26:30.975 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 09:26:30 smithi149 podman[68063]: 2024-04-24 09:26:30.675737936 +0000 UTC m=+5.469249235 container died 4506cdc0ee7dee9663db759636a3ba7c09254d57867d39726666d386957b8e46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T09:26:31.262 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 09:26:31 smithi149 podman[68063]: 2024-04-24 09:26:31.075156488 +0000 UTC m=+5.868667796 container cleanup 4506cdc0ee7dee9663db759636a3ba7c09254d57867d39726666d386957b8e46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T09:26:31.262 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 09:26:31 smithi149 bash[68063]: ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-5 2024-04-24T09:26:31.352 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:30 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:30.883+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:42.619691+0000 front 2024-04-24T09:25:42.619948+0000 (oldest deadline 2024-04-24T09:26:07.920133+0000) 2024-04-24T09:26:31.352 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:30 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:30.883+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:55.523989+0000 front 2024-04-24T09:25:55.524168+0000 (oldest deadline 2024-04-24T09:26:20.824236+0000) 2024-04-24T09:26:31.352 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:30 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:30.883+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.526723+0000 front 2024-04-24T09:26:02.526595+0000 (oldest deadline 2024-04-24T09:26:28.426856+0000) 2024-04-24T09:26:31.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:31 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:31.061+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:56.271466+0000 front 2024-04-24T09:25:56.271493+0000 (oldest deadline 2024-04-24T09:26:17.371645+0000) 2024-04-24T09:26:31.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:31 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:31.061+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.573220+0000 front 2024-04-24T09:26:02.573169+0000 (oldest deadline 2024-04-24T09:26:26.073363+0000) 2024-04-24T09:26:31.611 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 09:26:31 smithi149 podman[68080]: 2024-04-24 09:26:31.326662324 +0000 UTC m=+0.653416390 container remove 4506cdc0ee7dee9663db759636a3ba7c09254d57867d39726666d386957b8e46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-5, org.label-schema.schema-version=1.0, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-24T09:26:31.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:31 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:31.262+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.547664+0000 front 2024-04-24T09:25:54.547684+0000 (oldest deadline 2024-04-24T09:26:16.847855+0000) 2024-04-24T09:26:31.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:31 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:31.262+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:04.350005+0000 front 2024-04-24T09:26:04.349959+0000 (oldest deadline 2024-04-24T09:26:28.450238+0000) 2024-04-24T09:26:31.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:31 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:31.267+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:31.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:31 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:31.267+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:26:31.613 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:31 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:31.267+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:26:31.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:31 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:31.519+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:26:31.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:31 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:31.519+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.160252+0000 front 2024-04-24T09:25:54.159838+0000 (oldest deadline 2024-04-24T09:26:18.259963+0000) 2024-04-24T09:26:31.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:31 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:31.519+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.764867+0000 front 2024-04-24T09:26:02.764831+0000 (oldest deadline 2024-04-24T09:26:26.264981+0000) 2024-04-24T09:26:31.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:31 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:31.604+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:31.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:31 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:31.604+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:26:31.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:31 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:31.604+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:03.168384+0000 front 2024-04-24T09:26:03.168290+0000 (oldest deadline 2024-04-24T09:26:27.268338+0000) 2024-04-24T09:26:31.996 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 09:26:31 smithi149 podman[68168]: 2024-04-24 09:26:31.593008843 +0000 UTC m=+0.018345053 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:26:31.996 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 09:26:31 smithi149 podman[68168]: 2024-04-24 09:26:31.711014287 +0000 UTC m=+0.136350485 container create c4b55c354554dbe6fe2d8b5d81e1c80bfc542f67d73c74b15858a697f3c6b69a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-24T09:26:31.997 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 09:26:31 smithi149 podman[68168]: 2024-04-24 09:26:31.997438999 +0000 UTC m=+0.422775196 container init c4b55c354554dbe6fe2d8b5d81e1c80bfc542f67d73c74b15858a697f3c6b69a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-24T09:26:32.299 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:32 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:32.246+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.547664+0000 front 2024-04-24T09:25:54.547684+0000 (oldest deadline 2024-04-24T09:26:16.847855+0000) 2024-04-24T09:26:32.299 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:32 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:32.246+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:04.350005+0000 front 2024-04-24T09:26:04.349959+0000 (oldest deadline 2024-04-24T09:26:28.450238+0000) 2024-04-24T09:26:32.300 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 09:26:32 smithi149 podman[68168]: 2024-04-24 09:26:32.000653731 +0000 UTC m=+0.425989936 container start c4b55c354554dbe6fe2d8b5d81e1c80bfc542f67d73c74b15858a697f3c6b69a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-5-deactivate, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-24T09:26:32.300 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 09:26:32 smithi149 podman[68168]: 2024-04-24 09:26:32.09772068 +0000 UTC m=+0.523056880 container attach c4b55c354554dbe6fe2d8b5d81e1c80bfc542f67d73c74b15858a697f3c6b69a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T09:26:32.352 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:31 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:31.907+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:42.619691+0000 front 2024-04-24T09:25:42.619948+0000 (oldest deadline 2024-04-24T09:26:07.920133+0000) 2024-04-24T09:26:32.352 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:31 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:31.907+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:55.523989+0000 front 2024-04-24T09:25:55.524168+0000 (oldest deadline 2024-04-24T09:26:20.824236+0000) 2024-04-24T09:26:32.352 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:31 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:31.907+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.526723+0000 front 2024-04-24T09:26:02.526595+0000 (oldest deadline 2024-04-24T09:26:28.426856+0000) 2024-04-24T09:26:32.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:32 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:32.077+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:56.271466+0000 front 2024-04-24T09:25:56.271493+0000 (oldest deadline 2024-04-24T09:26:17.371645+0000) 2024-04-24T09:26:32.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:32 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:32.077+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.573220+0000 front 2024-04-24T09:26:02.573169+0000 (oldest deadline 2024-04-24T09:26:26.073363+0000) 2024-04-24T09:26:32.600 INFO:journalctl@ceph.osd.5.smithi149.stdout:Apr 24 09:26:32 smithi149 podman[68168]: 2024-04-24 09:26:32.308630011 +0000 UTC m=+0.733966209 container died c4b55c354554dbe6fe2d8b5d81e1c80bfc542f67d73c74b15858a697f3c6b69a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424) 2024-04-24T09:26:32.600 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:32 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:32.298+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:32.600 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:32 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:32.298+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:26:32.600 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:32 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:32.298+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:26:32.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:32 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:32.552+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:26:32.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:32 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:32.552+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.160252+0000 front 2024-04-24T09:25:54.159838+0000 (oldest deadline 2024-04-24T09:26:18.259963+0000) 2024-04-24T09:26:32.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:32 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:32.552+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.764867+0000 front 2024-04-24T09:26:02.764831+0000 (oldest deadline 2024-04-24T09:26:26.264981+0000) 2024-04-24T09:26:32.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:32 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:32.577+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:32.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:32 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:32.577+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:26:32.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:32 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:32.577+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:03.168384+0000 front 2024-04-24T09:26:03.168290+0000 (oldest deadline 2024-04-24T09:26:27.268338+0000) 2024-04-24T09:26:32.979 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@osd.5.service' 2024-04-24T09:26:33.016 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:26:33.016 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-24T09:26:33.017 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-24T09:26:33.017 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-30ee9be2-021a-11ef-bc93-c7b262605968@osd.6 2024-04-24T09:26:33.352 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:32 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:32.917+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:42.619691+0000 front 2024-04-24T09:25:42.619948+0000 (oldest deadline 2024-04-24T09:26:07.920133+0000) 2024-04-24T09:26:33.352 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:32 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:32.917+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:55.523989+0000 front 2024-04-24T09:25:55.524168+0000 (oldest deadline 2024-04-24T09:26:20.824236+0000) 2024-04-24T09:26:33.352 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:32 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:32.917+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.526723+0000 front 2024-04-24T09:26:02.526595+0000 (oldest deadline 2024-04-24T09:26:28.426856+0000) 2024-04-24T09:26:33.353 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:33 smithi090 systemd[1]: Stopping Ceph osd.6 for 30ee9be2-021a-11ef-bc93-c7b262605968... 2024-04-24T09:26:33.427 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:33 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:33.115+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:56.271466+0000 front 2024-04-24T09:25:56.271493+0000 (oldest deadline 2024-04-24T09:26:17.371645+0000) 2024-04-24T09:26:33.427 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:33 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:33.115+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.573220+0000 front 2024-04-24T09:26:02.573169+0000 (oldest deadline 2024-04-24T09:26:26.073363+0000) 2024-04-24T09:26:33.611 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:33 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:33.233+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.547664+0000 front 2024-04-24T09:25:54.547684+0000 (oldest deadline 2024-04-24T09:26:16.847855+0000) 2024-04-24T09:26:33.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:33 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:33.233+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:04.350005+0000 front 2024-04-24T09:26:04.349959+0000 (oldest deadline 2024-04-24T09:26:28.450238+0000) 2024-04-24T09:26:33.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:33 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:33.255+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:33.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:33 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:33.255+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:26:33.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:33 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:33.255+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:26:33.852 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:33 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:33.638+0000 7f3eca000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:26:33.852 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:33 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:33.638+0000 7f3eca000700 -1 osd.6 91 *** Got signal Terminated *** 2024-04-24T09:26:33.853 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:33 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:33.638+0000 7f3eca000700 -1 osd.6 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T09:26:33.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:33 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:33.530+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:26:33.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:33 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:33.530+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.160252+0000 front 2024-04-24T09:25:54.159838+0000 (oldest deadline 2024-04-24T09:26:18.259963+0000) 2024-04-24T09:26:33.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:33 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:33.530+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.764867+0000 front 2024-04-24T09:26:02.764831+0000 (oldest deadline 2024-04-24T09:26:26.264981+0000) 2024-04-24T09:26:33.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:33 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:33.543+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:33.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:33 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:33.543+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:26:33.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:33 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:33.543+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:03.168384+0000 front 2024-04-24T09:26:03.168290+0000 (oldest deadline 2024-04-24T09:26:27.268338+0000) 2024-04-24T09:26:34.352 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:33 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:33.911+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:42.619691+0000 front 2024-04-24T09:25:42.619948+0000 (oldest deadline 2024-04-24T09:26:07.920133+0000) 2024-04-24T09:26:34.352 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:33 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:33.911+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:55.523989+0000 front 2024-04-24T09:25:55.524168+0000 (oldest deadline 2024-04-24T09:26:20.824236+0000) 2024-04-24T09:26:34.353 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:33 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:33.911+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.526723+0000 front 2024-04-24T09:26:02.526595+0000 (oldest deadline 2024-04-24T09:26:28.426856+0000) 2024-04-24T09:26:34.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:34 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:34.129+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:56.271466+0000 front 2024-04-24T09:25:56.271493+0000 (oldest deadline 2024-04-24T09:26:17.371645+0000) 2024-04-24T09:26:34.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:34 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:34.129+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.573220+0000 front 2024-04-24T09:26:02.573169+0000 (oldest deadline 2024-04-24T09:26:26.073363+0000) 2024-04-24T09:26:34.611 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:34 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:34.208+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.547664+0000 front 2024-04-24T09:25:54.547684+0000 (oldest deadline 2024-04-24T09:26:16.847855+0000) 2024-04-24T09:26:34.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:34 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:34.208+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:04.350005+0000 front 2024-04-24T09:26:04.349959+0000 (oldest deadline 2024-04-24T09:26:28.450238+0000) 2024-04-24T09:26:34.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:34 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:34.239+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:34.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:34 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:34.239+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:26:34.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:34 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:34.239+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:26:34.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:34 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:34.521+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:26:34.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:34 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:34.521+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.160252+0000 front 2024-04-24T09:25:54.159838+0000 (oldest deadline 2024-04-24T09:26:18.259963+0000) 2024-04-24T09:26:34.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:34 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:34.521+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.764867+0000 front 2024-04-24T09:26:02.764831+0000 (oldest deadline 2024-04-24T09:26:26.264981+0000) 2024-04-24T09:26:34.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:34 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:34.543+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:34.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:34 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:34.543+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:26:34.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:34 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:34.543+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:03.168384+0000 front 2024-04-24T09:26:03.168290+0000 (oldest deadline 2024-04-24T09:26:27.268338+0000) 2024-04-24T09:26:34.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:34 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:34.543+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.369549+0000 front 2024-04-24T09:26:11.369459+0000 (oldest deadline 2024-04-24T09:26:33.669296+0000) 2024-04-24T09:26:35.352 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:34 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:34.935+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:42.619691+0000 front 2024-04-24T09:25:42.619948+0000 (oldest deadline 2024-04-24T09:26:07.920133+0000) 2024-04-24T09:26:35.352 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:34 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:34.935+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:55.523989+0000 front 2024-04-24T09:25:55.524168+0000 (oldest deadline 2024-04-24T09:26:20.824236+0000) 2024-04-24T09:26:35.353 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:34 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:34.935+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.526723+0000 front 2024-04-24T09:26:02.526595+0000 (oldest deadline 2024-04-24T09:26:28.426856+0000) 2024-04-24T09:26:35.353 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:34 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:34.935+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:08.427956+0000 front 2024-04-24T09:26:08.428100+0000 (oldest deadline 2024-04-24T09:26:34.328195+0000) 2024-04-24T09:26:35.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:35 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:35.158+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:56.271466+0000 front 2024-04-24T09:25:56.271493+0000 (oldest deadline 2024-04-24T09:26:17.371645+0000) 2024-04-24T09:26:35.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:35 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:35.158+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.573220+0000 front 2024-04-24T09:26:02.573169+0000 (oldest deadline 2024-04-24T09:26:26.073363+0000) 2024-04-24T09:26:35.611 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:35 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:35.236+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.547664+0000 front 2024-04-24T09:25:54.547684+0000 (oldest deadline 2024-04-24T09:26:16.847855+0000) 2024-04-24T09:26:35.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:35 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:35.236+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:04.350005+0000 front 2024-04-24T09:26:04.349959+0000 (oldest deadline 2024-04-24T09:26:28.450238+0000) 2024-04-24T09:26:35.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:35 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:35.218+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:35.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:35 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:35.218+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:26:35.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:35 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:35.218+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:26:35.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:35 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:35.548+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:26:35.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:35 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:35.548+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.160252+0000 front 2024-04-24T09:25:54.159838+0000 (oldest deadline 2024-04-24T09:26:18.259963+0000) 2024-04-24T09:26:35.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:35 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:35.548+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.764867+0000 front 2024-04-24T09:26:02.764831+0000 (oldest deadline 2024-04-24T09:26:26.264981+0000) 2024-04-24T09:26:35.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:35 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:35.515+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:35.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:35 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:35.515+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:26:35.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:35 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:35.515+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:03.168384+0000 front 2024-04-24T09:26:03.168290+0000 (oldest deadline 2024-04-24T09:26:27.268338+0000) 2024-04-24T09:26:35.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:35 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:35.515+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.369549+0000 front 2024-04-24T09:26:11.369459+0000 (oldest deadline 2024-04-24T09:26:33.669296+0000) 2024-04-24T09:26:36.352 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:35 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:35.947+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:42.619691+0000 front 2024-04-24T09:25:42.619948+0000 (oldest deadline 2024-04-24T09:26:07.920133+0000) 2024-04-24T09:26:36.352 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:35 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:35.947+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:55.523989+0000 front 2024-04-24T09:25:55.524168+0000 (oldest deadline 2024-04-24T09:26:20.824236+0000) 2024-04-24T09:26:36.352 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:35 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:35.947+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.526723+0000 front 2024-04-24T09:26:02.526595+0000 (oldest deadline 2024-04-24T09:26:28.426856+0000) 2024-04-24T09:26:36.352 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:35 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:35.947+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:08.427956+0000 front 2024-04-24T09:26:08.428100+0000 (oldest deadline 2024-04-24T09:26:34.328195+0000) 2024-04-24T09:26:36.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:36 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:36.138+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:56.271466+0000 front 2024-04-24T09:25:56.271493+0000 (oldest deadline 2024-04-24T09:26:17.371645+0000) 2024-04-24T09:26:36.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:36 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:36.138+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.573220+0000 front 2024-04-24T09:26:02.573169+0000 (oldest deadline 2024-04-24T09:26:26.073363+0000) 2024-04-24T09:26:36.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:36 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:36.138+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.475474+0000 front 2024-04-24T09:26:12.475548+0000 (oldest deadline 2024-04-24T09:26:35.375296+0000) 2024-04-24T09:26:36.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:36 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:36.192+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.547664+0000 front 2024-04-24T09:25:54.547684+0000 (oldest deadline 2024-04-24T09:26:16.847855+0000) 2024-04-24T09:26:36.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:36 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:36.192+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:04.350005+0000 front 2024-04-24T09:26:04.349959+0000 (oldest deadline 2024-04-24T09:26:28.450238+0000) 2024-04-24T09:26:36.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:36 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:36.177+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:36.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:36 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:36.177+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:26:36.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:36 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:36.177+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:26:36.613 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:36 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:36.177+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.811765+0000 front 2024-04-24T09:26:12.811682+0000 (oldest deadline 2024-04-24T09:26:35.711456+0000) 2024-04-24T09:26:36.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:36 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:36.530+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:26:36.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:36 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:36.530+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.160252+0000 front 2024-04-24T09:25:54.159838+0000 (oldest deadline 2024-04-24T09:26:18.259963+0000) 2024-04-24T09:26:36.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:36 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:36.530+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.764867+0000 front 2024-04-24T09:26:02.764831+0000 (oldest deadline 2024-04-24T09:26:26.264981+0000) 2024-04-24T09:26:36.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:36 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:36.530+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.567402+0000 front 2024-04-24T09:26:11.567591+0000 (oldest deadline 2024-04-24T09:26:36.267266+0000) 2024-04-24T09:26:36.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:36 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:36.482+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:36.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:36 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:36.482+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:26:36.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:36 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:36.482+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:03.168384+0000 front 2024-04-24T09:26:03.168290+0000 (oldest deadline 2024-04-24T09:26:27.268338+0000) 2024-04-24T09:26:36.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:36 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:36.482+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.369549+0000 front 2024-04-24T09:26:11.369459+0000 (oldest deadline 2024-04-24T09:26:33.669296+0000) 2024-04-24T09:26:37.352 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:36 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:36.899+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:42.619691+0000 front 2024-04-24T09:25:42.619948+0000 (oldest deadline 2024-04-24T09:26:07.920133+0000) 2024-04-24T09:26:37.352 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:36 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:36.899+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:55.523989+0000 front 2024-04-24T09:25:55.524168+0000 (oldest deadline 2024-04-24T09:26:20.824236+0000) 2024-04-24T09:26:37.353 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:36 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:36.899+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.526723+0000 front 2024-04-24T09:26:02.526595+0000 (oldest deadline 2024-04-24T09:26:28.426856+0000) 2024-04-24T09:26:37.353 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:36 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:36.899+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:08.427956+0000 front 2024-04-24T09:26:08.428100+0000 (oldest deadline 2024-04-24T09:26:34.328195+0000) 2024-04-24T09:26:37.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:37 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:37.167+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:56.271466+0000 front 2024-04-24T09:25:56.271493+0000 (oldest deadline 2024-04-24T09:26:17.371645+0000) 2024-04-24T09:26:37.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:37 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:37.167+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.573220+0000 front 2024-04-24T09:26:02.573169+0000 (oldest deadline 2024-04-24T09:26:26.073363+0000) 2024-04-24T09:26:37.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:37 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:37.167+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.475474+0000 front 2024-04-24T09:26:12.475548+0000 (oldest deadline 2024-04-24T09:26:35.375296+0000) 2024-04-24T09:26:37.611 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:37 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:37.225+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.547664+0000 front 2024-04-24T09:25:54.547684+0000 (oldest deadline 2024-04-24T09:26:16.847855+0000) 2024-04-24T09:26:37.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:37 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:37.225+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:04.350005+0000 front 2024-04-24T09:26:04.349959+0000 (oldest deadline 2024-04-24T09:26:28.450238+0000) 2024-04-24T09:26:37.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:37 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:37.196+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:37.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:37 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:37.196+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:26:37.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:37 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:37.196+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:26:37.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:37 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:37.196+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.811765+0000 front 2024-04-24T09:26:12.811682+0000 (oldest deadline 2024-04-24T09:26:35.711456+0000) 2024-04-24T09:26:37.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:37 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:37.497+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:26:37.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:37 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:37.497+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.160252+0000 front 2024-04-24T09:25:54.159838+0000 (oldest deadline 2024-04-24T09:26:18.259963+0000) 2024-04-24T09:26:37.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:37 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:37.497+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.764867+0000 front 2024-04-24T09:26:02.764831+0000 (oldest deadline 2024-04-24T09:26:26.264981+0000) 2024-04-24T09:26:37.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:37 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:37.497+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.567402+0000 front 2024-04-24T09:26:11.567591+0000 (oldest deadline 2024-04-24T09:26:36.267266+0000) 2024-04-24T09:26:37.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:37 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:37.456+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:37.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:37 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:37.456+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:26:37.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:37 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:37.456+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:03.168384+0000 front 2024-04-24T09:26:03.168290+0000 (oldest deadline 2024-04-24T09:26:27.268338+0000) 2024-04-24T09:26:37.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:37 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:37.456+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.369549+0000 front 2024-04-24T09:26:11.369459+0000 (oldest deadline 2024-04-24T09:26:33.669296+0000) 2024-04-24T09:26:38.157 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:38 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:38.155+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:56.271466+0000 front 2024-04-24T09:25:56.271493+0000 (oldest deadline 2024-04-24T09:26:17.371645+0000) 2024-04-24T09:26:38.157 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:38 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:38.155+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.573220+0000 front 2024-04-24T09:26:02.573169+0000 (oldest deadline 2024-04-24T09:26:26.073363+0000) 2024-04-24T09:26:38.352 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:37 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:37.914+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:42.619691+0000 front 2024-04-24T09:25:42.619948+0000 (oldest deadline 2024-04-24T09:26:07.920133+0000) 2024-04-24T09:26:38.352 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:37 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:37.914+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:55.523989+0000 front 2024-04-24T09:25:55.524168+0000 (oldest deadline 2024-04-24T09:26:20.824236+0000) 2024-04-24T09:26:38.352 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:37 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:37.914+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.526723+0000 front 2024-04-24T09:26:02.526595+0000 (oldest deadline 2024-04-24T09:26:28.426856+0000) 2024-04-24T09:26:38.353 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:37 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6[55572]: 2024-04-24T09:26:37.914+0000 7f3ec5800700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:08.427956+0000 front 2024-04-24T09:26:08.428100+0000 (oldest deadline 2024-04-24T09:26:34.328195+0000) 2024-04-24T09:26:38.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:38 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:38.155+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.475474+0000 front 2024-04-24T09:26:12.475548+0000 (oldest deadline 2024-04-24T09:26:35.375296+0000) 2024-04-24T09:26:38.611 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:38 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:38.218+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.547664+0000 front 2024-04-24T09:25:54.547684+0000 (oldest deadline 2024-04-24T09:26:16.847855+0000) 2024-04-24T09:26:38.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:38 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:38.218+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:04.350005+0000 front 2024-04-24T09:26:04.349959+0000 (oldest deadline 2024-04-24T09:26:28.450238+0000) 2024-04-24T09:26:38.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:38 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:38.218+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.351583+0000 front 2024-04-24T09:26:11.351431+0000 (oldest deadline 2024-04-24T09:26:37.251291+0000) 2024-04-24T09:26:38.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:38 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:38.163+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:38.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:38 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:38.163+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:26:38.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:38 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:38.163+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:26:38.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:38 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:38.163+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.811765+0000 front 2024-04-24T09:26:12.811682+0000 (oldest deadline 2024-04-24T09:26:35.711456+0000) 2024-04-24T09:26:38.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:38 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:38.505+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:26:38.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:38 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:38.505+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.160252+0000 front 2024-04-24T09:25:54.159838+0000 (oldest deadline 2024-04-24T09:26:18.259963+0000) 2024-04-24T09:26:38.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:38 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:38.505+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.764867+0000 front 2024-04-24T09:26:02.764831+0000 (oldest deadline 2024-04-24T09:26:26.264981+0000) 2024-04-24T09:26:38.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:38 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:38.505+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.567402+0000 front 2024-04-24T09:26:11.567591+0000 (oldest deadline 2024-04-24T09:26:36.267266+0000) 2024-04-24T09:26:38.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:38 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:38.475+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:38.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:38 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:38.475+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:26:38.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:38 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:38.475+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:03.168384+0000 front 2024-04-24T09:26:03.168290+0000 (oldest deadline 2024-04-24T09:26:27.268338+0000) 2024-04-24T09:26:38.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:38 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:38.475+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.369549+0000 front 2024-04-24T09:26:11.369459+0000 (oldest deadline 2024-04-24T09:26:33.669296+0000) 2024-04-24T09:26:39.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:39 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:39.163+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:56.271466+0000 front 2024-04-24T09:25:56.271493+0000 (oldest deadline 2024-04-24T09:26:17.371645+0000) 2024-04-24T09:26:39.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:39 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:39.163+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.573220+0000 front 2024-04-24T09:26:02.573169+0000 (oldest deadline 2024-04-24T09:26:26.073363+0000) 2024-04-24T09:26:39.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:39 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:39.163+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.475474+0000 front 2024-04-24T09:26:12.475548+0000 (oldest deadline 2024-04-24T09:26:35.375296+0000) 2024-04-24T09:26:39.573 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:39 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:39.470+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:26:39.573 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:39 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:39.470+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.160252+0000 front 2024-04-24T09:25:54.159838+0000 (oldest deadline 2024-04-24T09:26:18.259963+0000) 2024-04-24T09:26:39.574 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:39 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:39.470+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.764867+0000 front 2024-04-24T09:26:02.764831+0000 (oldest deadline 2024-04-24T09:26:26.264981+0000) 2024-04-24T09:26:39.574 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:39 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:39.470+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.567402+0000 front 2024-04-24T09:26:11.567591+0000 (oldest deadline 2024-04-24T09:26:36.267266+0000) 2024-04-24T09:26:39.574 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:39 smithi090 podman[89525]: 2024-04-24 09:26:39.234537123 +0000 UTC m=+5.813519450 container died b5538ae378bb05309c8af4a009ec57a3491e0911136a0ed4414a935949753ada (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6, GIT_CLEAN=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-24T09:26:39.611 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:39 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:39.183+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.547664+0000 front 2024-04-24T09:25:54.547684+0000 (oldest deadline 2024-04-24T09:26:16.847855+0000) 2024-04-24T09:26:39.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:39 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:39.183+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:04.350005+0000 front 2024-04-24T09:26:04.349959+0000 (oldest deadline 2024-04-24T09:26:28.450238+0000) 2024-04-24T09:26:39.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:39 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:39.183+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.351583+0000 front 2024-04-24T09:26:11.351431+0000 (oldest deadline 2024-04-24T09:26:37.251291+0000) 2024-04-24T09:26:39.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:39 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:39.190+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:39.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:39 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:39.190+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:26:39.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:39 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:39.190+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:26:39.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:39 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:39.190+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.811765+0000 front 2024-04-24T09:26:12.811682+0000 (oldest deadline 2024-04-24T09:26:35.711456+0000) 2024-04-24T09:26:39.852 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:39 smithi090 podman[89525]: 2024-04-24 09:26:39.791952108 +0000 UTC m=+6.370934432 container cleanup b5538ae378bb05309c8af4a009ec57a3491e0911136a0ed4414a935949753ada (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True) 2024-04-24T09:26:39.852 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:39 smithi090 bash[89525]: ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6 2024-04-24T09:26:39.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:39 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:39.503+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:39.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:39 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:39.503+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:26:39.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:39 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:39.503+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:03.168384+0000 front 2024-04-24T09:26:03.168290+0000 (oldest deadline 2024-04-24T09:26:27.268338+0000) 2024-04-24T09:26:39.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:39 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:39.503+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.369549+0000 front 2024-04-24T09:26:11.369459+0000 (oldest deadline 2024-04-24T09:26:33.669296+0000) 2024-04-24T09:26:40.352 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:40 smithi090 podman[89544]: 2024-04-24 09:26:40.103290438 +0000 UTC m=+0.848370524 container remove b5538ae378bb05309c8af4a009ec57a3491e0911136a0ed4414a935949753ada (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-24T09:26:40.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:40 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:40.158+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:56.271466+0000 front 2024-04-24T09:25:56.271493+0000 (oldest deadline 2024-04-24T09:26:17.371645+0000) 2024-04-24T09:26:40.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:40 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:40.158+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.573220+0000 front 2024-04-24T09:26:02.573169+0000 (oldest deadline 2024-04-24T09:26:26.073363+0000) 2024-04-24T09:26:40.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:40 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:40.158+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.475474+0000 front 2024-04-24T09:26:12.475548+0000 (oldest deadline 2024-04-24T09:26:35.375296+0000) 2024-04-24T09:26:40.611 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:40 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:40.152+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.547664+0000 front 2024-04-24T09:25:54.547684+0000 (oldest deadline 2024-04-24T09:26:16.847855+0000) 2024-04-24T09:26:40.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:40 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:40.152+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:04.350005+0000 front 2024-04-24T09:26:04.349959+0000 (oldest deadline 2024-04-24T09:26:28.450238+0000) 2024-04-24T09:26:40.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:40 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:40.152+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.351583+0000 front 2024-04-24T09:26:11.351431+0000 (oldest deadline 2024-04-24T09:26:37.251291+0000) 2024-04-24T09:26:40.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:40 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:40.232+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:40.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:40 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:40.232+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:26:40.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:40 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:40.232+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:26:40.613 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:40 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:40.232+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.811765+0000 front 2024-04-24T09:26:12.811682+0000 (oldest deadline 2024-04-24T09:26:35.711456+0000) 2024-04-24T09:26:40.837 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:40 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:40.446+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:26:40.837 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:40 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:40.446+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.160252+0000 front 2024-04-24T09:25:54.159838+0000 (oldest deadline 2024-04-24T09:26:18.259963+0000) 2024-04-24T09:26:40.837 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:40 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:40.446+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.764867+0000 front 2024-04-24T09:26:02.764831+0000 (oldest deadline 2024-04-24T09:26:26.264981+0000) 2024-04-24T09:26:40.837 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:40 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:40.446+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.567402+0000 front 2024-04-24T09:26:11.567591+0000 (oldest deadline 2024-04-24T09:26:36.267266+0000) 2024-04-24T09:26:40.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:40 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:40.457+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:40.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:40 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:40.457+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:26:40.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:40 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:40.457+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:03.168384+0000 front 2024-04-24T09:26:03.168290+0000 (oldest deadline 2024-04-24T09:26:27.268338+0000) 2024-04-24T09:26:40.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:40 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:40.457+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.369549+0000 front 2024-04-24T09:26:11.369459+0000 (oldest deadline 2024-04-24T09:26:33.669296+0000) 2024-04-24T09:26:41.102 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:40 smithi090 podman[89641]: 2024-04-24 09:26:40.741300805 +0000 UTC m=+0.030932007 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:26:41.102 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:40 smithi090 podman[89641]: 2024-04-24 09:26:40.898559373 +0000 UTC m=+0.188190605 container create b2be4251840f02b1df8ac731d58303a51b5256c10f54021fccd3bc66a1127ffc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6-deactivate, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T09:26:41.462 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:41 smithi090 podman[89641]: 2024-04-24 09:26:41.192180697 +0000 UTC m=+0.481811944 container init b2be4251840f02b1df8ac731d58303a51b5256c10f54021fccd3bc66a1127ffc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-24T09:26:41.463 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:41 smithi090 podman[89641]: 2024-04-24 09:26:41.203631212 +0000 UTC m=+0.493262458 container start b2be4251840f02b1df8ac731d58303a51b5256c10f54021fccd3bc66a1127ffc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T09:26:41.463 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:41 smithi090 podman[89641]: 2024-04-24 09:26:41.290921439 +0000 UTC m=+0.580552686 container attach b2be4251840f02b1df8ac731d58303a51b5256c10f54021fccd3bc66a1127ffc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-24T09:26:41.611 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:41 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:41.143+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.547664+0000 front 2024-04-24T09:25:54.547684+0000 (oldest deadline 2024-04-24T09:26:16.847855+0000) 2024-04-24T09:26:41.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:41 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:41.143+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:04.350005+0000 front 2024-04-24T09:26:04.349959+0000 (oldest deadline 2024-04-24T09:26:28.450238+0000) 2024-04-24T09:26:41.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:41 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:41.143+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.351583+0000 front 2024-04-24T09:26:11.351431+0000 (oldest deadline 2024-04-24T09:26:37.251291+0000) 2024-04-24T09:26:41.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:41 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:41.222+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:41.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:41 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:41.222+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:26:41.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:41 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:41.222+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:26:41.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:41 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:41.222+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.811765+0000 front 2024-04-24T09:26:12.811682+0000 (oldest deadline 2024-04-24T09:26:35.711456+0000) 2024-04-24T09:26:41.676 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:41 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:41.207+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:56.271466+0000 front 2024-04-24T09:25:56.271493+0000 (oldest deadline 2024-04-24T09:26:17.371645+0000) 2024-04-24T09:26:41.676 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:41 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:41.207+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.573220+0000 front 2024-04-24T09:26:02.573169+0000 (oldest deadline 2024-04-24T09:26:26.073363+0000) 2024-04-24T09:26:41.676 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:41 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:41.207+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.475474+0000 front 2024-04-24T09:26:12.475548+0000 (oldest deadline 2024-04-24T09:26:35.375296+0000) 2024-04-24T09:26:41.676 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:41 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:41.410+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:41.677 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:41 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:41.410+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:26:41.677 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:41 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:41.410+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:03.168384+0000 front 2024-04-24T09:26:03.168290+0000 (oldest deadline 2024-04-24T09:26:27.268338+0000) 2024-04-24T09:26:41.677 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:41 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:41.410+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.369549+0000 front 2024-04-24T09:26:11.369459+0000 (oldest deadline 2024-04-24T09:26:33.669296+0000) 2024-04-24T09:26:41.853 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 24 09:26:41 smithi090 podman[89641]: 2024-04-24 09:26:41.647616436 +0000 UTC m=+0.937247654 container died b2be4251840f02b1df8ac731d58303a51b5256c10f54021fccd3bc66a1127ffc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-24T09:26:41.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:41 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:41.462+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:26:41.854 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:41 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:41.462+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.160252+0000 front 2024-04-24T09:25:54.159838+0000 (oldest deadline 2024-04-24T09:26:18.259963+0000) 2024-04-24T09:26:41.854 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:41 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:41.462+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.764867+0000 front 2024-04-24T09:26:02.764831+0000 (oldest deadline 2024-04-24T09:26:26.264981+0000) 2024-04-24T09:26:41.854 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:41 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:41.462+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.567402+0000 front 2024-04-24T09:26:11.567591+0000 (oldest deadline 2024-04-24T09:26:36.267266+0000) 2024-04-24T09:26:42.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:42 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:42.157+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:56.271466+0000 front 2024-04-24T09:25:56.271493+0000 (oldest deadline 2024-04-24T09:26:17.371645+0000) 2024-04-24T09:26:42.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:42 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:42.157+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.573220+0000 front 2024-04-24T09:26:02.573169+0000 (oldest deadline 2024-04-24T09:26:26.073363+0000) 2024-04-24T09:26:42.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:42 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:42.157+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.475474+0000 front 2024-04-24T09:26:12.475548+0000 (oldest deadline 2024-04-24T09:26:35.375296+0000) 2024-04-24T09:26:42.493 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@osd.6.service' 2024-04-24T09:26:42.596 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:26:42.597 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-24T09:26:42.597 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-24T09:26:42.597 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-30ee9be2-021a-11ef-bc93-c7b262605968@osd.7 2024-04-24T09:26:42.611 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:42 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:42.141+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.547664+0000 front 2024-04-24T09:25:54.547684+0000 (oldest deadline 2024-04-24T09:26:16.847855+0000) 2024-04-24T09:26:42.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:42 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:42.141+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:04.350005+0000 front 2024-04-24T09:26:04.349959+0000 (oldest deadline 2024-04-24T09:26:28.450238+0000) 2024-04-24T09:26:42.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:42 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:42.141+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.351583+0000 front 2024-04-24T09:26:11.351431+0000 (oldest deadline 2024-04-24T09:26:37.251291+0000) 2024-04-24T09:26:42.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:42 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:42.251+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:42.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:42 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:42.251+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:26:42.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:42 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:42.251+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:26:42.613 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:42 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:42.251+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.811765+0000 front 2024-04-24T09:26:12.811682+0000 (oldest deadline 2024-04-24T09:26:35.711456+0000) 2024-04-24T09:26:42.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:42 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:42.442+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:26:42.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:42 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:42.442+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.160252+0000 front 2024-04-24T09:25:54.159838+0000 (oldest deadline 2024-04-24T09:26:18.259963+0000) 2024-04-24T09:26:42.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:42 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:42.442+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.764867+0000 front 2024-04-24T09:26:02.764831+0000 (oldest deadline 2024-04-24T09:26:26.264981+0000) 2024-04-24T09:26:42.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:42 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:42.442+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.567402+0000 front 2024-04-24T09:26:11.567591+0000 (oldest deadline 2024-04-24T09:26:36.267266+0000) 2024-04-24T09:26:42.897 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:42 smithi114 systemd[1]: Stopping Ceph osd.7 for 30ee9be2-021a-11ef-bc93-c7b262605968... 2024-04-24T09:26:42.897 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:42 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:42.449+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:42.897 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:42 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:42.449+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:26:42.897 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:42 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:42.449+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:03.168384+0000 front 2024-04-24T09:26:03.168290+0000 (oldest deadline 2024-04-24T09:26:27.268338+0000) 2024-04-24T09:26:42.898 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:42 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:42.449+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.369549+0000 front 2024-04-24T09:26:11.369459+0000 (oldest deadline 2024-04-24T09:26:33.669296+0000) 2024-04-24T09:26:43.176 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:42 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:42.896+0000 7f47f1418700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:26:43.176 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:42 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:42.896+0000 7f47f1418700 -1 osd.7 91 *** Got signal Terminated *** 2024-04-24T09:26:43.176 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:42 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:42.896+0000 7f47f1418700 -1 osd.7 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T09:26:43.177 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:43 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:43.112+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:56.271466+0000 front 2024-04-24T09:25:56.271493+0000 (oldest deadline 2024-04-24T09:26:17.371645+0000) 2024-04-24T09:26:43.177 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:43 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:43.112+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.573220+0000 front 2024-04-24T09:26:02.573169+0000 (oldest deadline 2024-04-24T09:26:26.073363+0000) 2024-04-24T09:26:43.177 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:43 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:43.112+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.475474+0000 front 2024-04-24T09:26:12.475548+0000 (oldest deadline 2024-04-24T09:26:35.375296+0000) 2024-04-24T09:26:43.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:43 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:43.170+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.547664+0000 front 2024-04-24T09:25:54.547684+0000 (oldest deadline 2024-04-24T09:26:16.847855+0000) 2024-04-24T09:26:43.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:43 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:43.170+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:04.350005+0000 front 2024-04-24T09:26:04.349959+0000 (oldest deadline 2024-04-24T09:26:28.450238+0000) 2024-04-24T09:26:43.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:43 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:43.170+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.351583+0000 front 2024-04-24T09:26:11.351431+0000 (oldest deadline 2024-04-24T09:26:37.251291+0000) 2024-04-24T09:26:43.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:43 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:43.259+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:43.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:43 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:43.259+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:26:43.613 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:43 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:43.259+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:26:43.613 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:43 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:43.259+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.811765+0000 front 2024-04-24T09:26:12.811682+0000 (oldest deadline 2024-04-24T09:26:35.711456+0000) 2024-04-24T09:26:43.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:43 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:43.470+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:26:43.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:43 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:43.470+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.160252+0000 front 2024-04-24T09:25:54.159838+0000 (oldest deadline 2024-04-24T09:26:18.259963+0000) 2024-04-24T09:26:43.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:43 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:43.470+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.764867+0000 front 2024-04-24T09:26:02.764831+0000 (oldest deadline 2024-04-24T09:26:26.264981+0000) 2024-04-24T09:26:43.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:43 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:43.470+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.567402+0000 front 2024-04-24T09:26:11.567591+0000 (oldest deadline 2024-04-24T09:26:36.267266+0000) 2024-04-24T09:26:43.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:43 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:43.459+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:43.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:43 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:43.459+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:26:43.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:43 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:43.459+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:03.168384+0000 front 2024-04-24T09:26:03.168290+0000 (oldest deadline 2024-04-24T09:26:27.268338+0000) 2024-04-24T09:26:43.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:43 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:43.459+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.369549+0000 front 2024-04-24T09:26:11.369459+0000 (oldest deadline 2024-04-24T09:26:33.669296+0000) 2024-04-24T09:26:43.927 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:43 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:43.459+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:21.270674+0000 front 2024-04-24T09:26:21.270719+0000 (oldest deadline 2024-04-24T09:26:42.970795+0000) 2024-04-24T09:26:44.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:44 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:44.121+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:56.271466+0000 front 2024-04-24T09:25:56.271493+0000 (oldest deadline 2024-04-24T09:26:17.371645+0000) 2024-04-24T09:26:44.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:44 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:44.121+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.573220+0000 front 2024-04-24T09:26:02.573169+0000 (oldest deadline 2024-04-24T09:26:26.073363+0000) 2024-04-24T09:26:44.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:44 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:44.121+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.475474+0000 front 2024-04-24T09:26:12.475548+0000 (oldest deadline 2024-04-24T09:26:35.375296+0000) 2024-04-24T09:26:44.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:44 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:44.121+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:20.676217+0000 front 2024-04-24T09:26:20.676182+0000 (oldest deadline 2024-04-24T09:26:43.576316+0000) 2024-04-24T09:26:44.611 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:44 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:44.194+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.547664+0000 front 2024-04-24T09:25:54.547684+0000 (oldest deadline 2024-04-24T09:26:16.847855+0000) 2024-04-24T09:26:44.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:44 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:44.194+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:04.350005+0000 front 2024-04-24T09:26:04.349959+0000 (oldest deadline 2024-04-24T09:26:28.450238+0000) 2024-04-24T09:26:44.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:44 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:44.194+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.351583+0000 front 2024-04-24T09:26:11.351431+0000 (oldest deadline 2024-04-24T09:26:37.251291+0000) 2024-04-24T09:26:44.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:44 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:44.264+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:44.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:44 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:44.264+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:26:44.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:44 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:44.264+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:26:44.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:44 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:44.264+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.811765+0000 front 2024-04-24T09:26:12.811682+0000 (oldest deadline 2024-04-24T09:26:35.711456+0000) 2024-04-24T09:26:44.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:44 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:44.442+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:26:44.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:44 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:44.442+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.160252+0000 front 2024-04-24T09:25:54.159838+0000 (oldest deadline 2024-04-24T09:26:18.259963+0000) 2024-04-24T09:26:44.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:44 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:44.442+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.764867+0000 front 2024-04-24T09:26:02.764831+0000 (oldest deadline 2024-04-24T09:26:26.264981+0000) 2024-04-24T09:26:44.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:44 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:44.442+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.567402+0000 front 2024-04-24T09:26:11.567591+0000 (oldest deadline 2024-04-24T09:26:36.267266+0000) 2024-04-24T09:26:44.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:44 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:44.442+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:18.470531+0000 front 2024-04-24T09:26:18.470405+0000 (oldest deadline 2024-04-24T09:26:43.770549+0000) 2024-04-24T09:26:44.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:44 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:44.482+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:44.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:44 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:44.482+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:26:44.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:44 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:44.482+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:03.168384+0000 front 2024-04-24T09:26:03.168290+0000 (oldest deadline 2024-04-24T09:26:27.268338+0000) 2024-04-24T09:26:44.927 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:44 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:44.482+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.369549+0000 front 2024-04-24T09:26:11.369459+0000 (oldest deadline 2024-04-24T09:26:33.669296+0000) 2024-04-24T09:26:44.927 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:44 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:44.482+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:21.270674+0000 front 2024-04-24T09:26:21.270719+0000 (oldest deadline 2024-04-24T09:26:42.970795+0000) 2024-04-24T09:26:45.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:45 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:45.135+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:56.271466+0000 front 2024-04-24T09:25:56.271493+0000 (oldest deadline 2024-04-24T09:26:17.371645+0000) 2024-04-24T09:26:45.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:45 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:45.135+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.573220+0000 front 2024-04-24T09:26:02.573169+0000 (oldest deadline 2024-04-24T09:26:26.073363+0000) 2024-04-24T09:26:45.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:45 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:45.135+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.475474+0000 front 2024-04-24T09:26:12.475548+0000 (oldest deadline 2024-04-24T09:26:35.375296+0000) 2024-04-24T09:26:45.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:45 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:45.135+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:20.676217+0000 front 2024-04-24T09:26:20.676182+0000 (oldest deadline 2024-04-24T09:26:43.576316+0000) 2024-04-24T09:26:45.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:45 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:45.182+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.547664+0000 front 2024-04-24T09:25:54.547684+0000 (oldest deadline 2024-04-24T09:26:16.847855+0000) 2024-04-24T09:26:45.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:45 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:45.182+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:04.350005+0000 front 2024-04-24T09:26:04.349959+0000 (oldest deadline 2024-04-24T09:26:28.450238+0000) 2024-04-24T09:26:45.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:45 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:45.182+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.351583+0000 front 2024-04-24T09:26:11.351431+0000 (oldest deadline 2024-04-24T09:26:37.251291+0000) 2024-04-24T09:26:45.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:45 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:45.233+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:45.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:45 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:45.233+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:26:45.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:45 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:45.233+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:26:45.613 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:45 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:45.233+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.811765+0000 front 2024-04-24T09:26:12.811682+0000 (oldest deadline 2024-04-24T09:26:35.711456+0000) 2024-04-24T09:26:45.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:45 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:45.462+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:26:45.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:45 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:45.462+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.160252+0000 front 2024-04-24T09:25:54.159838+0000 (oldest deadline 2024-04-24T09:26:18.259963+0000) 2024-04-24T09:26:45.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:45 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:45.462+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.764867+0000 front 2024-04-24T09:26:02.764831+0000 (oldest deadline 2024-04-24T09:26:26.264981+0000) 2024-04-24T09:26:45.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:45 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:45.462+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.567402+0000 front 2024-04-24T09:26:11.567591+0000 (oldest deadline 2024-04-24T09:26:36.267266+0000) 2024-04-24T09:26:45.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:45 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:45.462+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:18.470531+0000 front 2024-04-24T09:26:18.470405+0000 (oldest deadline 2024-04-24T09:26:43.770549+0000) 2024-04-24T09:26:45.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:45 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:45.521+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:45.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:45 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:45.521+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:26:45.927 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:45 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:45.521+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:03.168384+0000 front 2024-04-24T09:26:03.168290+0000 (oldest deadline 2024-04-24T09:26:27.268338+0000) 2024-04-24T09:26:45.927 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:45 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:45.521+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.369549+0000 front 2024-04-24T09:26:11.369459+0000 (oldest deadline 2024-04-24T09:26:33.669296+0000) 2024-04-24T09:26:45.927 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:45 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:45.521+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:21.270674+0000 front 2024-04-24T09:26:21.270719+0000 (oldest deadline 2024-04-24T09:26:42.970795+0000) 2024-04-24T09:26:46.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:46 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:46.136+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:56.271466+0000 front 2024-04-24T09:25:56.271493+0000 (oldest deadline 2024-04-24T09:26:17.371645+0000) 2024-04-24T09:26:46.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:46 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:46.136+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.573220+0000 front 2024-04-24T09:26:02.573169+0000 (oldest deadline 2024-04-24T09:26:26.073363+0000) 2024-04-24T09:26:46.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:46 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:46.136+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.475474+0000 front 2024-04-24T09:26:12.475548+0000 (oldest deadline 2024-04-24T09:26:35.375296+0000) 2024-04-24T09:26:46.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:46 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:46.136+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:20.676217+0000 front 2024-04-24T09:26:20.676182+0000 (oldest deadline 2024-04-24T09:26:43.576316+0000) 2024-04-24T09:26:46.611 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:46 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:46.196+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.547664+0000 front 2024-04-24T09:25:54.547684+0000 (oldest deadline 2024-04-24T09:26:16.847855+0000) 2024-04-24T09:26:46.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:46 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:46.196+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:04.350005+0000 front 2024-04-24T09:26:04.349959+0000 (oldest deadline 2024-04-24T09:26:28.450238+0000) 2024-04-24T09:26:46.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:46 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:46.196+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.351583+0000 front 2024-04-24T09:26:11.351431+0000 (oldest deadline 2024-04-24T09:26:37.251291+0000) 2024-04-24T09:26:46.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:46 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:46.253+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:46.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:46 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:46.253+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:26:46.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:46 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:46.253+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:26:46.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:46 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:46.253+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.811765+0000 front 2024-04-24T09:26:12.811682+0000 (oldest deadline 2024-04-24T09:26:35.711456+0000) 2024-04-24T09:26:46.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:46 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:46.424+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:26:46.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:46 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:46.424+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.160252+0000 front 2024-04-24T09:25:54.159838+0000 (oldest deadline 2024-04-24T09:26:18.259963+0000) 2024-04-24T09:26:46.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:46 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:46.424+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.764867+0000 front 2024-04-24T09:26:02.764831+0000 (oldest deadline 2024-04-24T09:26:26.264981+0000) 2024-04-24T09:26:46.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:46 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:46.424+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.567402+0000 front 2024-04-24T09:26:11.567591+0000 (oldest deadline 2024-04-24T09:26:36.267266+0000) 2024-04-24T09:26:46.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:46 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:46.424+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:18.470531+0000 front 2024-04-24T09:26:18.470405+0000 (oldest deadline 2024-04-24T09:26:43.770549+0000) 2024-04-24T09:26:46.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:46 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:46.563+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:46.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:46 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:46.563+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:26:46.927 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:46 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:46.563+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:03.168384+0000 front 2024-04-24T09:26:03.168290+0000 (oldest deadline 2024-04-24T09:26:27.268338+0000) 2024-04-24T09:26:46.927 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:46 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:46.563+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.369549+0000 front 2024-04-24T09:26:11.369459+0000 (oldest deadline 2024-04-24T09:26:33.669296+0000) 2024-04-24T09:26:46.927 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:46 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:46.563+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:21.270674+0000 front 2024-04-24T09:26:21.270719+0000 (oldest deadline 2024-04-24T09:26:42.970795+0000) 2024-04-24T09:26:47.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:47 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:47.145+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:56.271466+0000 front 2024-04-24T09:25:56.271493+0000 (oldest deadline 2024-04-24T09:26:17.371645+0000) 2024-04-24T09:26:47.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:47 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:47.145+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.573220+0000 front 2024-04-24T09:26:02.573169+0000 (oldest deadline 2024-04-24T09:26:26.073363+0000) 2024-04-24T09:26:47.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:47 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:47.145+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.475474+0000 front 2024-04-24T09:26:12.475548+0000 (oldest deadline 2024-04-24T09:26:35.375296+0000) 2024-04-24T09:26:47.426 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:47 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7[51018]: 2024-04-24T09:26:47.145+0000 7f47e9a04700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:20.676217+0000 front 2024-04-24T09:26:20.676182+0000 (oldest deadline 2024-04-24T09:26:43.576316+0000) 2024-04-24T09:26:47.513 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:47 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:47.233+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.547664+0000 front 2024-04-24T09:25:54.547684+0000 (oldest deadline 2024-04-24T09:26:16.847855+0000) 2024-04-24T09:26:47.513 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:47 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:47.233+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:04.350005+0000 front 2024-04-24T09:26:04.349959+0000 (oldest deadline 2024-04-24T09:26:28.450238+0000) 2024-04-24T09:26:47.513 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:47 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:47.233+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.351583+0000 front 2024-04-24T09:26:11.351431+0000 (oldest deadline 2024-04-24T09:26:37.251291+0000) 2024-04-24T09:26:47.513 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:47 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:47.296+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:47.514 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:47 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:47.296+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:26:47.514 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:47 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:47.296+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:26:47.514 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:47 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:47.296+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.811765+0000 front 2024-04-24T09:26:12.811682+0000 (oldest deadline 2024-04-24T09:26:35.711456+0000) 2024-04-24T09:26:47.514 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:47 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:47.296+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:22.113177+0000 front 2024-04-24T09:26:22.112999+0000 (oldest deadline 2024-04-24T09:26:46.812888+0000) 2024-04-24T09:26:47.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:47 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:47.456+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:26:47.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:47 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:47.456+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.160252+0000 front 2024-04-24T09:25:54.159838+0000 (oldest deadline 2024-04-24T09:26:18.259963+0000) 2024-04-24T09:26:47.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:47 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:47.456+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.764867+0000 front 2024-04-24T09:26:02.764831+0000 (oldest deadline 2024-04-24T09:26:26.264981+0000) 2024-04-24T09:26:47.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:47 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:47.456+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.567402+0000 front 2024-04-24T09:26:11.567591+0000 (oldest deadline 2024-04-24T09:26:36.267266+0000) 2024-04-24T09:26:47.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:47 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:47.456+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:18.470531+0000 front 2024-04-24T09:26:18.470405+0000 (oldest deadline 2024-04-24T09:26:43.770549+0000) 2024-04-24T09:26:47.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:47 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:47.605+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:47.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:47 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:47.605+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:26:47.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:47 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:47.605+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:03.168384+0000 front 2024-04-24T09:26:03.168290+0000 (oldest deadline 2024-04-24T09:26:27.268338+0000) 2024-04-24T09:26:47.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:47 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:47.605+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.369549+0000 front 2024-04-24T09:26:11.369459+0000 (oldest deadline 2024-04-24T09:26:33.669296+0000) 2024-04-24T09:26:47.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:47 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:47.605+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:21.270674+0000 front 2024-04-24T09:26:21.270719+0000 (oldest deadline 2024-04-24T09:26:42.970795+0000) 2024-04-24T09:26:48.525 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:48 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:48.328+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:48.525 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:48 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:48.328+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:26:48.525 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:48 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:48.328+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:26:48.525 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:48 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:48.328+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.811765+0000 front 2024-04-24T09:26:12.811682+0000 (oldest deadline 2024-04-24T09:26:35.711456+0000) 2024-04-24T09:26:48.525 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:48 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:48.328+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:22.113177+0000 front 2024-04-24T09:26:22.112999+0000 (oldest deadline 2024-04-24T09:26:46.812888+0000) 2024-04-24T09:26:48.526 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:48 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:48.242+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.547664+0000 front 2024-04-24T09:25:54.547684+0000 (oldest deadline 2024-04-24T09:26:16.847855+0000) 2024-04-24T09:26:48.526 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:48 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:48.242+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:04.350005+0000 front 2024-04-24T09:26:04.349959+0000 (oldest deadline 2024-04-24T09:26:28.450238+0000) 2024-04-24T09:26:48.526 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:48 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:48.242+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.351583+0000 front 2024-04-24T09:26:11.351431+0000 (oldest deadline 2024-04-24T09:26:37.251291+0000) 2024-04-24T09:26:48.526 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:48 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:48.243+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:21.952602+0000 front 2024-04-24T09:26:21.952669+0000 (oldest deadline 2024-04-24T09:26:47.252587+0000) 2024-04-24T09:26:48.531 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:48 smithi114 podman[67496]: 2024-04-24 09:26:48.222700974 +0000 UTC m=+5.441444587 container died c8adb77cb1557e9a4e888fdb0761f11d475243dbdf3b668200630e723ebf3747 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-24T09:26:48.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:48 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:48.422+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:26:48.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:48 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:48.422+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.160252+0000 front 2024-04-24T09:25:54.159838+0000 (oldest deadline 2024-04-24T09:26:18.259963+0000) 2024-04-24T09:26:48.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:48 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:48.422+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.764867+0000 front 2024-04-24T09:26:02.764831+0000 (oldest deadline 2024-04-24T09:26:26.264981+0000) 2024-04-24T09:26:48.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:48 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:48.422+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.567402+0000 front 2024-04-24T09:26:11.567591+0000 (oldest deadline 2024-04-24T09:26:36.267266+0000) 2024-04-24T09:26:48.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:48 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:48.422+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:18.470531+0000 front 2024-04-24T09:26:18.470405+0000 (oldest deadline 2024-04-24T09:26:43.770549+0000) 2024-04-24T09:26:48.881 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:48 smithi114 podman[67496]: 2024-04-24 09:26:48.622664029 +0000 UTC m=+5.841407641 container cleanup c8adb77cb1557e9a4e888fdb0761f11d475243dbdf3b668200630e723ebf3747 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T09:26:48.907 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:48 smithi114 bash[67496]: ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7 2024-04-24T09:26:48.908 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:48 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:48.651+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:48.908 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:48 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:48.651+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:26:48.908 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:48 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:48.651+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:03.168384+0000 front 2024-04-24T09:26:03.168290+0000 (oldest deadline 2024-04-24T09:26:27.268338+0000) 2024-04-24T09:26:48.908 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:48 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:48.651+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.369549+0000 front 2024-04-24T09:26:11.369459+0000 (oldest deadline 2024-04-24T09:26:33.669296+0000) 2024-04-24T09:26:48.908 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:48 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:48.651+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:21.270674+0000 front 2024-04-24T09:26:21.270719+0000 (oldest deadline 2024-04-24T09:26:42.970795+0000) 2024-04-24T09:26:49.176 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:48 smithi114 podman[67512]: 2024-04-24 09:26:48.882143694 +0000 UTC m=+0.660313115 container remove c8adb77cb1557e9a4e888fdb0761f11d475243dbdf3b668200630e723ebf3747 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, io.buildah.version=1.33.5) 2024-04-24T09:26:49.496 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:49 smithi114 podman[67602]: 2024-04-24 09:26:49.084341721 +0000 UTC m=+0.017826888 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:26:49.497 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:49 smithi114 podman[67602]: 2024-04-24 09:26:49.208213422 +0000 UTC m=+0.141698574 container create fe0b75a3def145916249cf8533ba1debe6277f72bdc38fde1ca08cd6ecac8fbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-24T09:26:49.611 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:49 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:49.243+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.547664+0000 front 2024-04-24T09:25:54.547684+0000 (oldest deadline 2024-04-24T09:26:16.847855+0000) 2024-04-24T09:26:49.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:49 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:49.243+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:04.350005+0000 front 2024-04-24T09:26:04.349959+0000 (oldest deadline 2024-04-24T09:26:28.450238+0000) 2024-04-24T09:26:49.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:49 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:49.243+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.351583+0000 front 2024-04-24T09:26:11.351431+0000 (oldest deadline 2024-04-24T09:26:37.251291+0000) 2024-04-24T09:26:49.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:49 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:49.243+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:21.952602+0000 front 2024-04-24T09:26:21.952669+0000 (oldest deadline 2024-04-24T09:26:47.252587+0000) 2024-04-24T09:26:49.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:49 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:49.283+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:49.613 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:49 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:49.283+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:26:49.613 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:49 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:49.283+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:26:49.613 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:49 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:49.283+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.811765+0000 front 2024-04-24T09:26:12.811682+0000 (oldest deadline 2024-04-24T09:26:35.711456+0000) 2024-04-24T09:26:49.613 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:49 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:49.283+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:22.113177+0000 front 2024-04-24T09:26:22.112999+0000 (oldest deadline 2024-04-24T09:26:46.812888+0000) 2024-04-24T09:26:49.785 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:49 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:49.690+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:49.785 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:49 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:49.690+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:26:49.785 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:49 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:49.690+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:03.168384+0000 front 2024-04-24T09:26:03.168290+0000 (oldest deadline 2024-04-24T09:26:27.268338+0000) 2024-04-24T09:26:49.785 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:49 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:49.690+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.369549+0000 front 2024-04-24T09:26:11.369459+0000 (oldest deadline 2024-04-24T09:26:33.669296+0000) 2024-04-24T09:26:49.785 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:49 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:49.690+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:21.270674+0000 front 2024-04-24T09:26:21.270719+0000 (oldest deadline 2024-04-24T09:26:42.970795+0000) 2024-04-24T09:26:49.786 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:49 smithi114 podman[67602]: 2024-04-24 09:26:49.497248153 +0000 UTC m=+0.430733312 container init fe0b75a3def145916249cf8533ba1debe6277f72bdc38fde1ca08cd6ecac8fbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-24T09:26:49.786 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:49 smithi114 podman[67602]: 2024-04-24 09:26:49.500733284 +0000 UTC m=+0.434218435 container start fe0b75a3def145916249cf8533ba1debe6277f72bdc38fde1ca08cd6ecac8fbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240424) 2024-04-24T09:26:49.786 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:49 smithi114 podman[67602]: 2024-04-24 09:26:49.577393067 +0000 UTC m=+0.510878226 container attach fe0b75a3def145916249cf8533ba1debe6277f72bdc38fde1ca08cd6ecac8fbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T09:26:49.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:49 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:49.438+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:26:49.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:49 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:49.438+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.160252+0000 front 2024-04-24T09:25:54.159838+0000 (oldest deadline 2024-04-24T09:26:18.259963+0000) 2024-04-24T09:26:49.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:49 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:49.438+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.764867+0000 front 2024-04-24T09:26:02.764831+0000 (oldest deadline 2024-04-24T09:26:26.264981+0000) 2024-04-24T09:26:49.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:49 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:49.438+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.567402+0000 front 2024-04-24T09:26:11.567591+0000 (oldest deadline 2024-04-24T09:26:36.267266+0000) 2024-04-24T09:26:49.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:49 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:49.438+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:18.470531+0000 front 2024-04-24T09:26:18.470405+0000 (oldest deadline 2024-04-24T09:26:43.770549+0000) 2024-04-24T09:26:50.079 INFO:journalctl@ceph.osd.7.smithi114.stdout:Apr 24 09:26:49 smithi114 podman[67602]: 2024-04-24 09:26:49.786139724 +0000 UTC m=+0.719624873 container died fe0b75a3def145916249cf8533ba1debe6277f72bdc38fde1ca08cd6ecac8fbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-24T09:26:50.361 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@osd.7.service' 2024-04-24T09:26:50.398 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:26:50.398 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-24T09:26:50.399 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-24T09:26:50.399 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-30ee9be2-021a-11ef-bc93-c7b262605968@osd.8 2024-04-24T09:26:50.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:50 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:50.215+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.547664+0000 front 2024-04-24T09:25:54.547684+0000 (oldest deadline 2024-04-24T09:26:16.847855+0000) 2024-04-24T09:26:50.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:50 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:50.215+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:04.350005+0000 front 2024-04-24T09:26:04.349959+0000 (oldest deadline 2024-04-24T09:26:28.450238+0000) 2024-04-24T09:26:50.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:50 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:50.215+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.351583+0000 front 2024-04-24T09:26:11.351431+0000 (oldest deadline 2024-04-24T09:26:37.251291+0000) 2024-04-24T09:26:50.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:50 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:50.215+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:21.952602+0000 front 2024-04-24T09:26:21.952669+0000 (oldest deadline 2024-04-24T09:26:47.252587+0000) 2024-04-24T09:26:50.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:50 smithi149 systemd[1]: Stopping Ceph osd.8 for 30ee9be2-021a-11ef-bc93-c7b262605968... 2024-04-24T09:26:50.613 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:50 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:50.300+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:50.613 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:50 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:50.300+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:26:50.613 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:50 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:50.300+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:26:50.613 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:50 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:50.300+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.811765+0000 front 2024-04-24T09:26:12.811682+0000 (oldest deadline 2024-04-24T09:26:35.711456+0000) 2024-04-24T09:26:50.613 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:50 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:50.300+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:22.113177+0000 front 2024-04-24T09:26:22.112999+0000 (oldest deadline 2024-04-24T09:26:46.812888+0000) 2024-04-24T09:26:50.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:50 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:50.390+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:26:50.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:50 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:50.390+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.160252+0000 front 2024-04-24T09:25:54.159838+0000 (oldest deadline 2024-04-24T09:26:18.259963+0000) 2024-04-24T09:26:50.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:50 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:50.390+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.764867+0000 front 2024-04-24T09:26:02.764831+0000 (oldest deadline 2024-04-24T09:26:26.264981+0000) 2024-04-24T09:26:50.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:50 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:50.390+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.567402+0000 front 2024-04-24T09:26:11.567591+0000 (oldest deadline 2024-04-24T09:26:36.267266+0000) 2024-04-24T09:26:50.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:50 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:50.390+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:18.470531+0000 front 2024-04-24T09:26:18.470405+0000 (oldest deadline 2024-04-24T09:26:43.770549+0000) 2024-04-24T09:26:50.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:50 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:50.663+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:50.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:50 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:50.663+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:26:50.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:50 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:50.663+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:03.168384+0000 front 2024-04-24T09:26:03.168290+0000 (oldest deadline 2024-04-24T09:26:27.268338+0000) 2024-04-24T09:26:50.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:50 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:50.663+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.369549+0000 front 2024-04-24T09:26:11.369459+0000 (oldest deadline 2024-04-24T09:26:33.669296+0000) 2024-04-24T09:26:50.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:50 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:50.663+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:21.270674+0000 front 2024-04-24T09:26:21.270719+0000 (oldest deadline 2024-04-24T09:26:42.970795+0000) 2024-04-24T09:26:51.111 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:50 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:50.738+0000 7fce02885700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:26:51.112 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:50 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:50.738+0000 7fce02885700 -1 osd.8 91 *** Got signal Terminated *** 2024-04-24T09:26:51.112 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:50 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:50.738+0000 7fce02885700 -1 osd.8 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T09:26:51.611 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:51 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:51.264+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.547664+0000 front 2024-04-24T09:25:54.547684+0000 (oldest deadline 2024-04-24T09:26:16.847855+0000) 2024-04-24T09:26:51.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:51 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:51.264+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:04.350005+0000 front 2024-04-24T09:26:04.349959+0000 (oldest deadline 2024-04-24T09:26:28.450238+0000) 2024-04-24T09:26:51.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:51 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:51.264+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.351583+0000 front 2024-04-24T09:26:11.351431+0000 (oldest deadline 2024-04-24T09:26:37.251291+0000) 2024-04-24T09:26:51.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:51 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:51.264+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:21.952602+0000 front 2024-04-24T09:26:21.952669+0000 (oldest deadline 2024-04-24T09:26:47.252587+0000) 2024-04-24T09:26:51.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:51 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:51.320+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:51.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:51 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:51.320+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:26:51.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:51 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:51.320+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:26:51.613 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:51 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:51.320+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.811765+0000 front 2024-04-24T09:26:12.811682+0000 (oldest deadline 2024-04-24T09:26:35.711456+0000) 2024-04-24T09:26:51.613 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:51 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:51.320+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:22.113177+0000 front 2024-04-24T09:26:22.112999+0000 (oldest deadline 2024-04-24T09:26:46.812888+0000) 2024-04-24T09:26:51.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:51 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:51.411+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:26:51.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:51 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:51.411+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.160252+0000 front 2024-04-24T09:25:54.159838+0000 (oldest deadline 2024-04-24T09:26:18.259963+0000) 2024-04-24T09:26:51.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:51 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:51.411+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.764867+0000 front 2024-04-24T09:26:02.764831+0000 (oldest deadline 2024-04-24T09:26:26.264981+0000) 2024-04-24T09:26:51.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:51 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:51.411+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.567402+0000 front 2024-04-24T09:26:11.567591+0000 (oldest deadline 2024-04-24T09:26:36.267266+0000) 2024-04-24T09:26:51.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:51 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:51.411+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:18.470531+0000 front 2024-04-24T09:26:18.470405+0000 (oldest deadline 2024-04-24T09:26:43.770549+0000) 2024-04-24T09:26:51.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:51 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:51.627+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:51.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:51 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:51.627+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:26:51.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:51 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:51.627+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:03.168384+0000 front 2024-04-24T09:26:03.168290+0000 (oldest deadline 2024-04-24T09:26:27.268338+0000) 2024-04-24T09:26:51.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:51 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:51.627+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.369549+0000 front 2024-04-24T09:26:11.369459+0000 (oldest deadline 2024-04-24T09:26:33.669296+0000) 2024-04-24T09:26:51.927 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:51 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:51.627+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:21.270674+0000 front 2024-04-24T09:26:21.270719+0000 (oldest deadline 2024-04-24T09:26:42.970795+0000) 2024-04-24T09:26:52.611 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:52 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:52.222+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.547664+0000 front 2024-04-24T09:25:54.547684+0000 (oldest deadline 2024-04-24T09:26:16.847855+0000) 2024-04-24T09:26:52.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:52 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:52.222+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:04.350005+0000 front 2024-04-24T09:26:04.349959+0000 (oldest deadline 2024-04-24T09:26:28.450238+0000) 2024-04-24T09:26:52.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:52 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:52.222+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.351583+0000 front 2024-04-24T09:26:11.351431+0000 (oldest deadline 2024-04-24T09:26:37.251291+0000) 2024-04-24T09:26:52.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:52 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:52.222+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:21.952602+0000 front 2024-04-24T09:26:21.952669+0000 (oldest deadline 2024-04-24T09:26:47.252587+0000) 2024-04-24T09:26:52.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:52 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:52.222+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:28.953512+0000 front 2024-04-24T09:26:28.953562+0000 (oldest deadline 2024-04-24T09:26:51.853613+0000) 2024-04-24T09:26:52.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:52 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:52.298+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:52.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:52 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:52.298+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:26:52.613 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:52 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:52.298+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:26:52.613 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:52 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:52.298+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.811765+0000 front 2024-04-24T09:26:12.811682+0000 (oldest deadline 2024-04-24T09:26:35.711456+0000) 2024-04-24T09:26:52.613 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:52 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:52.298+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:22.113177+0000 front 2024-04-24T09:26:22.112999+0000 (oldest deadline 2024-04-24T09:26:46.812888+0000) 2024-04-24T09:26:52.613 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:52 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:52.298+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:29.113749+0000 front 2024-04-24T09:26:29.113777+0000 (oldest deadline 2024-04-24T09:26:52.013778+0000) 2024-04-24T09:26:52.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:52 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:52.442+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:26:52.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:52 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:52.442+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.160252+0000 front 2024-04-24T09:25:54.159838+0000 (oldest deadline 2024-04-24T09:26:18.259963+0000) 2024-04-24T09:26:52.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:52 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:52.442+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.764867+0000 front 2024-04-24T09:26:02.764831+0000 (oldest deadline 2024-04-24T09:26:26.264981+0000) 2024-04-24T09:26:52.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:52 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:52.442+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.567402+0000 front 2024-04-24T09:26:11.567591+0000 (oldest deadline 2024-04-24T09:26:36.267266+0000) 2024-04-24T09:26:52.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:52 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:52.442+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:18.470531+0000 front 2024-04-24T09:26:18.470405+0000 (oldest deadline 2024-04-24T09:26:43.770549+0000) 2024-04-24T09:26:52.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:52 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:52.585+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:52.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:52 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:52.585+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:26:52.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:52 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:52.585+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:03.168384+0000 front 2024-04-24T09:26:03.168290+0000 (oldest deadline 2024-04-24T09:26:27.268338+0000) 2024-04-24T09:26:52.927 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:52 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:52.585+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.369549+0000 front 2024-04-24T09:26:11.369459+0000 (oldest deadline 2024-04-24T09:26:33.669296+0000) 2024-04-24T09:26:52.927 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:52 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:52.585+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:21.270674+0000 front 2024-04-24T09:26:21.270719+0000 (oldest deadline 2024-04-24T09:26:42.970795+0000) 2024-04-24T09:26:52.927 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:52 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:52.585+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:27.572314+0000 front 2024-04-24T09:26:27.572352+0000 (oldest deadline 2024-04-24T09:26:52.272337+0000) 2024-04-24T09:26:53.611 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:53 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:53.198+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.547664+0000 front 2024-04-24T09:25:54.547684+0000 (oldest deadline 2024-04-24T09:26:16.847855+0000) 2024-04-24T09:26:53.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:53 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:53.198+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:04.350005+0000 front 2024-04-24T09:26:04.349959+0000 (oldest deadline 2024-04-24T09:26:28.450238+0000) 2024-04-24T09:26:53.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:53 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:53.198+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.351583+0000 front 2024-04-24T09:26:11.351431+0000 (oldest deadline 2024-04-24T09:26:37.251291+0000) 2024-04-24T09:26:53.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:53 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:53.198+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:21.952602+0000 front 2024-04-24T09:26:21.952669+0000 (oldest deadline 2024-04-24T09:26:47.252587+0000) 2024-04-24T09:26:53.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:53 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:53.198+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:28.953512+0000 front 2024-04-24T09:26:28.953562+0000 (oldest deadline 2024-04-24T09:26:51.853613+0000) 2024-04-24T09:26:53.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:53 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:53.306+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:53.613 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:53 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:53.306+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:26:53.613 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:53 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:53.306+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:26:53.613 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:53 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:53.306+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.811765+0000 front 2024-04-24T09:26:12.811682+0000 (oldest deadline 2024-04-24T09:26:35.711456+0000) 2024-04-24T09:26:53.613 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:53 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:53.306+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:22.113177+0000 front 2024-04-24T09:26:22.112999+0000 (oldest deadline 2024-04-24T09:26:46.812888+0000) 2024-04-24T09:26:53.613 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:53 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:53.306+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:29.113749+0000 front 2024-04-24T09:26:29.113777+0000 (oldest deadline 2024-04-24T09:26:52.013778+0000) 2024-04-24T09:26:53.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:53 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:53.420+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:26:53.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:53 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:53.420+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.160252+0000 front 2024-04-24T09:25:54.159838+0000 (oldest deadline 2024-04-24T09:26:18.259963+0000) 2024-04-24T09:26:53.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:53 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:53.420+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.764867+0000 front 2024-04-24T09:26:02.764831+0000 (oldest deadline 2024-04-24T09:26:26.264981+0000) 2024-04-24T09:26:53.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:53 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:53.420+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.567402+0000 front 2024-04-24T09:26:11.567591+0000 (oldest deadline 2024-04-24T09:26:36.267266+0000) 2024-04-24T09:26:53.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:53 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:53.420+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:18.470531+0000 front 2024-04-24T09:26:18.470405+0000 (oldest deadline 2024-04-24T09:26:43.770549+0000) 2024-04-24T09:26:53.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:53 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:53.600+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:53.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:53 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:53.600+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:26:53.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:53 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:53.600+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:03.168384+0000 front 2024-04-24T09:26:03.168290+0000 (oldest deadline 2024-04-24T09:26:27.268338+0000) 2024-04-24T09:26:53.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:53 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:53.600+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.369549+0000 front 2024-04-24T09:26:11.369459+0000 (oldest deadline 2024-04-24T09:26:33.669296+0000) 2024-04-24T09:26:53.927 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:53 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:53.600+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:21.270674+0000 front 2024-04-24T09:26:21.270719+0000 (oldest deadline 2024-04-24T09:26:42.970795+0000) 2024-04-24T09:26:53.927 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:53 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:53.600+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:27.572314+0000 front 2024-04-24T09:26:27.572352+0000 (oldest deadline 2024-04-24T09:26:52.272337+0000) 2024-04-24T09:26:54.611 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:54 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:54.185+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.547664+0000 front 2024-04-24T09:25:54.547684+0000 (oldest deadline 2024-04-24T09:26:16.847855+0000) 2024-04-24T09:26:54.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:54 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:54.185+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:04.350005+0000 front 2024-04-24T09:26:04.349959+0000 (oldest deadline 2024-04-24T09:26:28.450238+0000) 2024-04-24T09:26:54.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:54 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:54.185+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.351583+0000 front 2024-04-24T09:26:11.351431+0000 (oldest deadline 2024-04-24T09:26:37.251291+0000) 2024-04-24T09:26:54.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:54 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:54.185+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:21.952602+0000 front 2024-04-24T09:26:21.952669+0000 (oldest deadline 2024-04-24T09:26:47.252587+0000) 2024-04-24T09:26:54.612 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:54 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:54.185+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:28.953512+0000 front 2024-04-24T09:26:28.953562+0000 (oldest deadline 2024-04-24T09:26:51.853613+0000) 2024-04-24T09:26:54.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:54 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:54.319+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:54.613 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:54 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:54.319+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:26:54.613 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:54 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:54.319+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:26:54.613 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:54 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:54.319+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.811765+0000 front 2024-04-24T09:26:12.811682+0000 (oldest deadline 2024-04-24T09:26:35.711456+0000) 2024-04-24T09:26:54.613 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:54 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:54.319+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:22.113177+0000 front 2024-04-24T09:26:22.112999+0000 (oldest deadline 2024-04-24T09:26:46.812888+0000) 2024-04-24T09:26:54.613 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:54 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:54.319+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:29.113749+0000 front 2024-04-24T09:26:29.113777+0000 (oldest deadline 2024-04-24T09:26:52.013778+0000) 2024-04-24T09:26:54.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:54 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:54.416+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:26:54.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:54 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:54.416+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.160252+0000 front 2024-04-24T09:25:54.159838+0000 (oldest deadline 2024-04-24T09:26:18.259963+0000) 2024-04-24T09:26:54.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:54 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:54.416+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.764867+0000 front 2024-04-24T09:26:02.764831+0000 (oldest deadline 2024-04-24T09:26:26.264981+0000) 2024-04-24T09:26:54.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:54 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:54.416+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.567402+0000 front 2024-04-24T09:26:11.567591+0000 (oldest deadline 2024-04-24T09:26:36.267266+0000) 2024-04-24T09:26:54.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:54 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:54.416+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:18.470531+0000 front 2024-04-24T09:26:18.470405+0000 (oldest deadline 2024-04-24T09:26:43.770549+0000) 2024-04-24T09:26:54.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:54 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:54.416+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:29.375264+0000 front 2024-04-24T09:26:29.375382+0000 (oldest deadline 2024-04-24T09:26:53.475369+0000) 2024-04-24T09:26:54.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:54 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:54.592+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:54.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:54 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:54.592+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:26:54.927 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:54 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:54.592+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:03.168384+0000 front 2024-04-24T09:26:03.168290+0000 (oldest deadline 2024-04-24T09:26:27.268338+0000) 2024-04-24T09:26:54.927 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:54 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:54.592+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.369549+0000 front 2024-04-24T09:26:11.369459+0000 (oldest deadline 2024-04-24T09:26:33.669296+0000) 2024-04-24T09:26:54.927 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:54 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:54.592+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:21.270674+0000 front 2024-04-24T09:26:21.270719+0000 (oldest deadline 2024-04-24T09:26:42.970795+0000) 2024-04-24T09:26:54.927 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:54 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:54.592+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:27.572314+0000 front 2024-04-24T09:26:27.572352+0000 (oldest deadline 2024-04-24T09:26:52.272337+0000) 2024-04-24T09:26:55.613 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:55 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:55.157+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.547664+0000 front 2024-04-24T09:25:54.547684+0000 (oldest deadline 2024-04-24T09:26:16.847855+0000) 2024-04-24T09:26:55.614 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:55 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:55.157+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:04.350005+0000 front 2024-04-24T09:26:04.349959+0000 (oldest deadline 2024-04-24T09:26:28.450238+0000) 2024-04-24T09:26:55.614 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:55 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:55.157+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.351583+0000 front 2024-04-24T09:26:11.351431+0000 (oldest deadline 2024-04-24T09:26:37.251291+0000) 2024-04-24T09:26:55.614 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:55 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:55.157+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:21.952602+0000 front 2024-04-24T09:26:21.952669+0000 (oldest deadline 2024-04-24T09:26:47.252587+0000) 2024-04-24T09:26:55.614 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:55 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8[51703]: 2024-04-24T09:26:55.157+0000 7fcdfae71700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:28.953512+0000 front 2024-04-24T09:26:28.953562+0000 (oldest deadline 2024-04-24T09:26:51.853613+0000) 2024-04-24T09:26:55.614 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:55 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:55.289+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:55.615 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:55 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:55.289+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:26:55.615 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:55 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:55.289+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:26:55.615 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:55 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:55.289+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.811765+0000 front 2024-04-24T09:26:12.811682+0000 (oldest deadline 2024-04-24T09:26:35.711456+0000) 2024-04-24T09:26:55.615 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:55 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:55.289+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:22.113177+0000 front 2024-04-24T09:26:22.112999+0000 (oldest deadline 2024-04-24T09:26:46.812888+0000) 2024-04-24T09:26:55.615 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:55 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:55.289+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:29.113749+0000 front 2024-04-24T09:26:29.113777+0000 (oldest deadline 2024-04-24T09:26:52.013778+0000) 2024-04-24T09:26:55.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:55 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:55.450+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:26:55.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:55 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:55.450+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.160252+0000 front 2024-04-24T09:25:54.159838+0000 (oldest deadline 2024-04-24T09:26:18.259963+0000) 2024-04-24T09:26:55.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:55 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:55.450+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.764867+0000 front 2024-04-24T09:26:02.764831+0000 (oldest deadline 2024-04-24T09:26:26.264981+0000) 2024-04-24T09:26:55.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:55 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:55.450+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.567402+0000 front 2024-04-24T09:26:11.567591+0000 (oldest deadline 2024-04-24T09:26:36.267266+0000) 2024-04-24T09:26:55.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:55 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:55.450+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:18.470531+0000 front 2024-04-24T09:26:18.470405+0000 (oldest deadline 2024-04-24T09:26:43.770549+0000) 2024-04-24T09:26:55.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:55 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:55.450+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:29.375264+0000 front 2024-04-24T09:26:29.375382+0000 (oldest deadline 2024-04-24T09:26:53.475369+0000) 2024-04-24T09:26:55.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:55 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:55.544+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:55.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:55 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:55.544+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:26:55.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:55 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:55.544+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:03.168384+0000 front 2024-04-24T09:26:03.168290+0000 (oldest deadline 2024-04-24T09:26:27.268338+0000) 2024-04-24T09:26:55.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:55 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:55.544+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.369549+0000 front 2024-04-24T09:26:11.369459+0000 (oldest deadline 2024-04-24T09:26:33.669296+0000) 2024-04-24T09:26:55.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:55 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:55.544+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:21.270674+0000 front 2024-04-24T09:26:21.270719+0000 (oldest deadline 2024-04-24T09:26:42.970795+0000) 2024-04-24T09:26:55.927 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:55 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:55.544+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:27.572314+0000 front 2024-04-24T09:26:27.572352+0000 (oldest deadline 2024-04-24T09:26:52.272337+0000) 2024-04-24T09:26:56.555 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:56 smithi149 podman[68285]: 2024-04-24 09:26:56.242944169 +0000 UTC m=+5.646668943 container died fe0c490793e91ba7f7e7b7ce5931e0d08696ea6179aececba39225d316507db5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0) 2024-04-24T09:26:56.555 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:56 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:56.272+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:56.555 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:56 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:56.272+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:26:56.556 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:56 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:56.272+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:26:56.556 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:56 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:56.272+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.811765+0000 front 2024-04-24T09:26:12.811682+0000 (oldest deadline 2024-04-24T09:26:35.711456+0000) 2024-04-24T09:26:56.556 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:56 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:56.272+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:22.113177+0000 front 2024-04-24T09:26:22.112999+0000 (oldest deadline 2024-04-24T09:26:46.812888+0000) 2024-04-24T09:26:56.556 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:56 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:56.272+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:29.113749+0000 front 2024-04-24T09:26:29.113777+0000 (oldest deadline 2024-04-24T09:26:52.013778+0000) 2024-04-24T09:26:56.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:56 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:56.425+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:26:56.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:56 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:56.425+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.160252+0000 front 2024-04-24T09:25:54.159838+0000 (oldest deadline 2024-04-24T09:26:18.259963+0000) 2024-04-24T09:26:56.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:56 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:56.425+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.764867+0000 front 2024-04-24T09:26:02.764831+0000 (oldest deadline 2024-04-24T09:26:26.264981+0000) 2024-04-24T09:26:56.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:56 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:56.425+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.567402+0000 front 2024-04-24T09:26:11.567591+0000 (oldest deadline 2024-04-24T09:26:36.267266+0000) 2024-04-24T09:26:56.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:56 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:56.425+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:18.470531+0000 front 2024-04-24T09:26:18.470405+0000 (oldest deadline 2024-04-24T09:26:43.770549+0000) 2024-04-24T09:26:56.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:56 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:56.425+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:29.375264+0000 front 2024-04-24T09:26:29.375382+0000 (oldest deadline 2024-04-24T09:26:53.475369+0000) 2024-04-24T09:26:56.861 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:56 smithi149 podman[68285]: 2024-04-24 09:26:56.655399687 +0000 UTC m=+6.059124441 container cleanup fe0c490793e91ba7f7e7b7ce5931e0d08696ea6179aececba39225d316507db5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T09:26:56.861 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:56 smithi149 bash[68285]: ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8 2024-04-24T09:26:56.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:56 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:56.577+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:56.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:56 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:56.577+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:26:56.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:56 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:56.577+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:03.168384+0000 front 2024-04-24T09:26:03.168290+0000 (oldest deadline 2024-04-24T09:26:27.268338+0000) 2024-04-24T09:26:56.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:56 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:56.577+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.369549+0000 front 2024-04-24T09:26:11.369459+0000 (oldest deadline 2024-04-24T09:26:33.669296+0000) 2024-04-24T09:26:56.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:56 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:56.577+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:21.270674+0000 front 2024-04-24T09:26:21.270719+0000 (oldest deadline 2024-04-24T09:26:42.970795+0000) 2024-04-24T09:26:56.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:56 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:56.577+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:27.572314+0000 front 2024-04-24T09:26:27.572352+0000 (oldest deadline 2024-04-24T09:26:52.272337+0000) 2024-04-24T09:26:57.254 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:56 smithi149 podman[68305]: 2024-04-24 09:26:56.906818487 +0000 UTC m=+0.656502179 container remove fe0c490793e91ba7f7e7b7ce5931e0d08696ea6179aececba39225d316507db5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True) 2024-04-24T09:26:57.514 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:57 smithi149 podman[68394]: 2024-04-24 09:26:57.230506542 +0000 UTC m=+0.018047883 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:26:57.514 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:57 smithi149 podman[68394]: 2024-04-24 09:26:57.349365948 +0000 UTC m=+0.136907268 container create d67aad8af513ce6c2dcce69606c056df5f3df17c5154f28dd5bb4ec943c4a1ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-24T09:26:57.514 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:57 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:57.254+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:57.514 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:57 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:57.254+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:26:57.514 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:57 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:57.254+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:26:57.515 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:57 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:57.254+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.811765+0000 front 2024-04-24T09:26:12.811682+0000 (oldest deadline 2024-04-24T09:26:35.711456+0000) 2024-04-24T09:26:57.515 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:57 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:57.254+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:22.113177+0000 front 2024-04-24T09:26:22.112999+0000 (oldest deadline 2024-04-24T09:26:46.812888+0000) 2024-04-24T09:26:57.515 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:57 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:57.254+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:29.113749+0000 front 2024-04-24T09:26:29.113777+0000 (oldest deadline 2024-04-24T09:26:52.013778+0000) 2024-04-24T09:26:57.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:57 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:57.455+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:26:57.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:57 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:57.455+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.160252+0000 front 2024-04-24T09:25:54.159838+0000 (oldest deadline 2024-04-24T09:26:18.259963+0000) 2024-04-24T09:26:57.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:57 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:57.455+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.764867+0000 front 2024-04-24T09:26:02.764831+0000 (oldest deadline 2024-04-24T09:26:26.264981+0000) 2024-04-24T09:26:57.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:57 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:57.455+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.567402+0000 front 2024-04-24T09:26:11.567591+0000 (oldest deadline 2024-04-24T09:26:36.267266+0000) 2024-04-24T09:26:57.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:57 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:57.455+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:18.470531+0000 front 2024-04-24T09:26:18.470405+0000 (oldest deadline 2024-04-24T09:26:43.770549+0000) 2024-04-24T09:26:57.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:57 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:57.455+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:29.375264+0000 front 2024-04-24T09:26:29.375382+0000 (oldest deadline 2024-04-24T09:26:53.475369+0000) 2024-04-24T09:26:57.862 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:57 smithi149 podman[68394]: 2024-04-24 09:26:57.644028578 +0000 UTC m=+0.431569895 container init d67aad8af513ce6c2dcce69606c056df5f3df17c5154f28dd5bb4ec943c4a1ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-24T09:26:57.862 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:57 smithi149 podman[68394]: 2024-04-24 09:26:57.647373078 +0000 UTC m=+0.434914398 container start d67aad8af513ce6c2dcce69606c056df5f3df17c5154f28dd5bb4ec943c4a1ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T09:26:57.862 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:57 smithi149 podman[68394]: 2024-04-24 09:26:57.744002662 +0000 UTC m=+0.531543993 container attach d67aad8af513ce6c2dcce69606c056df5f3df17c5154f28dd5bb4ec943c4a1ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-24T09:26:57.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:57 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:57.581+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:57.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:57 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:57.582+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:26:57.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:57 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:57.582+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:03.168384+0000 front 2024-04-24T09:26:03.168290+0000 (oldest deadline 2024-04-24T09:26:27.268338+0000) 2024-04-24T09:26:57.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:57 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:57.582+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.369549+0000 front 2024-04-24T09:26:11.369459+0000 (oldest deadline 2024-04-24T09:26:33.669296+0000) 2024-04-24T09:26:57.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:57 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:57.582+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:21.270674+0000 front 2024-04-24T09:26:21.270719+0000 (oldest deadline 2024-04-24T09:26:42.970795+0000) 2024-04-24T09:26:57.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:57 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:57.582+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:27.572314+0000 front 2024-04-24T09:26:27.572352+0000 (oldest deadline 2024-04-24T09:26:52.272337+0000) 2024-04-24T09:26:58.225 INFO:journalctl@ceph.osd.8.smithi149.stdout:Apr 24 09:26:57 smithi149 podman[68394]: 2024-04-24 09:26:57.939017627 +0000 UTC m=+0.726558956 container died d67aad8af513ce6c2dcce69606c056df5f3df17c5154f28dd5bb4ec943c4a1ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T09:26:58.571 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:58 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:58.225+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:58.572 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:58 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:58.225+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:26:58.572 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:58 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:58.225+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:26:58.572 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:58 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:58.225+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.811765+0000 front 2024-04-24T09:26:12.811682+0000 (oldest deadline 2024-04-24T09:26:35.711456+0000) 2024-04-24T09:26:58.572 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:58 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:58.225+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:22.113177+0000 front 2024-04-24T09:26:22.112999+0000 (oldest deadline 2024-04-24T09:26:46.812888+0000) 2024-04-24T09:26:58.572 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:58 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:58.225+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:29.113749+0000 front 2024-04-24T09:26:29.113777+0000 (oldest deadline 2024-04-24T09:26:52.013778+0000) 2024-04-24T09:26:58.592 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@osd.8.service' 2024-04-24T09:26:58.634 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:26:58.634 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-24T09:26:58.635 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-24T09:26:58.635 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-30ee9be2-021a-11ef-bc93-c7b262605968@osd.9 2024-04-24T09:26:58.752 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:58 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:58.501+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:26:58.752 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:58 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:58.501+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.160252+0000 front 2024-04-24T09:25:54.159838+0000 (oldest deadline 2024-04-24T09:26:18.259963+0000) 2024-04-24T09:26:58.752 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:58 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:58.501+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.764867+0000 front 2024-04-24T09:26:02.764831+0000 (oldest deadline 2024-04-24T09:26:26.264981+0000) 2024-04-24T09:26:58.753 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:58 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:58.501+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.567402+0000 front 2024-04-24T09:26:11.567591+0000 (oldest deadline 2024-04-24T09:26:36.267266+0000) 2024-04-24T09:26:58.753 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:58 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:58.501+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:18.470531+0000 front 2024-04-24T09:26:18.470405+0000 (oldest deadline 2024-04-24T09:26:43.770549+0000) 2024-04-24T09:26:58.753 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:58 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:58.501+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:29.375264+0000 front 2024-04-24T09:26:29.375382+0000 (oldest deadline 2024-04-24T09:26:53.475369+0000) 2024-04-24T09:26:58.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:58 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:58.542+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:58.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:58 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:58.542+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:26:58.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:58 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:58.542+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:03.168384+0000 front 2024-04-24T09:26:03.168290+0000 (oldest deadline 2024-04-24T09:26:27.268338+0000) 2024-04-24T09:26:58.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:58 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:58.542+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.369549+0000 front 2024-04-24T09:26:11.369459+0000 (oldest deadline 2024-04-24T09:26:33.669296+0000) 2024-04-24T09:26:58.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:58 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:58.542+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:21.270674+0000 front 2024-04-24T09:26:21.270719+0000 (oldest deadline 2024-04-24T09:26:42.970795+0000) 2024-04-24T09:26:58.927 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:58 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:58.542+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:27.572314+0000 front 2024-04-24T09:26:27.572352+0000 (oldest deadline 2024-04-24T09:26:52.272337+0000) 2024-04-24T09:26:59.102 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:58 smithi090 systemd[1]: Stopping Ceph osd.9 for 30ee9be2-021a-11ef-bc93-c7b262605968... 2024-04-24T09:26:59.480 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:59 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:59.181+0000 7f6adc800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:26:59.480 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:59 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:59.181+0000 7f6adc800700 -1 osd.9 91 *** Got signal Terminated *** 2024-04-24T09:26:59.481 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:59 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:59.181+0000 7f6adc800700 -1 osd.9 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T09:26:59.481 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:59 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:59.479+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:26:59.481 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:59 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:59.479+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.160252+0000 front 2024-04-24T09:25:54.159838+0000 (oldest deadline 2024-04-24T09:26:18.259963+0000) 2024-04-24T09:26:59.481 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:59 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:59.479+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.764867+0000 front 2024-04-24T09:26:02.764831+0000 (oldest deadline 2024-04-24T09:26:26.264981+0000) 2024-04-24T09:26:59.481 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:59 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:59.479+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.567402+0000 front 2024-04-24T09:26:11.567591+0000 (oldest deadline 2024-04-24T09:26:36.267266+0000) 2024-04-24T09:26:59.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:59 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:59.200+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:26:59.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:59 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:59.200+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:26:59.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:59 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:59.200+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:26:59.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:59 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:59.200+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.811765+0000 front 2024-04-24T09:26:12.811682+0000 (oldest deadline 2024-04-24T09:26:35.711456+0000) 2024-04-24T09:26:59.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:59 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:59.200+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:22.113177+0000 front 2024-04-24T09:26:22.112999+0000 (oldest deadline 2024-04-24T09:26:46.812888+0000) 2024-04-24T09:26:59.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:26:59 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:26:59.200+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:29.113749+0000 front 2024-04-24T09:26:29.113777+0000 (oldest deadline 2024-04-24T09:26:52.013778+0000) 2024-04-24T09:26:59.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:59 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:59.479+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:18.470531+0000 front 2024-04-24T09:26:18.470405+0000 (oldest deadline 2024-04-24T09:26:43.770549+0000) 2024-04-24T09:26:59.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:26:59 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:26:59.479+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:29.375264+0000 front 2024-04-24T09:26:29.375382+0000 (oldest deadline 2024-04-24T09:26:53.475369+0000) 2024-04-24T09:26:59.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:59 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:59.508+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:26:59.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:59 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:59.508+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:26:59.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:59 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:59.508+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:03.168384+0000 front 2024-04-24T09:26:03.168290+0000 (oldest deadline 2024-04-24T09:26:27.268338+0000) 2024-04-24T09:26:59.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:59 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:59.508+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.369549+0000 front 2024-04-24T09:26:11.369459+0000 (oldest deadline 2024-04-24T09:26:33.669296+0000) 2024-04-24T09:26:59.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:59 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:59.508+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:21.270674+0000 front 2024-04-24T09:26:21.270719+0000 (oldest deadline 2024-04-24T09:26:42.970795+0000) 2024-04-24T09:26:59.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:26:59 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:26:59.508+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:27.572314+0000 front 2024-04-24T09:26:27.572352+0000 (oldest deadline 2024-04-24T09:26:52.272337+0000) 2024-04-24T09:27:00.611 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:00 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:00.172+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:27:00.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:00 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:00.172+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:27:00.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:00 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:00.172+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:27:00.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:00 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:00.172+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.811765+0000 front 2024-04-24T09:26:12.811682+0000 (oldest deadline 2024-04-24T09:26:35.711456+0000) 2024-04-24T09:27:00.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:00 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:00.172+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:22.113177+0000 front 2024-04-24T09:26:22.112999+0000 (oldest deadline 2024-04-24T09:26:46.812888+0000) 2024-04-24T09:27:00.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:00 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:00.172+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:29.113749+0000 front 2024-04-24T09:26:29.113777+0000 (oldest deadline 2024-04-24T09:26:52.013778+0000) 2024-04-24T09:27:00.676 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:00 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:00.492+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:27:00.676 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:00 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:00.492+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:27:00.676 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:00 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:00.492+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:03.168384+0000 front 2024-04-24T09:26:03.168290+0000 (oldest deadline 2024-04-24T09:26:27.268338+0000) 2024-04-24T09:27:00.676 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:00 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:00.492+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.369549+0000 front 2024-04-24T09:26:11.369459+0000 (oldest deadline 2024-04-24T09:26:33.669296+0000) 2024-04-24T09:27:00.676 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:00 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:00.492+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:21.270674+0000 front 2024-04-24T09:26:21.270719+0000 (oldest deadline 2024-04-24T09:26:42.970795+0000) 2024-04-24T09:27:00.677 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:00 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:00.492+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:27.572314+0000 front 2024-04-24T09:26:27.572352+0000 (oldest deadline 2024-04-24T09:26:52.272337+0000) 2024-04-24T09:27:00.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:27:00 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:27:00.527+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:27:00.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:27:00 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:27:00.527+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.160252+0000 front 2024-04-24T09:25:54.159838+0000 (oldest deadline 2024-04-24T09:26:18.259963+0000) 2024-04-24T09:27:00.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:27:00 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:27:00.527+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.764867+0000 front 2024-04-24T09:26:02.764831+0000 (oldest deadline 2024-04-24T09:26:26.264981+0000) 2024-04-24T09:27:00.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:27:00 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:27:00.527+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.567402+0000 front 2024-04-24T09:26:11.567591+0000 (oldest deadline 2024-04-24T09:26:36.267266+0000) 2024-04-24T09:27:00.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:27:00 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:27:00.527+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:18.470531+0000 front 2024-04-24T09:26:18.470405+0000 (oldest deadline 2024-04-24T09:26:43.770549+0000) 2024-04-24T09:27:00.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:27:00 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:27:00.527+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:29.375264+0000 front 2024-04-24T09:26:29.375382+0000 (oldest deadline 2024-04-24T09:26:53.475369+0000) 2024-04-24T09:27:01.611 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:01 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:01.222+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:27:01.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:01 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:01.222+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:27:01.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:01 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:01.222+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:27:01.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:01 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:01.222+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.811765+0000 front 2024-04-24T09:26:12.811682+0000 (oldest deadline 2024-04-24T09:26:35.711456+0000) 2024-04-24T09:27:01.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:01 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:01.222+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:22.113177+0000 front 2024-04-24T09:26:22.112999+0000 (oldest deadline 2024-04-24T09:26:46.812888+0000) 2024-04-24T09:27:01.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:01 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:01.222+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:29.113749+0000 front 2024-04-24T09:26:29.113777+0000 (oldest deadline 2024-04-24T09:26:52.013778+0000) 2024-04-24T09:27:01.854 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:27:01 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:27:01.530+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:27:01.854 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:27:01 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:27:01.530+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.160252+0000 front 2024-04-24T09:25:54.159838+0000 (oldest deadline 2024-04-24T09:26:18.259963+0000) 2024-04-24T09:27:01.854 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:27:01 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:27:01.530+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.764867+0000 front 2024-04-24T09:26:02.764831+0000 (oldest deadline 2024-04-24T09:26:26.264981+0000) 2024-04-24T09:27:01.855 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:27:01 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:27:01.530+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.567402+0000 front 2024-04-24T09:26:11.567591+0000 (oldest deadline 2024-04-24T09:26:36.267266+0000) 2024-04-24T09:27:01.855 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:27:01 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:27:01.530+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:18.470531+0000 front 2024-04-24T09:26:18.470405+0000 (oldest deadline 2024-04-24T09:26:43.770549+0000) 2024-04-24T09:27:01.855 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:27:01 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:27:01.530+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:29.375264+0000 front 2024-04-24T09:26:29.375382+0000 (oldest deadline 2024-04-24T09:26:53.475369+0000) 2024-04-24T09:27:01.931 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:01 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:01.539+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:27:01.932 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:01 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:01.539+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:27:01.932 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:01 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:01.539+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:03.168384+0000 front 2024-04-24T09:26:03.168290+0000 (oldest deadline 2024-04-24T09:26:27.268338+0000) 2024-04-24T09:27:01.932 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:01 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:01.539+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.369549+0000 front 2024-04-24T09:26:11.369459+0000 (oldest deadline 2024-04-24T09:26:33.669296+0000) 2024-04-24T09:27:01.932 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:01 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:01.539+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:21.270674+0000 front 2024-04-24T09:26:21.270719+0000 (oldest deadline 2024-04-24T09:26:42.970795+0000) 2024-04-24T09:27:01.932 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:01 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:01.539+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:27.572314+0000 front 2024-04-24T09:26:27.572352+0000 (oldest deadline 2024-04-24T09:26:52.272337+0000) 2024-04-24T09:27:02.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:02 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:02.227+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:27:02.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:02 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:02.227+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:27:02.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:02 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:02.227+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:27:02.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:02 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:02.227+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.811765+0000 front 2024-04-24T09:26:12.811682+0000 (oldest deadline 2024-04-24T09:26:35.711456+0000) 2024-04-24T09:27:02.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:02 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:02.227+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:22.113177+0000 front 2024-04-24T09:26:22.112999+0000 (oldest deadline 2024-04-24T09:26:46.812888+0000) 2024-04-24T09:27:02.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:02 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:02.227+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:29.113749+0000 front 2024-04-24T09:26:29.113777+0000 (oldest deadline 2024-04-24T09:26:52.013778+0000) 2024-04-24T09:27:02.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:27:02 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:27:02.510+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:27:02.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:27:02 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:27:02.510+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.160252+0000 front 2024-04-24T09:25:54.159838+0000 (oldest deadline 2024-04-24T09:26:18.259963+0000) 2024-04-24T09:27:02.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:27:02 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:27:02.510+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.764867+0000 front 2024-04-24T09:26:02.764831+0000 (oldest deadline 2024-04-24T09:26:26.264981+0000) 2024-04-24T09:27:02.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:27:02 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:27:02.510+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.567402+0000 front 2024-04-24T09:26:11.567591+0000 (oldest deadline 2024-04-24T09:26:36.267266+0000) 2024-04-24T09:27:02.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:27:02 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:27:02.510+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:18.470531+0000 front 2024-04-24T09:26:18.470405+0000 (oldest deadline 2024-04-24T09:26:43.770549+0000) 2024-04-24T09:27:02.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:27:02 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:27:02.510+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:29.375264+0000 front 2024-04-24T09:26:29.375382+0000 (oldest deadline 2024-04-24T09:26:53.475369+0000) 2024-04-24T09:27:02.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:02 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:02.528+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:27:02.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:02 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:02.528+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:27:02.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:02 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:02.528+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:03.168384+0000 front 2024-04-24T09:26:03.168290+0000 (oldest deadline 2024-04-24T09:26:27.268338+0000) 2024-04-24T09:27:02.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:02 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:02.528+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.369549+0000 front 2024-04-24T09:26:11.369459+0000 (oldest deadline 2024-04-24T09:26:33.669296+0000) 2024-04-24T09:27:02.927 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:02 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:02.528+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:21.270674+0000 front 2024-04-24T09:26:21.270719+0000 (oldest deadline 2024-04-24T09:26:42.970795+0000) 2024-04-24T09:27:02.927 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:02 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:02.528+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:27.572314+0000 front 2024-04-24T09:26:27.572352+0000 (oldest deadline 2024-04-24T09:26:52.272337+0000) 2024-04-24T09:27:02.927 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:02 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:02.528+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-04-24T09:26:36.274001+0000 front 2024-04-24T09:26:36.273943+0000 (oldest deadline 2024-04-24T09:27:02.173673+0000) 2024-04-24T09:27:03.611 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:03 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:03.205+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:27:03.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:03 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:03.205+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:27:03.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:03 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:03.205+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:27:03.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:03 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:03.205+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.811765+0000 front 2024-04-24T09:26:12.811682+0000 (oldest deadline 2024-04-24T09:26:35.711456+0000) 2024-04-24T09:27:03.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:03 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:03.205+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:22.113177+0000 front 2024-04-24T09:26:22.112999+0000 (oldest deadline 2024-04-24T09:26:46.812888+0000) 2024-04-24T09:27:03.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:03 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:03.205+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:29.113749+0000 front 2024-04-24T09:26:29.113777+0000 (oldest deadline 2024-04-24T09:26:52.013778+0000) 2024-04-24T09:27:03.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:27:03 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:27:03.481+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.456251+0000 front 2024-04-24T09:25:45.456536+0000 (oldest deadline 2024-04-24T09:26:10.657472+0000) 2024-04-24T09:27:03.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:27:03 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:27:03.481+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.160252+0000 front 2024-04-24T09:25:54.159838+0000 (oldest deadline 2024-04-24T09:26:18.259963+0000) 2024-04-24T09:27:03.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:27:03 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:27:03.481+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.764867+0000 front 2024-04-24T09:26:02.764831+0000 (oldest deadline 2024-04-24T09:26:26.264981+0000) 2024-04-24T09:27:03.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:27:03 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:27:03.481+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.567402+0000 front 2024-04-24T09:26:11.567591+0000 (oldest deadline 2024-04-24T09:26:36.267266+0000) 2024-04-24T09:27:03.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:27:03 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:27:03.481+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:18.470531+0000 front 2024-04-24T09:26:18.470405+0000 (oldest deadline 2024-04-24T09:26:43.770549+0000) 2024-04-24T09:27:03.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:27:03 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9[62458]: 2024-04-24T09:27:03.481+0000 7f6ad3000700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:29.375264+0000 front 2024-04-24T09:26:29.375382+0000 (oldest deadline 2024-04-24T09:26:53.475369+0000) 2024-04-24T09:27:03.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:03 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:03.533+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:27:03.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:03 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:03.533+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:27:03.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:03 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:03.533+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:03.168384+0000 front 2024-04-24T09:26:03.168290+0000 (oldest deadline 2024-04-24T09:26:27.268338+0000) 2024-04-24T09:27:03.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:03 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:03.533+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.369549+0000 front 2024-04-24T09:26:11.369459+0000 (oldest deadline 2024-04-24T09:26:33.669296+0000) 2024-04-24T09:27:03.927 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:03 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:03.533+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:21.270674+0000 front 2024-04-24T09:26:21.270719+0000 (oldest deadline 2024-04-24T09:26:42.970795+0000) 2024-04-24T09:27:03.927 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:03 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:03.533+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:27.572314+0000 front 2024-04-24T09:26:27.572352+0000 (oldest deadline 2024-04-24T09:26:52.272337+0000) 2024-04-24T09:27:03.927 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:03 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:03.533+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-04-24T09:26:36.274001+0000 front 2024-04-24T09:26:36.273943+0000 (oldest deadline 2024-04-24T09:27:02.173673+0000) 2024-04-24T09:27:04.611 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:04 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:04.227+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:27:04.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:04 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:04.227+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:27:04.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:04 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:04.227+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:27:04.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:04 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:04.227+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.811765+0000 front 2024-04-24T09:26:12.811682+0000 (oldest deadline 2024-04-24T09:26:35.711456+0000) 2024-04-24T09:27:04.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:04 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:04.227+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:22.113177+0000 front 2024-04-24T09:26:22.112999+0000 (oldest deadline 2024-04-24T09:26:46.812888+0000) 2024-04-24T09:27:04.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:04 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:04.227+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:29.113749+0000 front 2024-04-24T09:26:29.113777+0000 (oldest deadline 2024-04-24T09:26:52.013778+0000) 2024-04-24T09:27:04.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:04 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:04.559+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:27:04.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:04 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:04.559+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:27:04.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:04 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:04.559+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:03.168384+0000 front 2024-04-24T09:26:03.168290+0000 (oldest deadline 2024-04-24T09:26:27.268338+0000) 2024-04-24T09:27:04.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:04 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:04.559+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.369549+0000 front 2024-04-24T09:26:11.369459+0000 (oldest deadline 2024-04-24T09:26:33.669296+0000) 2024-04-24T09:27:04.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:04 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:04.559+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:21.270674+0000 front 2024-04-24T09:26:21.270719+0000 (oldest deadline 2024-04-24T09:26:42.970795+0000) 2024-04-24T09:27:04.927 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:04 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:04.559+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:27.572314+0000 front 2024-04-24T09:26:27.572352+0000 (oldest deadline 2024-04-24T09:26:52.272337+0000) 2024-04-24T09:27:04.927 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:04 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:04.559+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-04-24T09:26:36.274001+0000 front 2024-04-24T09:26:36.273943+0000 (oldest deadline 2024-04-24T09:27:02.173673+0000) 2024-04-24T09:27:04.966 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:27:04 smithi090 podman[89747]: 2024-04-24 09:27:04.62954886 +0000 UTC m=+5.570031251 container died 244bfccb3ecf1d2c49af38bb97d19096127fb44873f58bd2021e8d04ac16160b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-24T09:27:05.352 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:27:05 smithi090 podman[89747]: 2024-04-24 09:27:05.159439774 +0000 UTC m=+6.099922174 container cleanup 244bfccb3ecf1d2c49af38bb97d19096127fb44873f58bd2021e8d04ac16160b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-24T09:27:05.353 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:27:05 smithi090 bash[89747]: ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9 2024-04-24T09:27:05.611 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:05 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:05.253+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:27:05.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:05 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:05.253+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:27:05.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:05 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:05.253+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:27:05.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:05 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:05.253+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.811765+0000 front 2024-04-24T09:26:12.811682+0000 (oldest deadline 2024-04-24T09:26:35.711456+0000) 2024-04-24T09:27:05.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:05 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:05.253+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:22.113177+0000 front 2024-04-24T09:26:22.112999+0000 (oldest deadline 2024-04-24T09:26:46.812888+0000) 2024-04-24T09:27:05.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:05 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:05.253+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:29.113749+0000 front 2024-04-24T09:26:29.113777+0000 (oldest deadline 2024-04-24T09:26:52.013778+0000) 2024-04-24T09:27:05.852 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:27:05 smithi090 podman[89764]: 2024-04-24 09:27:05.436340002 +0000 UTC m=+0.786791723 container remove 244bfccb3ecf1d2c49af38bb97d19096127fb44873f58bd2021e8d04ac16160b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=) 2024-04-24T09:27:05.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:05 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:05.596+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:27:05.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:05 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:05.596+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:27:05.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:05 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:05.596+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:03.168384+0000 front 2024-04-24T09:26:03.168290+0000 (oldest deadline 2024-04-24T09:26:27.268338+0000) 2024-04-24T09:27:05.927 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:05 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:05.596+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.369549+0000 front 2024-04-24T09:26:11.369459+0000 (oldest deadline 2024-04-24T09:26:33.669296+0000) 2024-04-24T09:27:05.927 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:05 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:05.596+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:21.270674+0000 front 2024-04-24T09:26:21.270719+0000 (oldest deadline 2024-04-24T09:26:42.970795+0000) 2024-04-24T09:27:05.927 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:05 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:05.596+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:27.572314+0000 front 2024-04-24T09:26:27.572352+0000 (oldest deadline 2024-04-24T09:26:52.272337+0000) 2024-04-24T09:27:05.927 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:05 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:05.596+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-04-24T09:26:36.274001+0000 front 2024-04-24T09:26:36.273943+0000 (oldest deadline 2024-04-24T09:27:02.173673+0000) 2024-04-24T09:27:06.526 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:27:06 smithi090 podman[89852]: 2024-04-24 09:27:06.176096156 +0000 UTC m=+0.033534966 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:27:06.527 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:27:06 smithi090 podman[89852]: 2024-04-24 09:27:06.288571622 +0000 UTC m=+0.146010455 container create 0753388aae4e57ff82cf87ee3815b9d08caa06b458f1c133c880edbe2d2936e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-24T09:27:06.527 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:27:06 smithi090 podman[89852]: 2024-04-24 09:27:06.516847428 +0000 UTC m=+0.374286235 container init 0753388aae4e57ff82cf87ee3815b9d08caa06b458f1c133c880edbe2d2936e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-24T09:27:06.527 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:27:06 smithi090 podman[89852]: 2024-04-24 09:27:06.527265044 +0000 UTC m=+0.384703874 container start 0753388aae4e57ff82cf87ee3815b9d08caa06b458f1c133c880edbe2d2936e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True) 2024-04-24T09:27:06.611 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:06 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:06.283+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:27:06.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:06 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:06.283+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:27:06.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:06 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:06.283+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:27:06.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:06 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:06.283+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.811765+0000 front 2024-04-24T09:26:12.811682+0000 (oldest deadline 2024-04-24T09:26:35.711456+0000) 2024-04-24T09:27:06.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:06 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:06.283+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:22.113177+0000 front 2024-04-24T09:26:22.112999+0000 (oldest deadline 2024-04-24T09:26:46.812888+0000) 2024-04-24T09:27:06.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:06 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:06.283+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:29.113749+0000 front 2024-04-24T09:26:29.113777+0000 (oldest deadline 2024-04-24T09:26:52.013778+0000) 2024-04-24T09:27:06.854 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:27:06 smithi090 podman[89852]: 2024-04-24 09:27:06.674569929 +0000 UTC m=+0.532008735 container attach 0753388aae4e57ff82cf87ee3815b9d08caa06b458f1c133c880edbe2d2936e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-24T09:27:06.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:06 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:06.547+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:27:06.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:06 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:06.547+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:27:06.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:06 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:06.547+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:03.168384+0000 front 2024-04-24T09:26:03.168290+0000 (oldest deadline 2024-04-24T09:26:27.268338+0000) 2024-04-24T09:27:06.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:06 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:06.547+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.369549+0000 front 2024-04-24T09:26:11.369459+0000 (oldest deadline 2024-04-24T09:26:33.669296+0000) 2024-04-24T09:27:06.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:06 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:06.547+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:21.270674+0000 front 2024-04-24T09:26:21.270719+0000 (oldest deadline 2024-04-24T09:26:42.970795+0000) 2024-04-24T09:27:06.927 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:06 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:06.547+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:27.572314+0000 front 2024-04-24T09:26:27.572352+0000 (oldest deadline 2024-04-24T09:26:52.272337+0000) 2024-04-24T09:27:06.927 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:06 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:06.547+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-04-24T09:26:36.274001+0000 front 2024-04-24T09:26:36.273943+0000 (oldest deadline 2024-04-24T09:27:02.173673+0000) 2024-04-24T09:27:07.318 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 24 09:27:06 smithi090 podman[89852]: 2024-04-24 09:27:06.97782159 +0000 UTC m=+0.835260388 container died 0753388aae4e57ff82cf87ee3815b9d08caa06b458f1c133c880edbe2d2936e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-9-deactivate, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-24T09:27:07.611 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:07 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:07.311+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:27:07.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:07 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:07.311+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:27:07.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:07 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:07.311+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:27:07.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:07 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:07.311+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.811765+0000 front 2024-04-24T09:26:12.811682+0000 (oldest deadline 2024-04-24T09:26:35.711456+0000) 2024-04-24T09:27:07.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:07 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:07.311+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:22.113177+0000 front 2024-04-24T09:26:22.112999+0000 (oldest deadline 2024-04-24T09:26:46.812888+0000) 2024-04-24T09:27:07.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:07 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:07.311+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:29.113749+0000 front 2024-04-24T09:26:29.113777+0000 (oldest deadline 2024-04-24T09:26:52.013778+0000) 2024-04-24T09:27:07.763 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@osd.9.service' 2024-04-24T09:27:07.860 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:27:07.861 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-24T09:27:07.861 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-24T09:27:07.861 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-30ee9be2-021a-11ef-bc93-c7b262605968@osd.10 2024-04-24T09:27:07.887 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:07 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:07.515+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:27:07.887 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:07 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:07.515+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:27:07.887 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:07 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:07.515+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:03.168384+0000 front 2024-04-24T09:26:03.168290+0000 (oldest deadline 2024-04-24T09:26:27.268338+0000) 2024-04-24T09:27:07.887 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:07 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:07.515+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.369549+0000 front 2024-04-24T09:26:11.369459+0000 (oldest deadline 2024-04-24T09:26:33.669296+0000) 2024-04-24T09:27:07.887 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:07 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:07.515+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:21.270674+0000 front 2024-04-24T09:26:21.270719+0000 (oldest deadline 2024-04-24T09:26:42.970795+0000) 2024-04-24T09:27:07.888 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:07 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:07.515+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:27.572314+0000 front 2024-04-24T09:26:27.572352+0000 (oldest deadline 2024-04-24T09:26:52.272337+0000) 2024-04-24T09:27:07.888 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:07 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:07.515+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-04-24T09:26:36.274001+0000 front 2024-04-24T09:26:36.273943+0000 (oldest deadline 2024-04-24T09:27:02.173673+0000) 2024-04-24T09:27:08.166 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:07 smithi114 systemd[1]: Stopping Ceph osd.10 for 30ee9be2-021a-11ef-bc93-c7b262605968... 2024-04-24T09:27:08.426 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:08 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:08.166+0000 7f4993c30700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:27:08.426 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:08 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:08.166+0000 7f4993c30700 -1 osd.10 91 *** Got signal Terminated *** 2024-04-24T09:27:08.426 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:08 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:08.166+0000 7f4993c30700 -1 osd.10 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T09:27:08.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:08 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:08.323+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:27:08.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:08 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:08.323+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:27:08.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:08 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:08.323+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:27:08.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:08 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:08.323+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.811765+0000 front 2024-04-24T09:26:12.811682+0000 (oldest deadline 2024-04-24T09:26:35.711456+0000) 2024-04-24T09:27:08.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:08 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:08.323+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:22.113177+0000 front 2024-04-24T09:26:22.112999+0000 (oldest deadline 2024-04-24T09:26:46.812888+0000) 2024-04-24T09:27:08.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:08 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:08.323+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:29.113749+0000 front 2024-04-24T09:26:29.113777+0000 (oldest deadline 2024-04-24T09:26:52.013778+0000) 2024-04-24T09:27:08.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:08 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:08.536+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:27:08.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:08 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:08.536+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:27:08.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:08 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:08.536+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:03.168384+0000 front 2024-04-24T09:26:03.168290+0000 (oldest deadline 2024-04-24T09:26:27.268338+0000) 2024-04-24T09:27:08.927 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:08 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:08.536+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.369549+0000 front 2024-04-24T09:26:11.369459+0000 (oldest deadline 2024-04-24T09:26:33.669296+0000) 2024-04-24T09:27:08.927 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:08 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:08.536+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:21.270674+0000 front 2024-04-24T09:26:21.270719+0000 (oldest deadline 2024-04-24T09:26:42.970795+0000) 2024-04-24T09:27:08.927 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:08 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:08.536+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:27.572314+0000 front 2024-04-24T09:26:27.572352+0000 (oldest deadline 2024-04-24T09:26:52.272337+0000) 2024-04-24T09:27:08.927 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:08 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:08.536+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-04-24T09:26:36.274001+0000 front 2024-04-24T09:26:36.273943+0000 (oldest deadline 2024-04-24T09:27:02.173673+0000) 2024-04-24T09:27:09.611 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:09 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:09.338+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:27:09.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:09 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:09.338+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:27:09.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:09 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:09.338+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:27:09.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:09 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:09.338+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.811765+0000 front 2024-04-24T09:26:12.811682+0000 (oldest deadline 2024-04-24T09:26:35.711456+0000) 2024-04-24T09:27:09.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:09 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:09.338+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:22.113177+0000 front 2024-04-24T09:26:22.112999+0000 (oldest deadline 2024-04-24T09:26:46.812888+0000) 2024-04-24T09:27:09.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:09 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:09.338+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:29.113749+0000 front 2024-04-24T09:26:29.113777+0000 (oldest deadline 2024-04-24T09:26:52.013778+0000) 2024-04-24T09:27:09.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:09 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:09.567+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:27:09.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:09 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:09.567+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:27:09.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:09 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:09.567+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:03.168384+0000 front 2024-04-24T09:26:03.168290+0000 (oldest deadline 2024-04-24T09:26:27.268338+0000) 2024-04-24T09:27:09.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:09 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:09.567+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.369549+0000 front 2024-04-24T09:26:11.369459+0000 (oldest deadline 2024-04-24T09:26:33.669296+0000) 2024-04-24T09:27:09.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:09 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:09.567+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:21.270674+0000 front 2024-04-24T09:26:21.270719+0000 (oldest deadline 2024-04-24T09:26:42.970795+0000) 2024-04-24T09:27:09.927 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:09 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:09.567+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:27.572314+0000 front 2024-04-24T09:26:27.572352+0000 (oldest deadline 2024-04-24T09:26:52.272337+0000) 2024-04-24T09:27:09.927 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:09 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:09.567+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-04-24T09:26:36.274001+0000 front 2024-04-24T09:26:36.273943+0000 (oldest deadline 2024-04-24T09:27:02.173673+0000) 2024-04-24T09:27:10.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:10 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:10.319+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:27:10.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:10 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:10.319+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:27:10.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:10 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:10.319+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:27:10.613 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:10 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:10.319+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.811765+0000 front 2024-04-24T09:26:12.811682+0000 (oldest deadline 2024-04-24T09:26:35.711456+0000) 2024-04-24T09:27:10.613 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:10 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:10.319+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:22.113177+0000 front 2024-04-24T09:26:22.112999+0000 (oldest deadline 2024-04-24T09:26:46.812888+0000) 2024-04-24T09:27:10.613 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:10 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:10.319+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:29.113749+0000 front 2024-04-24T09:26:29.113777+0000 (oldest deadline 2024-04-24T09:26:52.013778+0000) 2024-04-24T09:27:10.827 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:10 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:10.549+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:27:10.827 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:10 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:10.549+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:27:10.827 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:10 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:10.549+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:03.168384+0000 front 2024-04-24T09:26:03.168290+0000 (oldest deadline 2024-04-24T09:26:27.268338+0000) 2024-04-24T09:27:10.828 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:10 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:10.549+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.369549+0000 front 2024-04-24T09:26:11.369459+0000 (oldest deadline 2024-04-24T09:26:33.669296+0000) 2024-04-24T09:27:10.828 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:10 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:10.549+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:21.270674+0000 front 2024-04-24T09:26:21.270719+0000 (oldest deadline 2024-04-24T09:26:42.970795+0000) 2024-04-24T09:27:10.828 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:10 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:10.549+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:27.572314+0000 front 2024-04-24T09:26:27.572352+0000 (oldest deadline 2024-04-24T09:26:52.272337+0000) 2024-04-24T09:27:10.828 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:10 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:10.549+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-04-24T09:26:36.274001+0000 front 2024-04-24T09:26:36.273943+0000 (oldest deadline 2024-04-24T09:27:02.173673+0000) 2024-04-24T09:27:11.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:11 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:11.329+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:27:11.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:11 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:11.329+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:27:11.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:11 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:11.329+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:27:11.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:11 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:11.329+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.811765+0000 front 2024-04-24T09:26:12.811682+0000 (oldest deadline 2024-04-24T09:26:35.711456+0000) 2024-04-24T09:27:11.613 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:11 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:11.329+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:22.113177+0000 front 2024-04-24T09:26:22.112999+0000 (oldest deadline 2024-04-24T09:26:46.812888+0000) 2024-04-24T09:27:11.613 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:11 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:11.329+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:29.113749+0000 front 2024-04-24T09:26:29.113777+0000 (oldest deadline 2024-04-24T09:26:52.013778+0000) 2024-04-24T09:27:11.844 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:11 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:11.542+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:27:11.844 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:11 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:11.542+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:27:11.844 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:11 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:11.542+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:03.168384+0000 front 2024-04-24T09:26:03.168290+0000 (oldest deadline 2024-04-24T09:26:27.268338+0000) 2024-04-24T09:27:11.845 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:11 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:11.542+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.369549+0000 front 2024-04-24T09:26:11.369459+0000 (oldest deadline 2024-04-24T09:26:33.669296+0000) 2024-04-24T09:27:11.845 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:11 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:11.542+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:21.270674+0000 front 2024-04-24T09:26:21.270719+0000 (oldest deadline 2024-04-24T09:26:42.970795+0000) 2024-04-24T09:27:11.845 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:11 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:11.542+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:27.572314+0000 front 2024-04-24T09:26:27.572352+0000 (oldest deadline 2024-04-24T09:26:52.272337+0000) 2024-04-24T09:27:11.845 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:11 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:11.542+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-04-24T09:26:36.274001+0000 front 2024-04-24T09:26:36.273943+0000 (oldest deadline 2024-04-24T09:27:02.173673+0000) 2024-04-24T09:27:12.611 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:12 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:12.358+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:27:12.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:12 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:12.358+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:27:12.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:12 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:12.358+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:27:12.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:12 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:12.358+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.811765+0000 front 2024-04-24T09:26:12.811682+0000 (oldest deadline 2024-04-24T09:26:35.711456+0000) 2024-04-24T09:27:12.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:12 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:12.358+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:22.113177+0000 front 2024-04-24T09:26:22.112999+0000 (oldest deadline 2024-04-24T09:26:46.812888+0000) 2024-04-24T09:27:12.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:12 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:12.358+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:29.113749+0000 front 2024-04-24T09:26:29.113777+0000 (oldest deadline 2024-04-24T09:26:52.013778+0000) 2024-04-24T09:27:12.612 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:12 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:12.358+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6820 osd.7 since back 2024-04-24T09:26:45.916262+0000 front 2024-04-24T09:26:45.916283+0000 (oldest deadline 2024-04-24T09:27:11.816280+0000) 2024-04-24T09:27:12.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:12 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:12.568+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.665471+0000 front 2024-04-24T09:25:45.665529+0000 (oldest deadline 2024-04-24T09:26:10.365253+0000) 2024-04-24T09:27:12.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:12 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:12.568+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.967232+0000 front 2024-04-24T09:25:54.967181+0000 (oldest deadline 2024-04-24T09:26:19.067285+0000) 2024-04-24T09:27:12.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:12 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:12.568+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:03.168384+0000 front 2024-04-24T09:26:03.168290+0000 (oldest deadline 2024-04-24T09:26:27.268338+0000) 2024-04-24T09:27:12.926 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:12 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:12.568+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:11.369549+0000 front 2024-04-24T09:26:11.369459+0000 (oldest deadline 2024-04-24T09:26:33.669296+0000) 2024-04-24T09:27:12.927 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:12 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:12.568+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:21.270674+0000 front 2024-04-24T09:26:21.270719+0000 (oldest deadline 2024-04-24T09:26:42.970795+0000) 2024-04-24T09:27:12.927 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:12 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:12.568+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:27.572314+0000 front 2024-04-24T09:26:27.572352+0000 (oldest deadline 2024-04-24T09:26:52.272337+0000) 2024-04-24T09:27:12.927 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:12 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10[57934]: 2024-04-24T09:27:12.568+0000 7f498c21c700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-04-24T09:26:36.274001+0000 front 2024-04-24T09:26:36.273943+0000 (oldest deadline 2024-04-24T09:27:02.173673+0000) 2024-04-24T09:27:13.841 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:13 smithi114 podman[67706]: 2024-04-24 09:27:13.575533422 +0000 UTC m=+5.523393802 container died 18bf6d3d68bef5a18e4a9c745cd7e9c0447cbe625cbda936da387959668009c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-24T09:27:13.861 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:13 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:13.397+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:27:13.862 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:13 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:13.397+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:27:13.862 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:13 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:13.397+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:27:13.862 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:13 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:13.397+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.811765+0000 front 2024-04-24T09:26:12.811682+0000 (oldest deadline 2024-04-24T09:26:35.711456+0000) 2024-04-24T09:27:13.862 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:13 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:13.397+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:22.113177+0000 front 2024-04-24T09:26:22.112999+0000 (oldest deadline 2024-04-24T09:26:46.812888+0000) 2024-04-24T09:27:13.862 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:13 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:13.397+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:29.113749+0000 front 2024-04-24T09:26:29.113777+0000 (oldest deadline 2024-04-24T09:26:52.013778+0000) 2024-04-24T09:27:13.862 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:13 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:13.397+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6820 osd.7 since back 2024-04-24T09:26:45.916262+0000 front 2024-04-24T09:26:45.916283+0000 (oldest deadline 2024-04-24T09:27:11.816280+0000) 2024-04-24T09:27:14.176 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:13 smithi114 podman[67706]: 2024-04-24 09:27:13.933217872 +0000 UTC m=+5.881078261 container cleanup 18bf6d3d68bef5a18e4a9c745cd7e9c0447cbe625cbda936da387959668009c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-24T09:27:14.176 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:13 smithi114 bash[67706]: ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10 2024-04-24T09:27:14.543 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:14 smithi114 podman[67735]: 2024-04-24 09:27:14.226174543 +0000 UTC m=+0.643143581 container remove 18bf6d3d68bef5a18e4a9c745cd7e9c0447cbe625cbda936da387959668009c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T09:27:14.820 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:14 smithi114 podman[67821]: 2024-04-24 09:27:14.446850872 +0000 UTC m=+0.017678765 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:27:14.820 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:14 smithi114 podman[67821]: 2024-04-24 09:27:14.543973179 +0000 UTC m=+0.114801102 container create 6140e984bc152b074ce7ffe348c2c2b9d6dc0c1c9b2d49128a6cfb3da9090e2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240424) 2024-04-24T09:27:14.821 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:14 smithi114 podman[67821]: 2024-04-24 09:27:14.746329645 +0000 UTC m=+0.317157511 container init 6140e984bc152b074ce7ffe348c2c2b9d6dc0c1c9b2d49128a6cfb3da9090e2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-24T09:27:14.821 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:14 smithi114 podman[67821]: 2024-04-24 09:27:14.749418555 +0000 UTC m=+0.320246427 container start 6140e984bc152b074ce7ffe348c2c2b9d6dc0c1c9b2d49128a6cfb3da9090e2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-24T09:27:14.821 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:14 smithi114 podman[67821]: 2024-04-24 09:27:14.821273301 +0000 UTC m=+0.392101190 container attach 6140e984bc152b074ce7ffe348c2c2b9d6dc0c1c9b2d49128a6cfb3da9090e2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424) 2024-04-24T09:27:14.861 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:14 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:14.436+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:27:14.862 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:14 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:14.436+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:27:14.862 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:14 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:14.436+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:27:14.862 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:14 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:14.436+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.811765+0000 front 2024-04-24T09:26:12.811682+0000 (oldest deadline 2024-04-24T09:26:35.711456+0000) 2024-04-24T09:27:14.862 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:14 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:14.436+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:22.113177+0000 front 2024-04-24T09:26:22.112999+0000 (oldest deadline 2024-04-24T09:26:46.812888+0000) 2024-04-24T09:27:14.862 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:14 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:14.436+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:29.113749+0000 front 2024-04-24T09:26:29.113777+0000 (oldest deadline 2024-04-24T09:26:52.013778+0000) 2024-04-24T09:27:14.862 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:14 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:14.436+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6820 osd.7 since back 2024-04-24T09:26:45.916262+0000 front 2024-04-24T09:26:45.916283+0000 (oldest deadline 2024-04-24T09:27:11.816280+0000) 2024-04-24T09:27:15.176 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:15 smithi114 podman[67821]: 2024-04-24 09:27:15.040276834 +0000 UTC m=+0.611104710 container died 6140e984bc152b074ce7ffe348c2c2b9d6dc0c1c9b2d49128a6cfb3da9090e2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-24T09:27:15.575 INFO:journalctl@ceph.osd.10.smithi114.stdout:Apr 24 09:27:15 smithi114 podman[67847]: 2024-04-24 09:27:15.574271416 +0000 UTC m=+0.525270789 container remove 6140e984bc152b074ce7ffe348c2c2b9d6dc0c1c9b2d49128a6cfb3da9090e2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-24T09:27:15.587 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@osd.10.service' 2024-04-24T09:27:15.666 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:27:15.666 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-24T09:27:15.667 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-24T09:27:15.667 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-30ee9be2-021a-11ef-bc93-c7b262605968@osd.11 2024-04-24T09:27:15.693 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:15 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:15.404+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:27:15.693 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:15 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:15.404+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:27:15.693 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:15 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:15.404+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:27:15.693 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:15 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:15.404+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.811765+0000 front 2024-04-24T09:26:12.811682+0000 (oldest deadline 2024-04-24T09:26:35.711456+0000) 2024-04-24T09:27:15.694 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:15 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:15.404+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:22.113177+0000 front 2024-04-24T09:26:22.112999+0000 (oldest deadline 2024-04-24T09:26:46.812888+0000) 2024-04-24T09:27:15.694 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:15 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:15.404+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:29.113749+0000 front 2024-04-24T09:26:29.113777+0000 (oldest deadline 2024-04-24T09:26:52.013778+0000) 2024-04-24T09:27:15.694 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:15 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:15.404+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6820 osd.7 since back 2024-04-24T09:26:45.916262+0000 front 2024-04-24T09:26:45.916283+0000 (oldest deadline 2024-04-24T09:27:11.816280+0000) 2024-04-24T09:27:15.958 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:15 smithi149 systemd[1]: Stopping Ceph osd.11 for 30ee9be2-021a-11ef-bc93-c7b262605968... 2024-04-24T09:27:16.361 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:15 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:15.958+0000 7f5ea13f6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:27:16.362 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:15 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:15.958+0000 7f5ea13f6700 -1 osd.11 91 *** Got signal Terminated *** 2024-04-24T09:27:16.362 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:15 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:15.958+0000 7f5ea13f6700 -1 osd.11 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T09:27:16.861 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:16 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:16.427+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:27:16.862 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:16 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:16.427+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:27:16.862 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:16 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:16.427+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:27:16.862 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:16 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:16.427+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.811765+0000 front 2024-04-24T09:26:12.811682+0000 (oldest deadline 2024-04-24T09:26:35.711456+0000) 2024-04-24T09:27:16.862 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:16 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:16.427+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:22.113177+0000 front 2024-04-24T09:26:22.112999+0000 (oldest deadline 2024-04-24T09:26:46.812888+0000) 2024-04-24T09:27:16.862 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:16 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:16.427+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:29.113749+0000 front 2024-04-24T09:26:29.113777+0000 (oldest deadline 2024-04-24T09:26:52.013778+0000) 2024-04-24T09:27:16.862 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:16 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:16.427+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6820 osd.7 since back 2024-04-24T09:26:45.916262+0000 front 2024-04-24T09:26:45.916283+0000 (oldest deadline 2024-04-24T09:27:11.816280+0000) 2024-04-24T09:27:17.763 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:17 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:17.421+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:27:17.764 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:17 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:17.421+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:27:17.764 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:17 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:17.421+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:27:17.764 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:17 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:17.421+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.811765+0000 front 2024-04-24T09:26:12.811682+0000 (oldest deadline 2024-04-24T09:26:35.711456+0000) 2024-04-24T09:27:17.764 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:17 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:17.421+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:22.113177+0000 front 2024-04-24T09:26:22.112999+0000 (oldest deadline 2024-04-24T09:26:46.812888+0000) 2024-04-24T09:27:17.764 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:17 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:17.421+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:29.113749+0000 front 2024-04-24T09:26:29.113777+0000 (oldest deadline 2024-04-24T09:26:52.013778+0000) 2024-04-24T09:27:17.764 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:17 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:17.421+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6820 osd.7 since back 2024-04-24T09:26:45.916262+0000 front 2024-04-24T09:26:45.916283+0000 (oldest deadline 2024-04-24T09:27:11.816280+0000) 2024-04-24T09:27:17.764 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:17 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:17.421+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6820 osd.8 since back 2024-04-24T09:26:51.816701+0000 front 2024-04-24T09:26:51.816663+0000 (oldest deadline 2024-04-24T09:27:17.116702+0000) 2024-04-24T09:27:18.862 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:18 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:18.446+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:27:18.862 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:18 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:18.446+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:27:18.862 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:18 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:18.446+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:27:18.862 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:18 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:18.446+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.811765+0000 front 2024-04-24T09:26:12.811682+0000 (oldest deadline 2024-04-24T09:26:35.711456+0000) 2024-04-24T09:27:18.863 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:18 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:18.446+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:22.113177+0000 front 2024-04-24T09:26:22.112999+0000 (oldest deadline 2024-04-24T09:26:46.812888+0000) 2024-04-24T09:27:18.863 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:18 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:18.446+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:29.113749+0000 front 2024-04-24T09:26:29.113777+0000 (oldest deadline 2024-04-24T09:26:52.013778+0000) 2024-04-24T09:27:18.863 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:18 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:18.446+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6820 osd.7 since back 2024-04-24T09:26:45.916262+0000 front 2024-04-24T09:26:45.916283+0000 (oldest deadline 2024-04-24T09:27:11.816280+0000) 2024-04-24T09:27:18.863 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:18 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:18.446+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6820 osd.8 since back 2024-04-24T09:26:51.816701+0000 front 2024-04-24T09:26:51.816663+0000 (oldest deadline 2024-04-24T09:27:17.116702+0000) 2024-04-24T09:27:19.862 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:19 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:19.398+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:27:19.862 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:19 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:19.398+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:27:19.862 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:19 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:19.398+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:27:19.862 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:19 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:19.398+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.811765+0000 front 2024-04-24T09:26:12.811682+0000 (oldest deadline 2024-04-24T09:26:35.711456+0000) 2024-04-24T09:27:19.862 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:19 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:19.398+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:22.113177+0000 front 2024-04-24T09:26:22.112999+0000 (oldest deadline 2024-04-24T09:26:46.812888+0000) 2024-04-24T09:27:19.863 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:19 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:19.398+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:29.113749+0000 front 2024-04-24T09:26:29.113777+0000 (oldest deadline 2024-04-24T09:26:52.013778+0000) 2024-04-24T09:27:19.863 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:19 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:19.398+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6820 osd.7 since back 2024-04-24T09:26:45.916262+0000 front 2024-04-24T09:26:45.916283+0000 (oldest deadline 2024-04-24T09:27:11.816280+0000) 2024-04-24T09:27:19.863 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:19 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:19.398+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6820 osd.8 since back 2024-04-24T09:26:51.816701+0000 front 2024-04-24T09:26:51.816663+0000 (oldest deadline 2024-04-24T09:27:17.116702+0000) 2024-04-24T09:27:20.861 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:20 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:20.440+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-24T09:25:45.005976+0000 front 2024-04-24T09:25:45.006056+0000 (oldest deadline 2024-04-24T09:26:07.906012+0000) 2024-04-24T09:27:20.862 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:20 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:20.440+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6804 osd.1 since back 2024-04-24T09:25:54.307832+0000 front 2024-04-24T09:25:54.307843+0000 (oldest deadline 2024-04-24T09:26:17.807904+0000) 2024-04-24T09:27:20.862 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:20 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:20.440+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2024-04-24T09:26:02.810551+0000 front 2024-04-24T09:26:02.810609+0000 (oldest deadline 2024-04-24T09:26:28.710680+0000) 2024-04-24T09:27:20.862 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:20 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:20.440+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-04-24T09:26:12.811765+0000 front 2024-04-24T09:26:12.811682+0000 (oldest deadline 2024-04-24T09:26:35.711456+0000) 2024-04-24T09:27:20.862 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:20 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:20.440+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6812 osd.4 since back 2024-04-24T09:26:22.113177+0000 front 2024-04-24T09:26:22.112999+0000 (oldest deadline 2024-04-24T09:26:46.812888+0000) 2024-04-24T09:27:20.862 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:20 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:20.440+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2024-04-24T09:26:29.113749+0000 front 2024-04-24T09:26:29.113777+0000 (oldest deadline 2024-04-24T09:26:52.013778+0000) 2024-04-24T09:27:20.862 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:20 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:20.440+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.114:6820 osd.7 since back 2024-04-24T09:26:45.916262+0000 front 2024-04-24T09:26:45.916283+0000 (oldest deadline 2024-04-24T09:27:11.816280+0000) 2024-04-24T09:27:20.863 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:20 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11[58691]: 2024-04-24T09:27:20.440+0000 7f5e999e2700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.149:6820 osd.8 since back 2024-04-24T09:26:51.816701+0000 front 2024-04-24T09:26:51.816663+0000 (oldest deadline 2024-04-24T09:27:17.116702+0000) 2024-04-24T09:27:21.533 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:21 smithi149 podman[68499]: 2024-04-24 09:27:21.263181643 +0000 UTC m=+5.402142007 container died f8562d2e7c5f99fe13cbc6ec7d902fdeea766ef6a6a231ccfc6628b86a9a8a42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T09:27:21.862 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:21 smithi149 podman[68499]: 2024-04-24 09:27:21.632935143 +0000 UTC m=+5.771895560 container cleanup f8562d2e7c5f99fe13cbc6ec7d902fdeea766ef6a6a231ccfc6628b86a9a8a42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T09:27:21.862 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:21 smithi149 bash[68499]: ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11 2024-04-24T09:27:22.199 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:21 smithi149 podman[68529]: 2024-04-24 09:27:21.884407446 +0000 UTC m=+0.620495790 container remove f8562d2e7c5f99fe13cbc6ec7d902fdeea766ef6a6a231ccfc6628b86a9a8a42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T09:27:22.437 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:22 smithi149 podman[68621]: 2024-04-24 09:27:22.102908099 +0000 UTC m=+0.018472602 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:27:22.438 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:22 smithi149 podman[68621]: 2024-04-24 09:27:22.226842608 +0000 UTC m=+0.142407097 container create 81c23e557f4c38a4c595796d8ce9bfd17db3195d20e0dfd56e79df63ef0f3497 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=) 2024-04-24T09:27:22.438 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:22 smithi149 podman[68621]: 2024-04-24 09:27:22.437498437 +0000 UTC m=+0.353062927 container init 81c23e557f4c38a4c595796d8ce9bfd17db3195d20e0dfd56e79df63ef0f3497 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T09:27:22.519 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:22 smithi149 podman[68621]: 2024-04-24 09:27:22.440724322 +0000 UTC m=+0.356288820 container start 81c23e557f4c38a4c595796d8ce9bfd17db3195d20e0dfd56e79df63ef0f3497 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-24T09:27:22.520 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:22 smithi149 podman[68621]: 2024-04-24 09:27:22.520721517 +0000 UTC m=+0.436286023 container attach 81c23e557f4c38a4c595796d8ce9bfd17db3195d20e0dfd56e79df63ef0f3497 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-24T09:27:22.862 INFO:journalctl@ceph.osd.11.smithi149.stdout:Apr 24 09:27:22 smithi149 podman[68621]: 2024-04-24 09:27:22.742061677 +0000 UTC m=+0.657626175 container died 81c23e557f4c38a4c595796d8ce9bfd17db3195d20e0dfd56e79df63ef0f3497 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T09:27:23.324 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@osd.11.service' 2024-04-24T09:27:23.403 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:27:23.403 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-24T09:27:23.403 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-24T09:27:23.404 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mds.a 2024-04-24T09:27:23.852 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 24 09:27:23 smithi090 systemd[1]: Stopping Ceph mds.a for 30ee9be2-021a-11ef-bc93-c7b262605968... 2024-04-24T09:27:24.352 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 24 09:27:23 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-a[69432]: 2024-04-24T09:27:23.981+0000 7f3018a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:27:24.352 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 24 09:27:23 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-a[69432]: 2024-04-24T09:27:23.981+0000 7f3018a00700 -1 mds.a *** got signal Terminated *** 2024-04-24T09:27:28.319 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 24 09:27:28 smithi090 podman[89970]: 2024-04-24 09:27:28.002550168 +0000 UTC m=+4.145063343 container died ec3fa743ec6c4ea68c526bbde9171b9fc1a8969c791fe7faf0104528c78c778d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-a, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T09:27:28.602 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 24 09:27:28 smithi090 podman[89970]: 2024-04-24 09:27:28.441519542 +0000 UTC m=+4.584032710 container cleanup ec3fa743ec6c4ea68c526bbde9171b9fc1a8969c791fe7faf0104528c78c778d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T09:27:28.602 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 24 09:27:28 smithi090 bash[89970]: ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-a 2024-04-24T09:27:29.102 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 24 09:27:28 smithi090 podman[89984]: 2024-04-24 09:27:28.701360307 +0000 UTC m=+0.684891406 container remove ec3fa743ec6c4ea68c526bbde9171b9fc1a8969c791fe7faf0104528c78c778d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-a, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-24T09:27:29.220 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mds.a.service' 2024-04-24T09:27:29.330 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:27:29.330 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-24T09:27:29.331 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-24T09:27:29.331 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mds.d 2024-04-24T09:27:29.603 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 24 09:27:29 smithi090 systemd[1]: Stopping Ceph mds.d for 30ee9be2-021a-11ef-bc93-c7b262605968... 2024-04-24T09:27:30.352 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 24 09:27:29 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-d[68433]: 2024-04-24T09:27:29.859+0000 7f1ca3600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:27:30.352 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 24 09:27:29 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-d[68433]: 2024-04-24T09:27:29.859+0000 7f1ca3600700 -1 mds.d *** got signal Terminated *** 2024-04-24T09:27:34.159 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 24 09:27:33 smithi090 podman[90089]: 2024-04-24 09:27:33.880801651 +0000 UTC m=+4.155377861 container died 65013c07581f6edae5a6c6298d948af91f66c24da1bfdde8509f1d47d740e6c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-d, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T09:27:34.490 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 24 09:27:34 smithi090 podman[90089]: 2024-04-24 09:27:34.338043801 +0000 UTC m=+4.612620004 container cleanup 65013c07581f6edae5a6c6298d948af91f66c24da1bfdde8509f1d47d740e6c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-d, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T09:27:34.490 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 24 09:27:34 smithi090 bash[90089]: ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-d 2024-04-24T09:27:34.852 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 24 09:27:34 smithi090 podman[90103]: 2024-04-24 09:27:34.490443222 +0000 UTC m=+0.599898159 container remove 65013c07581f6edae5a6c6298d948af91f66c24da1bfdde8509f1d47d740e6c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-d, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T09:27:34.987 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mds.d.service' 2024-04-24T09:27:35.094 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:27:35.095 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-24T09:27:35.095 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-24T09:27:35.095 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mds.g 2024-04-24T09:27:35.602 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 24 09:27:35 smithi090 systemd[1]: Stopping Ceph mds.g for 30ee9be2-021a-11ef-bc93-c7b262605968... 2024-04-24T09:27:36.102 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 24 09:27:35 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-g[69046]: 2024-04-24T09:27:35.699+0000 7fc23c200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:27:36.102 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 24 09:27:35 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-g[69046]: 2024-04-24T09:27:35.699+0000 7fc23c200700 -1 mds.g *** got signal Terminated *** 2024-04-24T09:27:37.082 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 24 09:27:36 smithi090 podman[90209]: 2024-04-24 09:27:36.740922588 +0000 UTC m=+1.219225798 container died 8d2f12b5f7dea0a340d2580fd34517df9702c7de3e9d70e59f0fe60d29abc6cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-g, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True) 2024-04-24T09:27:37.352 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 24 09:27:37 smithi090 podman[90209]: 2024-04-24 09:27:37.210115144 +0000 UTC m=+1.688418380 container cleanup 8d2f12b5f7dea0a340d2580fd34517df9702c7de3e9d70e59f0fe60d29abc6cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-g, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-24T09:27:37.352 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 24 09:27:37 smithi090 bash[90209]: ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-g 2024-04-24T09:27:37.852 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 24 09:27:37 smithi090 podman[90223]: 2024-04-24 09:27:37.413491249 +0000 UTC m=+0.655865236 container remove 8d2f12b5f7dea0a340d2580fd34517df9702c7de3e9d70e59f0fe60d29abc6cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-g, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-24T09:27:37.891 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mds.g.service' 2024-04-24T09:27:37.993 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:27:37.994 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-24T09:27:37.994 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-24T09:27:37.994 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mds.j 2024-04-24T09:27:38.578 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 24 09:27:38 smithi090 systemd[1]: Stopping Ceph mds.j for 30ee9be2-021a-11ef-bc93-c7b262605968... 2024-04-24T09:27:38.852 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 24 09:27:38 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-j[68052]: 2024-04-24T09:27:38.578+0000 7fcd8b200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:27:38.853 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 24 09:27:38 smithi090 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-j[68052]: 2024-04-24T09:27:38.578+0000 7fcd8b200700 -1 mds.j *** got signal Terminated *** 2024-04-24T09:27:40.670 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 24 09:27:40 smithi090 podman[90329]: 2024-04-24 09:27:40.369450159 +0000 UTC m=+1.910669239 container died 5902404428e682c853e8757243755db4e44d12dda9b5fa2ed9c5d176d3350c8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-j, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T09:27:41.025 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 24 09:27:40 smithi090 podman[90329]: 2024-04-24 09:27:40.791315315 +0000 UTC m=+2.332534415 container cleanup 5902404428e682c853e8757243755db4e44d12dda9b5fa2ed9c5d176d3350c8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-j, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T09:27:41.025 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 24 09:27:40 smithi090 bash[90329]: ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-j 2024-04-24T09:27:41.352 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 24 09:27:41 smithi090 podman[90343]: 2024-04-24 09:27:41.025325827 +0000 UTC m=+0.634742077 container remove 5902404428e682c853e8757243755db4e44d12dda9b5fa2ed9c5d176d3350c8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-j, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-24T09:27:41.548 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mds.j.service' 2024-04-24T09:27:41.651 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:27:41.652 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-24T09:27:41.652 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-24T09:27:41.652 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mds.b 2024-04-24T09:27:42.032 INFO:journalctl@ceph.mds.b.smithi114.stdout:Apr 24 09:27:41 smithi114 systemd[1]: Stopping Ceph mds.b for 30ee9be2-021a-11ef-bc93-c7b262605968... 2024-04-24T09:27:42.299 INFO:journalctl@ceph.mds.b.smithi114.stdout:Apr 24 09:27:42 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-b[63608]: 2024-04-24T09:27:42.032+0000 7f1265165700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:27:42.299 INFO:journalctl@ceph.mds.b.smithi114.stdout:Apr 24 09:27:42 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-b[63608]: 2024-04-24T09:27:42.032+0000 7f1265165700 -1 mds.b *** got signal Terminated *** 2024-04-24T09:27:44.658 INFO:journalctl@ceph.mds.b.smithi114.stdout:Apr 24 09:27:44 smithi114 podman[67927]: 2024-04-24 09:27:44.288556197 +0000 UTC m=+2.447034836 container died a4d4b1357aac81c5119ea551977cf821f220f08a9204ef98aa2e475262d2e74d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T09:27:44.926 INFO:journalctl@ceph.mds.b.smithi114.stdout:Apr 24 09:27:44 smithi114 podman[67927]: 2024-04-24 09:27:44.741379593 +0000 UTC m=+2.899858243 container cleanup a4d4b1357aac81c5119ea551977cf821f220f08a9204ef98aa2e475262d2e74d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-b, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-24T09:27:44.926 INFO:journalctl@ceph.mds.b.smithi114.stdout:Apr 24 09:27:44 smithi114 bash[67927]: ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-b 2024-04-24T09:27:45.176 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mds.b.service' 2024-04-24T09:27:45.211 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:27:45.211 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-24T09:27:45.212 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-24T09:27:45.212 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mds.e 2024-04-24T09:27:45.676 INFO:journalctl@ceph.mds.e.smithi114.stdout:Apr 24 09:27:45 smithi114 systemd[1]: Stopping Ceph mds.e for 30ee9be2-021a-11ef-bc93-c7b262605968... 2024-04-24T09:27:45.676 INFO:journalctl@ceph.mds.e.smithi114.stdout:Apr 24 09:27:45 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-e[62977]: 2024-04-24T09:27:45.520+0000 7f78e0356700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:27:45.676 INFO:journalctl@ceph.mds.e.smithi114.stdout:Apr 24 09:27:45 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-e[62977]: 2024-04-24T09:27:45.520+0000 7f78e0356700 -1 mds.e *** got signal Terminated *** 2024-04-24T09:27:48.628 INFO:journalctl@ceph.mds.e.smithi114.stdout:Apr 24 09:27:48 smithi114 podman[68060]: 2024-04-24 09:27:48.275753507 +0000 UTC m=+2.848703132 container died 668c731cf63b2dd70d5c88ef75e206b8a5f8a6a5e6c3d51d2d440ca4e11856f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-e, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T09:27:48.903 INFO:journalctl@ceph.mds.e.smithi114.stdout:Apr 24 09:27:48 smithi114 podman[68060]: 2024-04-24 09:27:48.703419917 +0000 UTC m=+3.276369545 container cleanup 668c731cf63b2dd70d5c88ef75e206b8a5f8a6a5e6c3d51d2d440ca4e11856f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-e, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True) 2024-04-24T09:27:48.904 INFO:journalctl@ceph.mds.e.smithi114.stdout:Apr 24 09:27:48 smithi114 bash[68060]: ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-e 2024-04-24T09:27:49.123 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mds.e.service' 2024-04-24T09:27:49.155 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:27:49.155 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-24T09:27:49.155 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-24T09:27:49.155 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mds.h 2024-04-24T09:27:49.426 INFO:journalctl@ceph.mds.h.smithi114.stdout:Apr 24 09:27:49 smithi114 systemd[1]: Stopping Ceph mds.h for 30ee9be2-021a-11ef-bc93-c7b262605968... 2024-04-24T09:27:49.926 INFO:journalctl@ceph.mds.h.smithi114.stdout:Apr 24 09:27:49 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-h[63285]: 2024-04-24T09:27:49.473+0000 7fcfab490700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:27:49.926 INFO:journalctl@ceph.mds.h.smithi114.stdout:Apr 24 09:27:49 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-h[63285]: 2024-04-24T09:27:49.473+0000 7fcfab490700 -1 mds.h *** got signal Terminated *** 2024-04-24T09:27:52.631 INFO:journalctl@ceph.mds.h.smithi114.stdout:Apr 24 09:27:52 smithi114 podman[68180]: 2024-04-24 09:27:52.280713536 +0000 UTC m=+2.901706159 container died 6c22bab096ce8940148f0dc73ce623c12f4ade40d35fe89ae67842365d427dbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-h, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T09:27:52.899 INFO:journalctl@ceph.mds.h.smithi114.stdout:Apr 24 09:27:52 smithi114 podman[68180]: 2024-04-24 09:27:52.706961285 +0000 UTC m=+3.327953897 container cleanup 6c22bab096ce8940148f0dc73ce623c12f4ade40d35fe89ae67842365d427dbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-h, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-24T09:27:52.899 INFO:journalctl@ceph.mds.h.smithi114.stdout:Apr 24 09:27:52 smithi114 bash[68180]: ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-h 2024-04-24T09:27:53.100 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mds.h.service' 2024-04-24T09:27:53.131 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:27:53.132 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-24T09:27:53.132 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-24T09:27:53.132 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mds.k 2024-04-24T09:27:53.443 INFO:journalctl@ceph.mds.k.smithi114.stdout:Apr 24 09:27:53 smithi114 systemd[1]: Stopping Ceph mds.k for 30ee9be2-021a-11ef-bc93-c7b262605968... 2024-04-24T09:27:53.926 INFO:journalctl@ceph.mds.k.smithi114.stdout:Apr 24 09:27:53 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-k[62652]: 2024-04-24T09:27:53.443+0000 7f4ff4d24700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:27:53.926 INFO:journalctl@ceph.mds.k.smithi114.stdout:Apr 24 09:27:53 smithi114 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-k[62652]: 2024-04-24T09:27:53.443+0000 7f4ff4d24700 -1 mds.k *** got signal Terminated *** 2024-04-24T09:27:56.635 INFO:journalctl@ceph.mds.k.smithi114.stdout:Apr 24 09:27:56 smithi114 podman[68299]: 2024-04-24 09:27:56.283089291 +0000 UTC m=+2.936035912 container died fd4362b3443be26aa087e3e1f4383fec34499c5da5da1a4e0a6d8a13f8244dce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-k, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-24T09:27:56.893 INFO:journalctl@ceph.mds.k.smithi114.stdout:Apr 24 09:27:56 smithi114 podman[68299]: 2024-04-24 09:27:56.71857611 +0000 UTC m=+3.371522730 container cleanup fd4362b3443be26aa087e3e1f4383fec34499c5da5da1a4e0a6d8a13f8244dce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-k, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424) 2024-04-24T09:27:56.894 INFO:journalctl@ceph.mds.k.smithi114.stdout:Apr 24 09:27:56 smithi114 bash[68299]: ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-k 2024-04-24T09:27:57.118 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mds.k.service' 2024-04-24T09:27:57.148 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:27:57.149 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-24T09:27:57.149 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-24T09:27:57.149 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mds.c 2024-04-24T09:27:57.457 INFO:journalctl@ceph.mds.c.smithi149.stdout:Apr 24 09:27:57 smithi149 systemd[1]: Stopping Ceph mds.c for 30ee9be2-021a-11ef-bc93-c7b262605968... 2024-04-24T09:27:57.736 INFO:journalctl@ceph.mds.c.smithi149.stdout:Apr 24 09:27:57 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-c[64043]: 2024-04-24T09:27:57.457+0000 7f4d83c88700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:27:57.736 INFO:journalctl@ceph.mds.c.smithi149.stdout:Apr 24 09:27:57 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-c[64043]: 2024-04-24T09:27:57.457+0000 7f4d83c88700 -1 mds.c *** got signal Terminated *** 2024-04-24T09:28:00.611 INFO:journalctl@ceph.mds.c.smithi149.stdout:Apr 24 09:28:00 smithi149 podman[68726]: 2024-04-24 09:28:00.298207724 +0000 UTC m=+2.954760887 container died b7df5ffb9489fe665be8064c33ca4a3fe4f3904f20b4df04667ad7a28fc1841c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-c, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-24T09:28:00.926 INFO:journalctl@ceph.mds.c.smithi149.stdout:Apr 24 09:28:00 smithi149 podman[68726]: 2024-04-24 09:28:00.698518689 +0000 UTC m=+3.355071890 container cleanup b7df5ffb9489fe665be8064c33ca4a3fe4f3904f20b4df04667ad7a28fc1841c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-24T09:28:00.927 INFO:journalctl@ceph.mds.c.smithi149.stdout:Apr 24 09:28:00 smithi149 bash[68726]: ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-c 2024-04-24T09:28:01.057 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mds.c.service' 2024-04-24T09:28:01.094 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:28:01.094 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-24T09:28:01.095 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-24T09:28:01.095 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mds.f 2024-04-24T09:28:01.611 INFO:journalctl@ceph.mds.f.smithi149.stdout:Apr 24 09:28:01 smithi149 systemd[1]: Stopping Ceph mds.f for 30ee9be2-021a-11ef-bc93-c7b262605968... 2024-04-24T09:28:01.611 INFO:journalctl@ceph.mds.f.smithi149.stdout:Apr 24 09:28:01 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-f[63729]: 2024-04-24T09:28:01.435+0000 7fbd5b2bc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:28:01.612 INFO:journalctl@ceph.mds.f.smithi149.stdout:Apr 24 09:28:01 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-f[63729]: 2024-04-24T09:28:01.435+0000 7fbd5b2bc700 -1 mds.f *** got signal Terminated *** 2024-04-24T09:28:04.577 INFO:journalctl@ceph.mds.f.smithi149.stdout:Apr 24 09:28:04 smithi149 podman[68856]: 2024-04-24 09:28:04.292543851 +0000 UTC m=+2.966471968 container died 01b2fff4c5a1bd5f71a128b84b2a96348e243b4c9bae4a1e1f6a16aefbdf9a39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-f, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T09:28:04.862 INFO:journalctl@ceph.mds.f.smithi149.stdout:Apr 24 09:28:04 smithi149 podman[68856]: 2024-04-24 09:28:04.643656571 +0000 UTC m=+3.317584682 container cleanup 01b2fff4c5a1bd5f71a128b84b2a96348e243b4c9bae4a1e1f6a16aefbdf9a39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-f, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T09:28:04.862 INFO:journalctl@ceph.mds.f.smithi149.stdout:Apr 24 09:28:04 smithi149 bash[68856]: ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-f 2024-04-24T09:28:05.061 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mds.f.service' 2024-04-24T09:28:05.094 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:28:05.094 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-24T09:28:05.095 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-24T09:28:05.095 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mds.i 2024-04-24T09:28:05.361 INFO:journalctl@ceph.mds.i.smithi149.stdout:Apr 24 09:28:05 smithi149 systemd[1]: Stopping Ceph mds.i for 30ee9be2-021a-11ef-bc93-c7b262605968... 2024-04-24T09:28:05.861 INFO:journalctl@ceph.mds.i.smithi149.stdout:Apr 24 09:28:05 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-i[64347]: 2024-04-24T09:28:05.422+0000 7fa6a27de700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:28:05.862 INFO:journalctl@ceph.mds.i.smithi149.stdout:Apr 24 09:28:05 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-i[64347]: 2024-04-24T09:28:05.422+0000 7fa6a27de700 -1 mds.i *** got signal Terminated *** 2024-04-24T09:28:08.561 INFO:journalctl@ceph.mds.i.smithi149.stdout:Apr 24 09:28:08 smithi149 podman[68978]: 2024-04-24 09:28:08.281614218 +0000 UTC m=+2.976656121 container died 35e767d0635deb1489a42c375b43025d9f0b3b5e93ccb56f237120dcdf16885a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-i, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424) 2024-04-24T09:28:08.861 INFO:journalctl@ceph.mds.i.smithi149.stdout:Apr 24 09:28:08 smithi149 podman[68978]: 2024-04-24 09:28:08.697039672 +0000 UTC m=+3.392081582 container cleanup 35e767d0635deb1489a42c375b43025d9f0b3b5e93ccb56f237120dcdf16885a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-i, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-24T09:28:08.862 INFO:journalctl@ceph.mds.i.smithi149.stdout:Apr 24 09:28:08 smithi149 bash[68978]: ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-i 2024-04-24T09:28:09.043 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mds.i.service' 2024-04-24T09:28:09.075 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:28:09.075 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-24T09:28:09.076 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-24T09:28:09.076 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mds.l 2024-04-24T09:28:09.611 INFO:journalctl@ceph.mds.l.smithi149.stdout:Apr 24 09:28:09 smithi149 systemd[1]: Stopping Ceph mds.l for 30ee9be2-021a-11ef-bc93-c7b262605968... 2024-04-24T09:28:09.612 INFO:journalctl@ceph.mds.l.smithi149.stdout:Apr 24 09:28:09 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-l[63416]: 2024-04-24T09:28:09.391+0000 7f2563a47700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:28:09.612 INFO:journalctl@ceph.mds.l.smithi149.stdout:Apr 24 09:28:09 smithi149 ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-l[63416]: 2024-04-24T09:28:09.391+0000 7f2563a47700 -1 mds.l *** got signal Terminated *** 2024-04-24T09:28:12.611 INFO:journalctl@ceph.mds.l.smithi149.stdout:Apr 24 09:28:12 smithi149 podman[69098]: 2024-04-24 09:28:12.281423876 +0000 UTC m=+2.991831832 container died 454120a5fc359ba6d9b8fb1954f7b994e76bfd7d1eba6c109c514261cef3838b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-l, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD) 2024-04-24T09:28:13.111 INFO:journalctl@ceph.mds.l.smithi149.stdout:Apr 24 09:28:12 smithi149 podman[69098]: 2024-04-24 09:28:12.825785741 +0000 UTC m=+3.536193705 container cleanup 454120a5fc359ba6d9b8fb1954f7b994e76bfd7d1eba6c109c514261cef3838b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-l, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-24T09:28:13.112 INFO:journalctl@ceph.mds.l.smithi149.stdout:Apr 24 09:28:12 smithi149 bash[69098]: ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-l 2024-04-24T09:28:13.112 INFO:journalctl@ceph.mds.l.smithi149.stdout:Apr 24 09:28:12 smithi149 podman[69112]: 2024-04-24 09:28:12.98510534 +0000 UTC m=+0.702759746 container remove 454120a5fc359ba6d9b8fb1954f7b994e76bfd7d1eba6c109c514261cef3838b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-30ee9be2-021a-11ef-bc93-c7b262605968-mds-l, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True) 2024-04-24T09:28:13.205 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-30ee9be2-021a-11ef-bc93-c7b262605968@mds.l.service' 2024-04-24T09:28:13.237 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:28:13.237 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-24T09:28:13.237 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-24T09:28:13.557 INFO:teuthology.orchestra.run.smithi090.stdout:Deleting cluster with fsid: 30ee9be2-021a-11ef-bc93-c7b262605968 2024-04-24T09:28:18.542 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-24T09:28:18.729 INFO:teuthology.orchestra.run.smithi114.stdout:Deleting cluster with fsid: 30ee9be2-021a-11ef-bc93-c7b262605968 2024-04-24T09:28:21.107 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-24T09:28:21.295 INFO:teuthology.orchestra.run.smithi149.stdout:Deleting cluster with fsid: 30ee9be2-021a-11ef-bc93-c7b262605968 2024-04-24T09:28:23.674 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T09:28:23.767 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T09:28:23.798 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T09:28:23.824 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-24T09:28:23.826 DEBUG:teuthology.misc:Transferring archived files from smithi090:/var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671225/remote/smithi090/crash 2024-04-24T09:28:23.827 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar c -f - -C /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/crash -- . 2024-04-24T09:28:23.932 INFO:teuthology.orchestra.run.smithi090.stderr:tar: /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-24T09:28:23.933 INFO:teuthology.orchestra.run.smithi090.stderr:tar: Error is not recoverable: exiting now 2024-04-24T09:28:23.937 DEBUG:teuthology.misc:Transferring archived files from smithi114:/var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671225/remote/smithi114/crash 2024-04-24T09:28:23.938 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar c -f - -C /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/crash -- . 2024-04-24T09:28:23.979 INFO:teuthology.orchestra.run.smithi114.stderr:tar: /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-24T09:28:23.979 INFO:teuthology.orchestra.run.smithi114.stderr:tar: Error is not recoverable: exiting now 2024-04-24T09:28:23.982 DEBUG:teuthology.misc:Transferring archived files from smithi149:/var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671225/remote/smithi149/crash 2024-04-24T09:28:23.983 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/crash -- . 2024-04-24T09:28:24.021 INFO:teuthology.orchestra.run.smithi149.stderr:tar: /var/lib/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-24T09:28:24.021 INFO:teuthology.orchestra.run.smithi149.stderr:tar: Error is not recoverable: exiting now 2024-04-24T09:28:24.022 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-24T09:28:24.022 DEBUG:teuthology.orchestra.run.smithi090:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-24T09:28:24.137 INFO:tasks.cephadm:Compressing logs... 2024-04-24T09:28:24.137 DEBUG:teuthology.orchestra.run.smithi090:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-24T09:28:24.142 DEBUG:teuthology.orchestra.run.smithi114:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-24T09:28:24.146 DEBUG:teuthology.orchestra.run.smithi149:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-24T09:28:24.174 INFO:teuthology.orchestra.run.smithi149.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose: No such file or directory -- 2024-04-24T09:28:24.174 INFO:teuthology.orchestra.run.smithi149.stderr: /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph.log 2024-04-24T09:28:24.174 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-24T09:28:24.174 INFO:teuthology.orchestra.run.smithi149.stderr:gzip/var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph.log: -5 --verbose -- /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-24T09:28:24.175 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-24T09:28:24.175 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-24T09:28:24.175 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-24T09:28:24.175 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-24T09:28:24.175 INFO:teuthology.orchestra.run.smithi149.stderr: 86.7%/var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -- replaced with /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-24T09:28:24.175 INFO:teuthology.orchestra.run.smithi149.stderr: -5 --verbose -- /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-24T09:28:24.175 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-24T09:28:24.175 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-24T09:28:24.177 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mds.i.log: 90.4% -- replaced with /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-24T09:28:24.181 INFO:teuthology.orchestra.run.smithi149.stderr: 92.5% -- replaced with /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-24T09:28:24.183 INFO:teuthology.orchestra.run.smithi114.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph.log 2024-04-24T09:28:24.183 INFO:teuthology.orchestra.run.smithi114.stderr:: No such file or directory 2024-04-24T09:28:24.183 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-24T09:28:24.183 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-24T09:28:24.183 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-24T09:28:24.183 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-24T09:28:24.183 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-osd.7.log: /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-24T09:28:24.183 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-24T09:28:24.184 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-24T09:28:24.184 INFO:teuthology.orchestra.run.smithi114.stderr: 86.6% -- replaced with /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-24T09:28:24.184 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mgr.y.log: 90.3% -- replaced with /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-24T09:28:24.184 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-24T09:28:24.184 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose/var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mds.h.log: -- /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-24T09:28:24.184 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-24T09:28:24.184 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph.cephadm.log: 82.6% -- replaced with /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-24T09:28:24.184 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-24T09:28:24.185 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mds.e.log: 92.4% -- replaced with /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-24T09:28:24.185 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-24T09:28:24.189 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-24T09:28:24.189 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-24T09:28:24.189 INFO:teuthology.orchestra.run.smithi149.stderr: 82.6% -- replaced with /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-24T09:28:24.189 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-24T09:28:24.194 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-volume.log: 91.0% -- replaced with /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-24T09:28:24.195 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-24T09:28:24.200 INFO:teuthology.orchestra.run.smithi090.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-24T09:28:24.200 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph.log 2024-04-24T09:28:24.201 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-24T09:28:24.202 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-24T09:28:24.203 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-24T09:28:24.204 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-24T09:28:24.204 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-24T09:28:24.205 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mds.j.log: 86.7% -- replaced with /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-24T09:28:24.205 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-24T09:28:24.206 INFO:teuthology.orchestra.run.smithi090.stderr: 90.2% -- replaced with /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-24T09:28:24.207 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-24T09:28:24.207 INFO:teuthology.orchestra.run.smithi090.stderr: 94.5% -- replaced with /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-24T09:28:24.208 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-24T09:28:24.210 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-24T09:28:24.211 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-24T09:28:24.212 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-24T09:28:24.212 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-24T09:28:24.214 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.0% -- replaced with /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-24T09:28:24.215 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-24T09:28:24.217 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mds.a.log: /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-24T09:28:24.218 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-24T09:28:24.224 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-24T09:28:24.225 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-24T09:28:24.226 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-24T09:28:24.227 INFO:teuthology.orchestra.run.smithi114.stderr: 93.9% -- replaced with /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-24T09:28:24.233 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-osd.9.log: /var/log/ceph/cephadm.log: 94.6% -- replaced with /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-24T09:28:24.238 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 91.0% -- replaced with /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-24T09:28:24.240 INFO:teuthology.orchestra.run.smithi114.stderr: 93.2% -- replaced with /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-24T09:28:24.242 INFO:teuthology.orchestra.run.smithi090.stderr: 90.9% -- replaced with /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-24T09:28:24.247 INFO:teuthology.orchestra.run.smithi149.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-24T09:28:24.250 INFO:teuthology.orchestra.run.smithi090.stderr: 93.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-24T09:28:24.258 INFO:teuthology.orchestra.run.smithi114.stderr: 94.0% -- replaced with /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-24T09:28:24.292 INFO:teuthology.orchestra.run.smithi090.stderr: 92.9% -- replaced with /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-24T09:28:24.319 INFO:teuthology.orchestra.run.smithi149.stderr: 93.4% -- replaced with /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-24T09:28:24.320 INFO:teuthology.orchestra.run.smithi149.stderr: 93.3% -- replaced with /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-24T09:28:24.324 INFO:teuthology.orchestra.run.smithi149.stderr: 93.6% -- replaced with /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-24T09:28:24.326 INFO:teuthology.orchestra.run.smithi090.stderr: 94.3% -- replaced with /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-24T09:28:24.435 INFO:teuthology.orchestra.run.smithi114.stderr: 91.8% -- replaced with /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-24T09:28:24.448 INFO:teuthology.orchestra.run.smithi090.stderr: 90.0% -- replaced with /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-24T09:28:24.494 INFO:teuthology.orchestra.run.smithi149.stderr: 91.8% -- replaced with /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-24T09:28:24.864 INFO:teuthology.orchestra.run.smithi090.stderr: 90.5% -- replaced with /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-24T09:28:25.240 INFO:teuthology.orchestra.run.smithi149.stderr: 95.4% -- replaced with /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-24T09:28:25.360 INFO:teuthology.orchestra.run.smithi090.stderr: 91.8% -- replaced with /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-24T09:28:25.608 INFO:teuthology.orchestra.run.smithi090.stderr: 91.8% -- replaced with /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-24T09:28:25.639 INFO:teuthology.orchestra.run.smithi114.stderr: 92.2% -- replaced with /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-24T09:28:25.651 INFO:teuthology.orchestra.run.smithi149.stderr: 92.6% -- replaced with /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-24T09:28:25.678 INFO:teuthology.orchestra.run.smithi149.stderr: 91.8% -- replaced with /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-24T09:28:25.701 INFO:teuthology.orchestra.run.smithi090.stderr: 91.8% -- replaced with /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-24T09:28:25.816 INFO:teuthology.orchestra.run.smithi149.stderr: 92.6% -- replaced with /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-24T09:28:25.857 INFO:teuthology.orchestra.run.smithi090.stderr: 92.7% -- replaced with /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-24T09:28:25.861 INFO:teuthology.orchestra.run.smithi090.stderr: 2024-04-24T09:28:25.861 INFO:teuthology.orchestra.run.smithi090.stderr:real 0m1.687s 2024-04-24T09:28:25.861 INFO:teuthology.orchestra.run.smithi090.stderr:user 0m6.297s 2024-04-24T09:28:25.862 INFO:teuthology.orchestra.run.smithi090.stderr:sys 0m0.570s 2024-04-24T09:28:25.909 INFO:teuthology.orchestra.run.smithi114.stderr: 92.5% -- replaced with /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-24T09:28:25.955 INFO:teuthology.orchestra.run.smithi114.stderr: 92.4% -- replaced with /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-24T09:28:26.165 INFO:teuthology.orchestra.run.smithi149.stderr: 92.5% -- replaced with /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-24T09:28:26.168 INFO:teuthology.orchestra.run.smithi149.stderr: 2024-04-24T09:28:26.168 INFO:teuthology.orchestra.run.smithi149.stderr:real 0m2.006s 2024-04-24T09:28:26.168 INFO:teuthology.orchestra.run.smithi149.stderr:user 0m8.019s 2024-04-24T09:28:26.169 INFO:teuthology.orchestra.run.smithi149.stderr:sys 0m0.281s 2024-04-24T09:28:26.246 INFO:teuthology.orchestra.run.smithi114.stderr: 92.6% -- replaced with /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-24T09:28:37.418 INFO:teuthology.orchestra.run.smithi114.stderr: 93.3% -- replaced with /var/log/ceph/30ee9be2-021a-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-24T09:28:37.420 INFO:teuthology.orchestra.run.smithi114.stderr: 2024-04-24T09:28:37.420 INFO:teuthology.orchestra.run.smithi114.stderr:real 0m13.260s 2024-04-24T09:28:37.420 INFO:teuthology.orchestra.run.smithi114.stderr:user 0m20.034s 2024-04-24T09:28:37.420 INFO:teuthology.orchestra.run.smithi114.stderr:sys 0m0.673s 2024-04-24T09:28:37.421 INFO:tasks.cephadm:Archiving logs... 2024-04-24T09:28:37.421 DEBUG:teuthology.misc:Transferring archived files from smithi090:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671225/remote/smithi090/log 2024-04-24T09:28:37.422 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-24T09:28:38.470 DEBUG:teuthology.misc:Transferring archived files from smithi114:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671225/remote/smithi114/log 2024-04-24T09:28:38.471 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-24T09:28:41.490 DEBUG:teuthology.misc:Transferring archived files from smithi149:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671225/remote/smithi149/log 2024-04-24T09:28:41.491 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-24T09:28:42.697 INFO:tasks.cephadm:Removing cluster... 2024-04-24T09:28:42.697 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 --force 2024-04-24T09:28:43.016 INFO:teuthology.orchestra.run.smithi090.stdout:Deleting cluster with fsid: 30ee9be2-021a-11ef-bc93-c7b262605968 2024-04-24T09:28:43.641 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 --force 2024-04-24T09:28:43.831 INFO:teuthology.orchestra.run.smithi114.stdout:Deleting cluster with fsid: 30ee9be2-021a-11ef-bc93-c7b262605968 2024-04-24T09:28:44.103 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 30ee9be2-021a-11ef-bc93-c7b262605968 --force 2024-04-24T09:28:44.298 INFO:teuthology.orchestra.run.smithi149.stdout:Deleting cluster with fsid: 30ee9be2-021a-11ef-bc93-c7b262605968 2024-04-24T09:28:44.562 INFO:tasks.cephadm:Removing cephadm ... 2024-04-24T09:28:44.563 DEBUG:teuthology.orchestra.run.smithi090:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-24T09:28:44.606 DEBUG:teuthology.orchestra.run.smithi114:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-24T09:28:44.623 DEBUG:teuthology.orchestra.run.smithi149:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-24T09:28:44.639 INFO:tasks.cephadm:Teardown complete 2024-04-24T09:28:44.640 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-24T09:28:44.651 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-24T09:28:44.651 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-24T09:28:44.657 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-24T09:28:44.666 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-24T09:28:44.801 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-24T09:28:44.811 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-24T09:28:44.811 DEBUG:teuthology.orchestra.run.smithi090:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T09:28:44.819 DEBUG:teuthology.orchestra.run.smithi114:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T09:28:44.821 DEBUG:teuthology.orchestra.run.smithi149:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T09:28:44.834 INFO:teuthology.orchestra.run.smithi114.stderr:bash: line 1: ntpq: command not found 2024-04-24T09:28:44.835 INFO:teuthology.orchestra.run.smithi149.stderr:bash: line 1: ntpq: command not found 2024-04-24T09:28:44.838 INFO:teuthology.orchestra.run.smithi114.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T09:28:44.839 INFO:teuthology.orchestra.run.smithi114.stdout:=============================================================================== 2024-04-24T09:28:44.839 INFO:teuthology.orchestra.run.smithi114.stdout:^- hv01.front.sepia.ceph.com 2 6 377 6 +1667us[+1674us] +/- 573ms 2024-04-24T09:28:44.839 INFO:teuthology.orchestra.run.smithi114.stdout:^* hv02.front.sepia.ceph.com 3 6 377 6 -344us[ -337us] +/- 54ms 2024-04-24T09:28:44.839 INFO:teuthology.orchestra.run.smithi114.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 5 +398us[ +398us] +/- 66ms 2024-04-24T09:28:44.839 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T09:28:44.840 INFO:teuthology.orchestra.run.smithi149.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T09:28:44.841 INFO:teuthology.orchestra.run.smithi149.stdout:=============================================================================== 2024-04-24T09:28:44.841 INFO:teuthology.orchestra.run.smithi149.stdout:^- hv01.front.sepia.ceph.com 2 6 377 7 +1663us[+1678us] +/- 573ms 2024-04-24T09:28:44.841 INFO:teuthology.orchestra.run.smithi149.stdout:^* hv02.front.sepia.ceph.com 3 6 377 6 -336us[ -321us] +/- 54ms 2024-04-24T09:28:44.841 INFO:teuthology.orchestra.run.smithi149.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 5 +405us[ +405us] +/- 67ms 2024-04-24T09:28:44.841 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T09:28:44.860 INFO:teuthology.orchestra.run.smithi090.stderr:bash: line 1: ntpq: command not found 2024-04-24T09:28:44.868 INFO:teuthology.orchestra.run.smithi090.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T09:28:44.869 INFO:teuthology.orchestra.run.smithi090.stdout:=============================================================================== 2024-04-24T09:28:44.869 INFO:teuthology.orchestra.run.smithi090.stdout:^- hv01.front.sepia.ceph.com 2 7 377 3 +1721us[+1721us] +/- 573ms 2024-04-24T09:28:44.869 INFO:teuthology.orchestra.run.smithi090.stdout:^* hv02.front.sepia.ceph.com 3 6 377 5 -324us[ -312us] +/- 54ms 2024-04-24T09:28:44.869 INFO:teuthology.orchestra.run.smithi090.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 5 +412us[ +425us] +/- 67ms 2024-04-24T09:28:44.869 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T09:28:44.870 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-24T09:28:44.885 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-24T09:28:44.885 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-24T09:28:44.896 DEBUG:teuthology.orchestra.run.smithi090:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-24T09:28:45.087 DEBUG:teuthology.orchestra.run.smithi114:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-24T09:28:45.157 DEBUG:teuthology.orchestra.run.smithi149:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-24T09:28:45.222 DEBUG:teuthology.orchestra.run.smithi090:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T09:28:45.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T09:28:45.311 DEBUG:teuthology.orchestra.run.smithi114:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T09:28:45.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T09:28:45.341 DEBUG:teuthology.orchestra.run.smithi149:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T09:28:45.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T09:28:45.368 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-24T09:28:45.380 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-24T09:28:45.392 INFO:teuthology.task.internal:Duration was 1680.922028 seconds 2024-04-24T09:28:45.393 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-24T09:28:45.405 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-24T09:28:45.405 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-24T09:28:45.414 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-24T09:28:45.417 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-24T09:28:45.454 INFO:teuthology.orchestra.run.smithi149.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T09:28:45.455 INFO:teuthology.orchestra.run.smithi114.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T09:28:45.537 INFO:teuthology.orchestra.run.smithi090.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T09:28:45.970 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-24T09:28:45.970 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi090.front.sepia.ceph.com 2024-04-24T09:28:45.971 DEBUG:teuthology.orchestra.run.smithi090:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-24T09:28:46.043 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi114.front.sepia.ceph.com 2024-04-24T09:28:46.044 DEBUG:teuthology.orchestra.run.smithi114:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-24T09:28:46.069 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi149.front.sepia.ceph.com 2024-04-24T09:28:46.070 DEBUG:teuthology.orchestra.run.smithi149:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-24T09:28:46.100 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-24T09:28:46.101 DEBUG:teuthology.orchestra.run.smithi090:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-24T09:28:46.108 DEBUG:teuthology.orchestra.run.smithi114:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-24T09:28:46.112 DEBUG:teuthology.orchestra.run.smithi149:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-24T09:28:46.192 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-24T09:28:46.192 DEBUG:teuthology.orchestra.run.smithi090:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-24T09:28:46.197 DEBUG:teuthology.orchestra.run.smithi114:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-24T09:28:46.199 DEBUG:teuthology.orchestra.run.smithi149:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-24T09:28:46.688 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-24T09:28:46.700 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-24T09:28:46.701 DEBUG:teuthology.orchestra.run.smithi090:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-24T09:28:46.784 DEBUG:teuthology.orchestra.run.smithi114:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-24T09:28:46.809 DEBUG:teuthology.orchestra.run.smithi149:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-24T09:28:46.834 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-24T09:28:46.845 DEBUG:teuthology.orchestra.run.smithi090:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-24T09:28:46.854 DEBUG:teuthology.orchestra.run.smithi114:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-24T09:28:46.856 DEBUG:teuthology.orchestra.run.smithi149:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-24T09:28:46.876 INFO:teuthology.orchestra.run.smithi114.stdout:kernel.core_pattern = core 2024-04-24T09:28:46.897 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern = core 2024-04-24T09:28:46.942 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern = core 2024-04-24T09:28:46.982 DEBUG:teuthology.orchestra.run.smithi090:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-24T09:28:47.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T09:28:47.038 DEBUG:teuthology.orchestra.run.smithi114:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-24T09:28:47.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T09:28:47.052 DEBUG:teuthology.orchestra.run.smithi149:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-24T09:28:47.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T09:28:47.067 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-24T09:28:47.078 INFO:teuthology.task.internal:Transferring archived files... 2024-04-24T09:28:47.079 DEBUG:teuthology.misc:Transferring archived files from smithi090:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671225/remote/smithi090 2024-04-24T09:28:47.079 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-24T09:28:47.193 DEBUG:teuthology.misc:Transferring archived files from smithi114:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671225/remote/smithi114 2024-04-24T09:28:47.194 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-24T09:28:47.235 DEBUG:teuthology.misc:Transferring archived files from smithi149:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671225/remote/smithi149 2024-04-24T09:28:47.235 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-24T09:28:47.272 INFO:teuthology.task.internal:Removing archive directory... 2024-04-24T09:28:47.273 DEBUG:teuthology.orchestra.run.smithi090:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-24T09:28:47.281 DEBUG:teuthology.orchestra.run.smithi114:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-24T09:28:47.283 DEBUG:teuthology.orchestra.run.smithi149:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-24T09:28:47.329 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-24T09:28:47.342 INFO:teuthology.task.internal:Not uploading archives. 2024-04-24T09:28:47.342 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-24T09:28:47.357 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-24T09:28:47.357 DEBUG:teuthology.orchestra.run.smithi090:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-24T09:28:47.363 DEBUG:teuthology.orchestra.run.smithi114:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-24T09:28:47.365 DEBUG:teuthology.orchestra.run.smithi149:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-24T09:28:47.378 INFO:teuthology.orchestra.run.smithi114.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 24 09:28 /home/ubuntu/cephtest 2024-04-24T09:28:47.379 INFO:teuthology.orchestra.run.smithi149.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 24 09:28 /home/ubuntu/cephtest 2024-04-24T09:28:47.411 INFO:teuthology.orchestra.run.smithi090.stdout: 393251 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 24 09:28 /home/ubuntu/cephtest 2024-04-24T09:28:47.415 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-24T09:28:47.655 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-24T09:28:47.714 INFO:teuthology.nuke:Checking targets against current locks 2024-04-24T09:28:47.740 DEBUG:teuthology.nuke:shortname: smithi090 2024-04-24T09:28:47.740 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-24T09:28:47.764 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi090.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671225', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 08:50:44.519274', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK1UBMLD+nRVl0oMMWDQf2IwkCKv5zuhPfJbc1j1JbUDzNNDUJjbducopq88qDIxujCiscRPaSnx8IN4vtqh3/M='} 2024-04-24T09:28:47.770 DEBUG:teuthology.nuke:shortname: smithi114 2024-04-24T09:28:47.770 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-24T09:28:47.792 DEBUG:teuthology.nuke:shortname: smithi149 2024-04-24T09:28:47.792 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-24T09:28:47.795 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi114.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671225', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 08:50:44.515765', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH98UXW7tWWhJR9vA8aAMTU0MvISAGp74/oo/ZsBqHkECDNHSaUieKk9zatMsSgsDznu90xl8L2kToTnbF6DWIM='} 2024-04-24T09:28:47.818 INFO:teuthology.orchestra.console.smithi090:Power off 2024-04-24T09:28:47.819 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-24T09:28:47.846 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi149.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671225', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 08:50:44.517670', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB3WggCYnS3qkyggvta3gVJxUZVLDSdh1fIUvPO8pAGA+AMRKJwQy6xS1kxpVX6ZvddQQJxCECFayXG71dns0+M='} 2024-04-24T09:28:47.857 DEBUG:teuthology.orchestra.console.smithi090:power off output: Chassis Power Control: Down/Off 2024-04-24T09:28:47.857 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T09:28:47.881 INFO:teuthology.orchestra.console.smithi114:Power off 2024-04-24T09:28:47.882 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-24T09:28:47.896 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is on 2024-04-24T09:28:47.907 DEBUG:teuthology.orchestra.console.smithi114:power off output: Chassis Power Control: Down/Off 2024-04-24T09:28:47.908 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T09:28:47.927 INFO:teuthology.orchestra.console.smithi149:Power off 2024-04-24T09:28:47.928 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-24T09:28:47.937 DEBUG:teuthology.orchestra.console.smithi114:check power output: Chassis Power is on 2024-04-24T09:28:47.950 DEBUG:teuthology.orchestra.console.smithi149:power off output: Chassis Power Control: Down/Off 2024-04-24T09:28:47.951 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T09:28:47.975 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is on 2024-04-24T09:28:51.897 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T09:28:51.939 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T09:28:52.290 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T09:28:52.308 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is on 2024-04-24T09:28:52.392 DEBUG:teuthology.orchestra.console.smithi114:check power output: Chassis Power is on 2024-04-24T09:28:52.408 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is on 2024-04-24T09:28:56.311 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T09:28:56.393 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T09:28:56.409 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T09:28:56.423 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is on 2024-04-24T09:28:56.507 DEBUG:teuthology.orchestra.console.smithi114:check power output: Chassis Power is off 2024-04-24T09:28:56.519 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is off 2024-04-24T09:28:56.609 INFO:teuthology.orchestra.console.smithi114:Power off completed 2024-04-24T09:28:56.620 INFO:teuthology.orchestra.console.smithi149:Power off completed 2024-04-24T09:29:00.426 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T09:29:00.539 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is off 2024-04-24T09:29:00.640 INFO:teuthology.orchestra.console.smithi090:Power off completed 2024-04-24T09:29:00.748 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/suites/fsx}} duration: 1680.9220278263092 failure_reason: 'Command failed (workunit test suites/fsx.sh) on smithi090 with status 2: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1f81fb397ae98da7563d451a78c61574c8f4e6e0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh''' flavor: default owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=8fbee611dfa140be809da4c148483a7b status: fail success: false 2024-04-24T09:29:00.748 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-24T09:29:00.843 INFO:teuthology.run:FAIL